You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: hypothesis-python/docs/changelog.rst
+50Lines changed: 50 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,6 +18,56 @@ Hypothesis 6.x
18
18
19
19
.. include:: ../RELEASE.rst
20
20
21
+
.. _v6.138.7:
22
+
23
+
--------------------
24
+
6.138.7 - 2025-08-28
25
+
--------------------
26
+
27
+
Improves upon the cache eviction problem workaround
28
+
of :v:`6.135.12`.
29
+
30
+
.. _v6.138.6:
31
+
32
+
--------------------
33
+
6.138.6 - 2025-08-27
34
+
--------------------
35
+
36
+
Documentation tweaks.
37
+
38
+
.. _v6.138.5:
39
+
40
+
--------------------
41
+
6.138.5 - 2025-08-27
42
+
--------------------
43
+
44
+
Fixes a race condition under threading for strategies which trigger our filter-rewriting rules, like ``st.integers().filter(lambda x: abs(x) > 100)``.
45
+
46
+
.. _v6.138.4:
47
+
48
+
--------------------
49
+
6.138.4 - 2025-08-27
50
+
--------------------
51
+
52
+
One of our shrinking passes for reducing failing inputs targets failures which require two numbers to add to the same value. This pass previously only worked for positive numbers. This patch fixes that, so it also works for negative numbers.
53
+
54
+
.. _v6.138.3:
55
+
56
+
--------------------
57
+
6.138.3 - 2025-08-24
58
+
--------------------
59
+
60
+
This patch slightly improves the cache-hit rate for
61
+
|st.dictionaries| and certain unique |st.lists|.
62
+
63
+
.. _v6.138.2:
64
+
65
+
--------------------
66
+
6.138.2 - 2025-08-16
67
+
--------------------
68
+
69
+
The type annotations for |st.register_type_strategy| now indicate that it accepts registering types created with |TypeAliasType| (aka ``type MyType = int``).
Copy file name to clipboardExpand all lines: hypothesis-python/docs/compatibility.rst
+71-60Lines changed: 71 additions & 60 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,7 @@
1
1
Compatibility
2
2
=============
3
3
4
-
Hypothesis does its level best to be compatible with everything you could
5
-
possibly need it to be compatible with. Generally you should just try it and
6
-
expect it to work. If it doesn't, you can be surprised and check this document
7
-
for the details.
4
+
Hypothesis generally does its level best to be compatible with everything you could need it to be compatible with. This document outlines our compatibility status and guarantees.
8
5
9
6
Hypothesis versions
10
7
-------------------
@@ -14,7 +11,7 @@ Backwards compatibility is better than backporting fixes, so we use
14
11
version of Hypothesis.
15
12
16
13
Documented APIs will not break except between major version bumps.
17
-
All APIs mentioned in this documentation are public unless explicitly
14
+
All APIs mentioned in the Hypothesis documentation are public unless explicitly
18
15
noted as provisional, in which case they may be changed in minor releases.
19
16
Undocumented attributes, modules, and behaviour may include breaking
20
17
changes in patch releases.
@@ -25,80 +22,94 @@ changes in patch releases.
25
22
Deprecations
26
23
------------
27
24
28
-
Deprecated features will emit warnings for at least six
29
-
months, and then be removed in the following major release.
25
+
Deprecated features will emit |HypothesisDeprecationWarning| for at least six months, and then be removed in the following major release.
30
26
31
-
Note however that not all warnings are subject to this grace period;
32
-
sometimes we strengthen validation by adding a warning and these may
33
-
become errors immediately at a major release.
34
-
35
-
We use custom exception and warning types, so you can see
36
-
exactly where an error came from, or turn only our warnings into errors.
Note however that not all warnings are subject to this grace period; sometimes we strengthen validation by adding a warning, and these may become errors immediately at a major release.
39
28
29
+
We use custom exception and warning types, so you can see exactly where an error came from, or turn only our warnings into errors.
40
30
41
31
Python versions
42
32
---------------
43
33
44
-
Hypothesis is supported and tested on CPython 3.9+, i.e.
45
-
`all versions of CPython with upstream support <https://devguide.python.org/versions/>`_,
46
-
along with PyPy for the same versions.
34
+
Hypothesis is supported and tested on CPython and PyPy 3.9+, i.e. all Python versions `that are still supported <https://devguide.python.org/versions/>`_.
47
35
32-bit builds of CPython also work, though we only test them on Windows.
48
36
49
-
In general Hypothesis does not officially support anything except the latest
50
-
patch release of any version of Python it supports. Earlier releases should work
51
-
and bugs in them will get fixed if reported, but they're not tested in CI and
52
-
no guarantees are made.
37
+
Hypothesis does not officially support anything except the latest patch release of each supported Python version. We will fix bugs in earlier patch releases if reported, but they're not tested in CI and no guarantees are made.
53
38
54
39
Operating systems
55
40
-----------------
56
41
57
-
In theory Hypothesis should work anywhere that Python does. In practice it is
58
-
only known to work and regularly tested on OS X, Windows and Linux, and you may
59
-
experience issues running it elsewhere.
42
+
In theory, Hypothesis should work anywhere that Python does. In practice, it is
43
+
known to work and regularly tested on macOS, Windows, Linux, and `Emscripten <https://peps.python.org/pep-0776/>`_.
60
44
61
-
If you're using something else and it doesn't work, do get in touch and I'll try
62
-
to help, but unless you can come up with a way for me to run a CI server on that
63
-
operating system it probably won't stay fixed due to the inevitable march of time.
45
+
If you experience issues running Hypothesis on other operating systems, we are
46
+
happy to accept bug reports which either clearly point to the problem or contain
47
+
reproducing instructions for a Hypothesis maintainer who does not have the ability
48
+
to run that OS. It's hard to fix something we can't reproduce!
64
49
65
50
.. _framework-compatibility:
66
51
67
52
Testing frameworks
68
53
------------------
69
54
70
-
In general Hypothesis goes to quite a lot of effort to generate things that
71
-
look like normal Python test functions that behave as closely to the originals
72
-
as possible, so it should work sensibly out of the box with every test framework.
73
-
74
-
If your testing relies on doing something other than calling a function and seeing
75
-
if it raises an exception then it probably *won't* work out of the box. In particular
76
-
things like tests which return generators and expect you to do something with them
77
-
(e.g. nose's yield based tests) will not work. Use a decorator or similar to wrap the
78
-
test to take this form, or ask the framework maintainer to support our
79
-
:ref:`hooks for inserting such a wrapper later <custom-function-execution>`.
80
-
81
-
In terms of what's actually *known* to work:
82
-
83
-
* Hypothesis integrates as smoothly with pytest and unittest as we can make it,
84
-
and this is verified as part of the CI.
85
-
* :pypi:`pytest` fixtures work in the usual way for tests that have been decorated
86
-
with |@given| - just avoid passing a strategy for
87
-
each argument that will be supplied by a fixture. However, function-scoped fixtures
88
-
will run only once for the whole function, not per example. To proactively warn you about
89
-
this case, we raise |HealthCheck.function_scoped_fixture|, unless suppressed with
90
-
|settings.suppress_health_check|.
91
-
* The :func:`python:unittest.mock.patch` decorator works with
92
-
|@given|, but we recommend using it as a context
93
-
manager within the decorated test to ensure that the mock is per-test-case
94
-
and avoid poor interactions with Pytest fixtures.
95
-
* Nose works fine with Hypothesis, and this is tested as part of the CI. ``yield`` based
96
-
tests simply won't work.
97
-
* Integration with Django's testing requires use of the :ref:`hypothesis-django` extra.
98
-
The issue is that in Django's tests' normal mode of execution it will reset the
99
-
database once per test rather than once per example, which is not what you want.
100
-
* :pypi:`coverage` works out of the box with Hypothesis; our own test suite has
101
-
100% branch coverage.
55
+
In general, Hypothesis goes to quite a lot of effort to return a function from |@given| that behaves as closely to a normal test function as possible. This means that most things should work sensibly with most testing frameworks.
56
+
57
+
Maintainers of testing frameworks may be interested in our support for :ref:`custom function execution <custom-function-execution>`, which may make some Hypothesis interactions possible to support.
58
+
59
+
pytest
60
+
~~~~~~
61
+
62
+
The main interaction to be aware of between Hypothesis and :pypi:`pytest` is fixtures.
63
+
64
+
pytest fixtures are automatically passed to |@given| tests, as usual. Note that |@given| supplies parameters from the right, so tests which use a fixture should either be written with the fixture placed first, or with keyword arguments:
65
+
66
+
.. code-block:: python
67
+
68
+
@given(st.integers())
69
+
deftest_use_fixture(myfixture, n):
70
+
pass
71
+
72
+
@given(n=st.integers())
73
+
deftest_use_fixture(n, myfixture):
74
+
pass
75
+
76
+
However, function-scoped fixtures run only once for the entire test, not per-input. This can be surprising for fixtures which are expected to set up per-input state. To proactively warn about this, we raise |HealthCheck.function_scoped_fixture| (unless suppressed with |settings.suppress_health_check|).
77
+
78
+
Combining |@given| and |pytest.mark.parametrize| is fully supported, again keeping in mind that |@given| supplies parameters from the right:
79
+
80
+
.. code-block:: python
81
+
82
+
@pytest.mark.parametrize("s", ["a", "b", "c"])
83
+
@given(st.integers())
84
+
deftest_use_parametrize(s, n):
85
+
assertisinstance(s, str)
86
+
assertisinstance(n, int)
87
+
88
+
unittest
89
+
~~~~~~~~
90
+
91
+
:pypi:`unittest` works out of the box with Hypothesis.
92
+
93
+
The :func:`python:unittest.mock.patch` decorator works with |@given|, but we recommend using it as a context manager within the test instead, to ensure that the mock is per-input, and to avoid poor interactions with Pytest fixtures.
94
+
95
+
:meth:`unittest.TestCase.subTest` is a no-op under Hypothesis, because it interacts poorly with Hypothesis generating hundreds of inputs at a time.
96
+
97
+
Django
98
+
~~~~~~
99
+
100
+
Integration with Django's testing requires use of the :ref:`hypothesis-django` extra. The issue is that Django tests reset the database once per test, rather than once per input.
101
+
102
+
:pypi:`pytest-django` doesn't yet implement Hypothesis compatibility. You can follow issue `pytest-django#912 <https://github.com/pytest-dev/pytest-django/issues/912>`_ for updates.
103
+
104
+
coverage.py
105
+
~~~~~~~~~~~
106
+
107
+
:pypi:`coverage` works out of the box with Hypothesis. Our own test suite has 100% branch coverage.
108
+
109
+
HypoFuzz
110
+
~~~~~~~~
111
+
112
+
`HypoFuzz <https://hypofuzz.com/>`_ is built on top of Hypothesis and has native support for it. See also the ``hypofuzz`` |alternative backend|.
Copy file name to clipboardExpand all lines: hypothesis-python/docs/explanation/domain.rst
+10-7Lines changed: 10 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -3,22 +3,25 @@ Domain and distribution
3
3
4
4
.. note::
5
5
6
-
This page is mainly for users who are familiar with other property-based testing libraries, and who expect a way to control the distribution of inputs in Hypothesis, via e.g. a scale parameter for size or a frequency parameter for relative probabilities.
6
+
This page is primarily for users who may be familiar with other property-based testing libraries, and who expect control over the distribution of inputs in Hypothesis, via e.g. a ``scale`` parameter for size or a ``frequency`` parameter for relative probabilities.
7
7
8
8
Hypothesis makes a distinction between the *domain* of a strategy, and the *distribution* of a strategy.
9
9
10
10
The *domain* is the set of inputs that should be possible to generate. For instance, in ``lists(integers())``, the domain is lists of integers. For other strategies, particularly those that use |st.composite| or |assume| in their definition, the domain might be more complex.
11
11
12
12
The *distribution* is the probability with which different elements in the domain should be generated. For ``lists(integers())``, should Hypothesis generate many small lists? Large lists? More positive or more negative numbers? etc.
13
13
14
-
Hypothesis takes a philosophical stance that property-based testing libraries, not the user, should be responsible for selecting the distribution. As an intentional design choice, Hypothesis therefore lets you control the domain of inputs to your test, but not the distribution.
14
+
Hypothesis takes a philosophical stance that while users may be responsible for selecting the domain, the property-based testing library—not the user—should be responsible for selecting the distribution. As an intentional design choice, Hypothesis therefore lets you control the domain of inputs to your test, but not the distribution.
15
15
16
-
There are a few reasons for this. One is that humans are pretty bad at choosing bug-finding distributions! Some bugs are "known unknowns": you had a good idea that some part of the code was buggy in a particular way. Others are "unknown unknowns": you had no idea that a certain kind of bug was even possible until you stumbled across it. Humans tend to overtune distributions for the former kind of bug, and not enough for the latter.
16
+
Why not let users control the distribution?
17
+
-------------------------------------------
17
18
18
-
To complicate things, the ideal distribution of a strategy depends not only on your project, but also on the property being tested. A strategy used in many places may be well-tuned by hand for one property, but badly tuned for another.
19
+
There are a few reasons Hypothesis doesn't let users control the distribution.
19
20
20
-
Another reason is that the distribution of inputs is a deeply internal implementation detail. We frequently make changes to the distributions of strategies, either in an explicit change for that strategy, or as an implicit consequence from other work on the Hypothesis engine. Exposing control over the distribution would lock us into a public API that may make improvements to Hypothesis more difficult.
21
+
* Humans are pretty bad at choosing bug-finding distributions! Some bugs are "known unknowns": you suspected that a part of the codebase was buggy in a particular way. Others are "unknown unknowns": you didn't know that a bug was possible until stumbling across it. Humans tend to overtune distributions for the former kind of bug, and not enough for the latter.
22
+
* The ideal strategy distribution depends not only on the codebase, but also on the property being tested. A strategy used in many places may have a good distribution for one property, but not another.
23
+
* The distribution of inputs is a deeply internal implementation detail. We sometimes change strategy distributions, either explicitly, or implicitly from other work on the Hypothesis engine. Exposing this would lock us into a public API that may make improvements to Hypothesis more difficult.
21
24
22
-
We're not saying that control over the distribution isn't useful! We occasionally receive requests to expose the distribution in Hypothesis (`e.g. <https://github.com/HypothesisWorks/hypothesis/issues/4205>`__), and users wouldn't be asking for it if it wasn't.
25
+
Finally, we think distribution control is better handled with |alternative backends|. If existing backends like ``hypofuzz`` and ``crosshair`` don't suit your needs, you can also write your own. Backends can automatically generalize and adapt to the strategy and property being tested and avoid most of the problems above.
23
26
24
-
However, adding this would make it easy for users to unknowingly weaken their tests and would add maintenance overhead to Hypothesis, and so we haven't yet done so.
27
+
We're not saying that control over the distribution isn't useful! We occasionally receive requests to expose the distribution in Hypothesis (`e.g. <https://github.com/HypothesisWorks/hypothesis/issues/4205>`__), and users wouldn't be asking for it if it wasn't. However, adding this to the public strategy API would make it easy for users to unknowingly weaken their tests, and would add maintenance overhead to Hypothesis, and so we haven't yet done so.
Copy file name to clipboardExpand all lines: hypothesis-python/docs/explanation/example-count.rst
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -66,7 +66,7 @@ Examples which are too large
66
66
67
67
For performance reasons, Hypothesis places an internal limit on the size of a single example. If an example exceeds this size limit, we will retry generating it and will not count it towards the |max_examples| limit. (And if we see too many of these large examples, we will raise |HealthCheck.data_too_large|, unless suppressed with |settings.suppress_health_check|).
68
68
69
-
The specific value of the size limit is an undocumented implementation detail. The majority of Hypothesis tests do not come close to hitting it.
69
+
The specific value of this size limit is an undocumented implementation detail. The majority of Hypothesis tests do not come close to hitting it.
Copy file name to clipboardExpand all lines: hypothesis-python/docs/how-to/index.rst
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,7 +1,7 @@
1
1
How-to guides
2
2
=============
3
3
4
-
Practical guides for applying Hypothesis in specific scenarios. Each guide addresses a particular question about using Hypothesis.
4
+
These how-to pages are practical guides for applying Hypothesis in specific scenarios. Each page addresses a particular question about using Hypothesis.
0 commit comments