Skip to content

Using Hypothesis Strategies with Schemathesis

Use your own Hypothesis strategies to inject custom data into Schemathesis tests and draw Schemathesis test cases inside your own Hypothesis tests.

Prerequisites

  • Schemathesis and pytest installed
  • Familiarity with Hypothesis strategies and @given

The examples load the schema from an ASGI app in myapp.py with POST /users, GET/PUT/DELETE /users/{user_id} and POST /posts. You can use any schema loader, for example schemathesis.openapi.from_url(...).

Turn API operations into strategies

Every API operation is a Hypothesis strategy that generates Case objects:

import schemathesis

from myapp import app

schema = schemathesis.openapi.from_asgi("/openapi.json", app)

# Single operation strategy
create_user = schema["/users"]["POST"].as_strategy()
get_user = schema["/users/{user_id}"]["GET"].as_strategy()

# Multiple operations combined
user_operations = create_user | get_user

# All operations for a path
all_user_operations = schema["/users/{user_id}"].as_strategy()

# All operations in the schema
all_operations = schema.as_strategy()

A Case holds the method, path, headers, query parameters and request body needed to make a request. The strategies behave like any other Hypothesis strategy; see the Hypothesis documentation.

The snippets below assume these imports and the schema defined above:

from hypothesis import HealthCheck, given, settings
from hypothesis import strategies as st

Add custom data to Schemathesis tests

Inject values with @schema.given()

@schema.given() works like Hypothesis's @given and combines with @schema.parametrize():

@schema.given(auth_token=st.sampled_from(["token1", "token2", "token3"]))
@schema.parametrize()
def test_api_with_auth(case, auth_token):
    case.headers["Authorization"] = f"Bearer {auth_token}"
    case.call_and_validate()


existing_user_ids = [1, 42, 123, 456]


@schema.given(user_id=st.sampled_from(existing_user_ids))
@schema.parametrize()
def test_user_endpoints(case, user_id):
    if "user_id" in case.path_parameters:
        case.path_parameters["user_id"] = user_id
    case.call_and_validate()

Each test runs many Hypothesis examples, so the custom values are sampled across different generated test cases.

Schema examples and @schema.given()

If your schema contains examples (in parameters or request bodies), a test that uses @schema.given() fails with IncorrectUsage: Cannot combine @schema.given() with schema examples. Schema examples only provide the case argument, and the extra arguments have no value.

Split the test by Hypothesis phase:

from hypothesis import Phase, settings


# Schema examples, without @schema.given()
@schema.parametrize()
@settings(phases=[Phase.explicit])
def test_user_endpoints_with_examples(case):
    case.call_and_validate()


# Generated cases with custom strategies
@schema.given(user_id=st.sampled_from(existing_user_ids))
@schema.parametrize()
@settings(phases=[Phase.generate])
def test_user_endpoints_with_custom_data(case, user_id):
    if "user_id" in case.path_parameters:
        case.path_parameters["user_id"] = user_id
    case.call_and_validate()

Create and clean up records per case

db here is your own fixture that creates and deletes users. With pytest-django, override its db fixture in conftest.py so the helpers run with database access:

import pytest

from users.models import User


class UserStore:
    def create_user(self, data):
        return User.objects.create(**data).id

    def delete_user(self, user_id):
        User.objects.filter(id=user_id).delete()


@pytest.fixture
def db(db):
    return UserStore()

The test then uses it:

@schema.given(
    user_data=st.fixed_dictionaries(
        {
            "name": st.text(min_size=1, max_size=50),
            "role": st.sampled_from(["user", "admin"]),
        }
    )
)
@schema.parametrize()
@settings(suppress_health_check=[HealthCheck.function_scoped_fixture])
def test_api_with_db_setup(db, case, user_data):
    user_id = db.create_user(user_data)
    try:
        if "user_id" in case.path_parameters:
            case.path_parameters["user_id"] = user_id
        case.call_and_validate()
    finally:
        db.delete_user(user_id)

pytest creates a function-scoped fixture such as db once per test function, not once per generated case, so Hypothesis raises FailedHealthCheck unless you suppress HealthCheck.function_scoped_fixture. The try/finally block creates and removes a user for every generated case.

With Django, load the schema from the WSGI application, schemathesis.openapi.from_wsgi("/openapi.json", get_wsgi_application()). WSGI requests run in the test's thread and see the rows the fixture creates inside pytest-django's transaction. Django's ASGI application serves requests on another database connection, outside that transaction, and the test fails.

Choose the next request from a response

st.data() lets the test draw more values while it runs. Use the response to POST /users to choose the next request:

get_user_operation = schema["/users/{user_id}"]["GET"].as_strategy()
create_post_operation = schema["/posts"]["POST"].as_strategy()


@schema.given(data=st.data())
@schema.parametrize()
def test_user_workflow(case, data):
    response = case.call_and_validate()
    if case.method == "POST" and case.path == "/users" and response.status_code == 201:
        user = response.json()
        if user["role"] == "admin":
            next_case = data.draw(get_user_operation)
            next_case.path_parameters = {"user_id": user["id"]}
        else:
            next_case = data.draw(create_post_operation)
            next_case.body = {**next_case.body, "author_id": user["id"]}
        next_case.call_and_validate()

Use Schemathesis strategies in your own tests

Build a request sequence

With plain @given, you control the order of requests and Schemathesis generates the data for each step:

create_user_strategy = schema["/users"]["POST"].as_strategy()
update_user_strategy = schema["/users/{user_id}"]["PUT"].as_strategy()
delete_user_strategy = schema["/users/{user_id}"]["DELETE"].as_strategy()


@given(data=st.data())
def test_user_lifecycle(data):
    # Step 1: always create a user
    create_case = data.draw(create_user_strategy)
    response = create_case.call_and_validate()
    user_id = response.json()["id"]

    # Step 2: optional steps
    if data.draw(st.booleans()):
        update_case = data.draw(update_user_strategy)
        update_case.path_parameters = {"user_id": user_id}
        update_case.call_and_validate()

    if data.draw(st.booleans()):
        post_case = data.draw(schema["/posts"]["POST"].as_strategy())
        post_case.body = {**post_case.body, "author_id": user_id}
        post_case.call_and_validate()

    # Step 3: always clean up
    delete_case = data.draw(delete_user_strategy)
    delete_case.path_parameters = {"user_id": user_id}
    delete_case.call_and_validate()

Use strategies with unittest

Strategies work with @given in any test framework that supports Hypothesis:

from unittest import TestCase

from hypothesis import given

create_user_strategy = schema["/users"]["POST"].as_strategy()
get_user_strategy = schema["/users/{user_id}"]["GET"].as_strategy()


class TestAPI(TestCase):
    @given(case=create_user_strategy)
    def test_create_user(self, case):
        response = case.call_and_validate()
        self.assertIn("id", response.json())

    @given(create_case=create_user_strategy, get_case=get_user_strategy)
    def test_create_then_get(self, create_case, get_case):
        create_response = create_case.call_and_validate()
        user_id = create_response.json()["id"]

        get_case.path_parameters = {"user_id": user_id}
        get_response = get_case.call_and_validate()

        self.assertEqual(get_response.json()["id"], user_id)

Run these tests with pytest or python -m unittest:

..
----------------------------------------------------------------------
Ran 2 tests in 0.717s

OK

Troubleshooting

FailedHealthCheck: '...' uses a function-scoped fixture '...'. Suppress HealthCheck.function_scoped_fixture as shown above, or give the fixture a wider scope.

IncorrectUsage: Cannot combine `@schema.given()` with schema examples. Split the test into an explicit-examples test and a generated-data test, as shown in the warning above.

OperationNotFound: `/users/{id}` not found. Did you mean `/users/{user_id}`? The path in schema["/path"]["METHOD"] must match the schema exactly, including path parameter names such as {user_id}. An unknown method raises LookupError: Method `POST` not found. Available methods: GET.