How Schemathesis Integrates with Pytest
Schemathesis integrates with pytest to generate and run property-based API tests.
Test Execution Flow
When you run a Schemathesis pytest test, here's what happens:
import schemathesis
schema = schemathesis.openapi.from_url(
"http://127.0.0.1:8080/openapi.json",
)
@schema.parametrize()
def test_api(case):
case.call_and_validate()
A test run goes through these steps:
- Schema Loading: Schemathesis loads your API schema (happens at module import time)
- Test Collection: pytest discovers your test and
@schema.parametrize()creates one parametrized test per API operation found in the schema - Test Execution: For each operation (e.g.,
POST /users), pytest starts executing the parametrized test - Property-Based Testing: Hypothesis runs the enabled phases for the operation: cases built from schema examples, coverage cases for schema constraints, and randomly generated fuzzing cases. The first case that fails the checks ends the operation's test, so fuzzing does not run after a failing example or coverage case
- Test Function Calls: Your test function runs once per generated case, receiving each
caseobject
The pytest output has one line per API operation:
test.py::test_api[GET /users] PASSED # One line per API operation
test.py::test_api[POST /users] FAILED # Each runs many Hypothesis examples
test.py::test_api[DELETE /users/{id}] PASSED
How many examples run?
max_examples (100 by default) caps the cases generated by the fuzzing phase per operation; the examples and coverage phases add their own cases. Simple schemas (e.g., enum: ["A", "B"]) produce fewer fuzzing cases than the cap, while complex schemas may hit it.
Runs only under schemathesis run
| Feature | Under pytest |
|---|---|
| Stateful phase | Not run by @schema.parametrize(); test sequences of operations with schema.as_state_machine() |
| API capability probes | Not sent, so path parameters may carry backslashes or control characters that the server rejects before routing |
| Error feedback | Cases follow the schema as written; rejected requests do not narrow later cases |
| Resource pool | IDs and other values are generated, never taken from earlier responses |
| Automatic sign-up | Requests carry only the credentials you configure |
| Auth inference | Credentials for a security scheme go only to operations that declare it, even after a 401 or 403 |
Skipping operations that keep returning 405 |
Tested in every phase like other operations |
baseline |
Not read; every failure fails the test |
rate-limit = "auto" |
No wait or retry on 429; a fixed limit such as "100/m" applies |
--generation-maximize |
Not applied; call hypothesis.target() in the test |
ndjson, json and wfc reports |
Not written; pytest writes junit, vcr, har, allure and html |
Crash files for schemathesis replay |
Not written; the failure output includes a curl command to reproduce the case |
The Case Object
Each case represents one Hypothesis-generated example for an API operation:
def test_api(case):
# case.method = "POST"
# case.path = "/users"
# case.body = {"name": "generated_string", "age": 42}
# case.headers = {"Content-Type": "application/json"}
The case includes:
- Operation metadata: HTTP method, path, operation details
- Generated data: headers, query params, path params, request body
- Methods:
call(),call_and_validate()for making requests
Case Object Reference
See the Case Object Reference for complete attributes and methods.
Deferred Discovery with Fixtures
When you need pytest fixtures for schema setup, use schemathesis.pytest.from_fixture:
@pytest.fixture
def api_schema(database):
return schemathesis.openapi.from_asgi("/openapi.json", app)
schema = schemathesis.pytest.from_fixture("api_schema")
@schema.parametrize()
def test_api(case):
case.call_and_validate()
Key differences:
- Schema loading: Happens during test execution (not import time)
- Test output: Uses pytest subtests instead of parametrization
- Fixture support: Schema can depend on other pytest fixtures
--collect-only shows one item per test function
Because operations are discovered lazily at runtime, --collect-only cannot enumerate them upfront - this is by design.
Testing Multiple Schemas
schemathesis.pytest.parametrize() accepts named schemas as keyword arguments and generates test items from all of them under a single function:
@schemathesis.pytest.parametrize(
users=schemathesis.openapi.from_wsgi("/openapi.json", users_app),
orders=schemathesis.openapi.from_wsgi("/openapi.json", orders_app),
)
def test_api(case):
case.call_and_validate()
The keyword name becomes the first bracket segment in the test ID:
test_api.py::test_api[users][GET /users] PASSED
test_api.py::test_api[orders][GET /orders] PASSED
Error Handling and Reporting
The distinct failures of the failing case are reported together, with reproduction steps for debugging:
test_api.py::test_api[POST /bookings] FAILED
test_api.py::test_api[GET /bookings/{booking_id}] PASSED
test_api.py::test_api[GET /health] PASSED
================================== FAILURES ===================================
__________________________ test_api[POST /bookings] ___________________________
+ Exception Group Traceback (most recent call last):
| # snip
| schemathesis.core.failures.FailureGroup: Schemathesis found 2 distinct failures
|
| - ST101 Server error
|
| - ST103 Undocumented status code
|
| Received: 500
| Documented: 200, 422
|
| [500] Internal Server Error:
|
| `Internal Server Error`
|
| Reproduce with:
|
| curl -X POST -H 'Authorization: Bearer secret-token' \
| -H 'Content-Type: application/json' \
| -d '{"guest_name": "00", "nights": 1, "room_type": ""}' \
| http://127.0.0.1:8080/bookings
|
| (2 sub-exceptions)
+-+---------------- 1 ----------------
Async Support
Schemathesis supports asynchronous test functions with no additional configuration beyond installing pytest-asyncio or pytest-trio:
import httpx
import pytest
import schemathesis
schema = schemathesis.openapi.from_url("http://127.0.0.1:8080/openapi.json")
async def send_and_validate(case):
kwargs = case.as_transport_kwargs()
# httpx expects cookies on the client, not per request
cookies = kwargs.pop("cookies", None)
# httpx takes raw bytes or text as `content`, not `data`
if isinstance(kwargs.get("data"), (bytes, str)):
kwargs["content"] = kwargs.pop("data")
async with httpx.AsyncClient(cookies=cookies) as client:
response = await client.request(**kwargs)
case.validate_response(response)
@pytest.mark.asyncio
@schema.parametrize()
async def test_api_async(case):
await send_and_validate(case)
# Or with trio
@pytest.mark.trio
@schema.parametrize()
async def test_api_trio(case):
await send_and_validate(case)
Async Network Calls
case.call() makes synchronous network calls. To use an async client, send the request yourself: case.as_transport_kwargs() returns the method, URL, headers, query parameters, cookies, and serialized body, and case.validate_response() runs the checks on the response you received.