Skip to content

How Schemathesis Integrates with Pytest

Schemathesis integrates with pytest to generate and run property-based API tests.

Test Execution Flow

When you run a Schemathesis pytest test, here's what happens:

import schemathesis

schema = schemathesis.openapi.from_url(
    "http://127.0.0.1:8080/openapi.json",
)


@schema.parametrize()
def test_api(case):
    case.call_and_validate()

A test run goes through these steps:

  1. Schema Loading: Schemathesis loads your API schema (happens at module import time)
  2. Test Collection: pytest discovers your test and @schema.parametrize() creates one parametrized test per API operation found in the schema
  3. Test Execution: For each operation (e.g., POST /users), pytest starts executing the parametrized test
  4. Property-Based Testing: Hypothesis runs the enabled phases for the operation: cases built from schema examples, coverage cases for schema constraints, and randomly generated fuzzing cases. The first case that fails the checks ends the operation's test, so fuzzing does not run after a failing example or coverage case
  5. Test Function Calls: Your test function runs once per generated case, receiving each case object

The pytest output has one line per API operation:

test.py::test_api[GET /users] PASSED    # One line per API operation
test.py::test_api[POST /users] FAILED   # Each runs many Hypothesis examples
test.py::test_api[DELETE /users/{id}] PASSED

How many examples run?

max_examples (100 by default) caps the cases generated by the fuzzing phase per operation; the examples and coverage phases add their own cases. Simple schemas (e.g., enum: ["A", "B"]) produce fewer fuzzing cases than the cap, while complex schemas may hit it.

Runs only under schemathesis run

Feature Under pytest
Stateful phase Not run by @schema.parametrize(); test sequences of operations with schema.as_state_machine()
API capability probes Not sent, so path parameters may carry backslashes or control characters that the server rejects before routing
Error feedback Cases follow the schema as written; rejected requests do not narrow later cases
Resource pool IDs and other values are generated, never taken from earlier responses
Automatic sign-up Requests carry only the credentials you configure
Auth inference Credentials for a security scheme go only to operations that declare it, even after a 401 or 403
Skipping operations that keep returning 405 Tested in every phase like other operations
baseline Not read; every failure fails the test
rate-limit = "auto" No wait or retry on 429; a fixed limit such as "100/m" applies
--generation-maximize Not applied; call hypothesis.target() in the test
ndjson, json and wfc reports Not written; pytest writes junit, vcr, har, allure and html
Crash files for schemathesis replay Not written; the failure output includes a curl command to reproduce the case

The Case Object

Each case represents one Hypothesis-generated example for an API operation:

def test_api(case):
    # case.method = "POST" 
    # case.path = "/users"
    # case.body = {"name": "generated_string", "age": 42}
    # case.headers = {"Content-Type": "application/json"}

The case includes:

  • Operation metadata: HTTP method, path, operation details
  • Generated data: headers, query params, path params, request body
  • Methods: call(), call_and_validate() for making requests

Case Object Reference

See the Case Object Reference for complete attributes and methods.

Deferred Discovery with Fixtures

When you need pytest fixtures for schema setup, use schemathesis.pytest.from_fixture:

@pytest.fixture
def api_schema(database):
    return schemathesis.openapi.from_asgi("/openapi.json", app)


schema = schemathesis.pytest.from_fixture("api_schema")


@schema.parametrize()
def test_api(case):
    case.call_and_validate()

Key differences:

  • Schema loading: Happens during test execution (not import time)
  • Test output: Uses pytest subtests instead of parametrization
  • Fixture support: Schema can depend on other pytest fixtures

--collect-only shows one item per test function

Because operations are discovered lazily at runtime, --collect-only cannot enumerate them upfront - this is by design.

Testing Multiple Schemas

schemathesis.pytest.parametrize() accepts named schemas as keyword arguments and generates test items from all of them under a single function:

@schemathesis.pytest.parametrize(
    users=schemathesis.openapi.from_wsgi("/openapi.json", users_app),
    orders=schemathesis.openapi.from_wsgi("/openapi.json", orders_app),
)
def test_api(case):
    case.call_and_validate()

The keyword name becomes the first bracket segment in the test ID:

test_api.py::test_api[users][GET /users] PASSED
test_api.py::test_api[orders][GET /orders] PASSED

Error Handling and Reporting

The distinct failures of the failing case are reported together, with reproduction steps for debugging:

test_api.py::test_api[POST /bookings] FAILED
test_api.py::test_api[GET /bookings/{booking_id}] PASSED
test_api.py::test_api[GET /health] PASSED

================================== FAILURES ===================================
__________________________ test_api[POST /bookings] ___________________________
+ Exception Group Traceback (most recent call last):
  | # snip
  | schemathesis.core.failures.FailureGroup: Schemathesis found 2 distinct failures
  |
  | - ST101 Server error
  |
  | - ST103 Undocumented status code
  |
  |     Received: 500
  |     Documented: 200, 422
  |
  | [500] Internal Server Error:
  |
  |     `Internal Server Error`
  |
  | Reproduce with:
  |
  |     curl -X POST -H 'Authorization: Bearer secret-token' \
  |       -H 'Content-Type: application/json' \
  |       -d '{"guest_name": "00", "nights": 1, "room_type": ""}' \
  |       http://127.0.0.1:8080/bookings
  |
  |  (2 sub-exceptions)
  +-+---------------- 1 ----------------

Async Support

Schemathesis supports asynchronous test functions with no additional configuration beyond installing pytest-asyncio or pytest-trio:

import httpx
import pytest
import schemathesis

schema = schemathesis.openapi.from_url("http://127.0.0.1:8080/openapi.json")


async def send_and_validate(case):
    kwargs = case.as_transport_kwargs()
    # httpx expects cookies on the client, not per request
    cookies = kwargs.pop("cookies", None)
    # httpx takes raw bytes or text as `content`, not `data`
    if isinstance(kwargs.get("data"), (bytes, str)):
        kwargs["content"] = kwargs.pop("data")
    async with httpx.AsyncClient(cookies=cookies) as client:
        response = await client.request(**kwargs)
    case.validate_response(response)


@pytest.mark.asyncio
@schema.parametrize()
async def test_api_async(case):
    await send_and_validate(case)


# Or with trio
@pytest.mark.trio
@schema.parametrize()
async def test_api_trio(case):
    await send_and_validate(case)

Async Network Calls

case.call() makes synchronous network calls. To use an async client, send the request yourself: case.as_transport_kwargs() returns the method, URL, headers, query parameters, cookies, and serialized body, and case.validate_response() runs the checks on the response you received.