Skip to content

Frequently Asked Questions

How do I restrict the range of generated values?

Schemathesis generates values across the full range permitted by the spec — including far-future dates, large integers, and unusual Unicode characters. This is intentional: servers that crash on extreme-but-valid input have real bugs. If your application already handles those cases correctly and you want to reduce noise, there are three levers:

Restrict a string format — override the built-in strategy for any format keyword:

from datetime import date
from hypothesis import strategies as st
import schemathesis

today = date.today()
schemathesis.openapi.format("date", st.dates(max_value=today).map(str))

See Overriding built-in formats for more detail.

Restrict the string character set — limit generated strings to a specific encoding, which also constrains the range of code points:

[generation]
codec = "ascii"

Disable null bytes — some systems mishandle the \x00 byte, causing spurious failures unrelated to your application logic:

[generation]
allow-x00 = false

What kind of data does Schemathesis generate?

Schemathesis generates three types of data:

  • Schema examples from your API documentation
  • Valid test data that follows schema constraints
  • Invalid test data that deliberately breaks constraints

The data covers all JSON Schema types for OpenAPI and both valid and invalid queries for GraphQL.

Note, that some generated data may be rejected by your API if the validation rules are not expressed in your schema.

What types of API issues can Schemathesis find?

Schemathesis identifies problems in three main categories:

Schema Compliance Issues

  • Response bodies not matching schemas
  • Undocumented status codes
  • Missing required headers
  • Wrong content types

Implementation Flaws

  • Server crashes (5xx responses)
  • Accepting invalid data
  • Rejecting valid data
  • Allowing missing required headers
  • Authentication bypasses

Stateful Behavior Issues

  • Deleted resources still accessible
  • Created resources not available

See more details in the Checks reference.

How should I run Schemathesis?

  • CLI: Complete feature set with all test phases, and reporting. Recommended for most users.
  • Python library: Integrates with pytest test suites but has fewer features than the CLI.

What if my application doesn't have an API schema?

If your API doesn't have a schema, you have several options:

  1. Generate a schema: Use tools like flasgger (Python), GrapeSwagger (Ruby), or Swashbuckle (ASP.NET) to automatically generate an initial schema from your code.

  2. Write a minimal schema: Create a basic schema manually covering just the endpoints you want to test first, then expand it over time.

  3. Use schema inference tools: Some third-party tools can observe API traffic and generate a schema based on observed requests and responses.

Starting with an imperfect schema is fine - Schemathesis can help you refine it by identifying inconsistencies between your schema and implementation.

How long does it usually take for Schemathesis to test an API?

Usually 30 seconds to 5 minutes, depending on:

  • API complexity (number of endpoints and parameters)
  • Test configuration (--max-examples setting)
  • API response time
  • Schema complexity

Control duration with --max-examples and --workers options.

How is Schemathesis different from other API testing tools?

Schemathesis differs from other API testing tools in several ways:

  • Property-based testing: Tests API properties (like "all responses should match their schema") rather than specific input-output pairs, automatically exploring the input space to find violations.

  • Stateful testing: Schemathesis can test sequences of API calls to find issues that only appear in specific request orders.

  • Failure minimization: When issues are found, Schemathesis automatically simplifies the failing test case to the minimal example that reproduces the problem.

  • Schema-first workflow: While tools like Postman or Insomnia focus on manual request creation, Schemathesis derives all test cases directly from your API specification.

Compared to tools like Dredd, Schemathesis focuses more on finding unexpected edge cases through property-based testing rather than verifying documented examples.

Why is Schemathesis skipping my Authorization header?

Schemathesis intentionally removes or modifies authentication in some test cases. This is security testing, not a bug.

Why this happens:

Schemathesis verifies that your API properly validates authentication by testing with: - No authentication credentials - Incorrect authentication credentials

This helps catch authentication bypass vulnerabilities where APIs accept requests they should reject.

When you'll see this:

  • The ignored_auth check makes additional requests without auth or with invalid credentials
  • Some test cases in the coverage phase may omit required headers including Authorization
  • You'll see failures if your API accepts requests it should reject

The majority of test cases still use your provided authentication normally. Only specific security-focused tests intentionally modify it.

How do I authenticate when my API issues tokens from a login endpoint?

Declare the login endpoint in schemathesis.toml and Schemathesis fetches the token for you:

[auth.dynamic.openapi.BearerAuth]
path = "/auth/token"
payload = { username = "${USERNAME}", password = "${PASSWORD}" }
extract_selector = "/access_token"

Replace BearerAuth with the securitySchemes name from your OpenAPI spec. The token is fetched once, cached for the run, and applied to every request requiring that scheme.

For tokens that expire mid-run or need per-scope caching, see Dynamic Token Authentication.

Can I use Schemathesis with Allure?

Yes. Use --report-allure-path to write Allure-compatible result files directly:

schemathesis run your_schema.yaml --report-allure-path allure-results
allure generate allure-results -o allure-report
allure open allure-report

See the Allure Integration guide for full details including Docker usage and pytest plugin support.

Prerequisites

Install schemathesis[allure] and the Allure CLI.

Alternative: JUnit XML

If you cannot install the allure extra, export JUnit XML and copy it into an Allure results directory:

schemathesis run your_schema.yaml
cp schemathesis-report/*.xml allure-results/
allure generate allure-results
allure open