Schemathesis: Property-based testing for API schemas๏
Schemathesis is a specification-centric API testing tool for Open API and GraphQL-based applications.
It reads the application schema and generates test cases, which will ensure that your application is compliant with its schema and never crashes.
The application under test could be written in any language; the only thing you need is a valid API schema in a supported format.
Simple to use and yet powerful to uncover hard-to-find errors thanks to the property-based testing approach backed by state-of-the-art Hypothesis library.
You can use Schemathesis in the command line directly:
st run https://example.schemathesis.io/openapi.json
Or via Docker:
docker run schemathesis/schemathesis:stable run https://example.schemathesis.io/openapi.json
Or in your Python tests:
import schemathesis
schema = schemathesis.from_uri("https://example.schemathesis.io/openapi.json")
@schema.parametrize()
def test_api(case):
case.call_and_validate()
Both examples above will run hundreds of requests against the API under test and report all found failures and inconsistencies along with instructions to reproduce them.
Note
You can also use our SaaS to run more comprehensive tests and visualise the outcomes!
Features๏
Open API: Schema conformance, explicit examples, stateful testing;
GraphQL: queries generation;
Multi-worker test execution;
Storing and replaying tests;
ASGI / WSGI support;
Generated code samples (cURL, Python);
Docker image;
Customizable checks & test generation
Note
๐ Join our Discord, weโd love to hear your feedback ๐
Userโs Guide๏
- Introduction
- Command Line Interface
- Basic usage
- Testing specific operations
- Tests configuration
- How are responses checked?
- Concurrent testing
- Code samples style
- ASGI / WSGI support
- Storing and replaying test cases
- JUnit support
- Base URL configuration
- Extending CLI
- Rate limiting
- Debugging
- Running CLI via Docker
- Full list of CLI options
- Writing Python tests
- Continuous Integration
- Schemathesis as a Service
- Authentication
- Additional features
- Stateful testing
- Why is it useful?
- How to specify connections?
- Minimal example
- Lazy schema loading
- How it works behind the scenes?
- How to customize tests
- Using pytest fixtures
- Hypothesis configuration
- How to provide initial data for test scenarios?
- Examples
- Reproducing failures
- Corner cases
- Command Line Interface
- Open API links limitations
- Data generation
- Compatibility
- Examples in API schemas
- GraphQL
- Targeted property-based testing
- Extending Schemathesis
Support๏
If you want to integrate Schemathesis into your company workflows or improve its effectiveness, feel free to reach out to support@schemathesis.io.
Schemathesis.io also runs workshops about effective API testing. Signup here
Resources๏
Deriving Semantics-Aware Fuzzers from Web API Schemas by Zac-HD and @Stranger6667
An article about Schemathesis by @Stranger6667
Effective API schemas testing from DevConf.cz by @Stranger6667
How to use Schemathesis to test Flask API in GitHub Actions by @lina-is-here
A video from EuroPython 2020 by @hultner
Schemathesis tutorial with an accompanying video by Red Hat
Using Hypothesis and Schemathesis to Test FastAPI by @amalshaji
Additional notes๏
- Recipes
- Public API reference
- Frequently Asked Questions
- Changelog
- Unreleased - TBD
- 3.19.5 - 2023-06-03
- 3.19.4 - 2023-06-03
- 3.19.3 - 2023-05-25
- 3.19.2 - 2023-05-20
- 3.19.1 - 2023-04-26
- 3.19.0 - 2023-03-22
- 3.18.5 - 2023-02-18
- 3.18.4 - 2023-02-16
- 3.18.3 - 2023-02-12
- 3.18.2 - 2023-02-08
- 3.18.1 - 2023-02-06
- 3.18.0 - 2023-02-01
- 3.17.5 - 2022-11-08
- 3.17.4 - 2022-10-19
- 3.17.3 - 2022-10-10
- 3.17.2 - 2022-08-27
- 3.17.1 - 2022-08-19
- 3.17.0 - 2022-08-17
- 3.16.5 - 2022-08-11
- 3.16.4 - 2022-08-09
- 3.16.3 - 2022-08-08
- 3.16.2 - 2022-08-05
- 3.16.1 - 2022-07-29
- 3.16.0 - 2022-07-22
- 3.15.6 - 2022-06-23
- 3.15.5 - 2022-06-21
- 3.15.4 - 2022-06-06
- 3.15.3 - 2022-05-28
- 3.15.2 - 2022-05-09
- 3.15.1 - 2022-05-03
- 3.15.0 - 2022-05-01
- 3.14.2 - 2022-04-21
- 3.14.1 - 2022-04-18
- 3.14.0 - 2022-04-17
- 3.13.9 - 2022-04-14
- 3.13.8 - 2022-04-05
- 3.13.7 - 2022-04-02
- 3.13.6 - 2022-03-31
- 3.13.5 - 2022-03-31
- 3.13.4 - 2022-03-29
- 3.13.3 - 2022-02-20
- 3.13.2 - 2022-02-16
- 3.13.1 - 2022-02-10
- 3.13.0 - 2022-02-09
- 3.12.3 - 2022-01-13
- 3.12.2 - 2022-01-12
- 3.12.1 - 2021-12-31
- 3.12.0 - 2021-12-29
- 3.11.7 - 2021-12-23
- 3.11.6 - 2021-12-20
- 3.11.5 - 2021-12-04
- 3.11.4 - 2021-12-03
- 3.11.3 - 2021-12-02
- 3.11.2 - 2021-11-30
- 3.11.1 - 2021-11-20
- 3.11.0 - 2021-11-03
- 3.10.1 - 2021-10-04
- 3.10.0 - 2021-09-13
- 3.9.7 - 2021-07-26
- 3.9.6 - 2021-07-15
- 3.9.5 - 2021-07-14
- 3.9.4 - 2021-07-09
- 3.9.3 - 2021-06-22
- 3.9.2 - 2021-06-16
- 3.9.1 - 2021-06-13
- 3.9.0 - 2021-06-07
- 3.8.0 - 2021-06-03
- 3.7.8 - 2021-06-02
- 3.7.7 - 2021-06-01
- 3.7.6 - 2021-05-31
- 3.7.5 - 2021-05-31
- 3.7.4 - 2021-05-28
- 3.7.3 - 2021-05-28
- 3.7.2 - 2021-05-27
- 3.7.1 - 2021-05-23
- 3.7.0 - 2021-05-23
- 3.6.11 - 2021-05-20
- 3.6.10 - 2021-05-17
- 3.6.9 - 2021-05-14
- 3.6.8 - 2021-05-13
- 3.6.7 - 2021-05-12
- 3.6.6 - 2021-05-07
- 3.6.5 - 2021-05-07
- 3.6.4 - 2021-04-30
- 3.6.3 - 2021-04-20
- 3.6.2 - 2021-04-15
- 3.6.1 - 2021-04-09
- 3.6.0 - 2021-04-04
- 3.5.3 - 2021-03-27
- 3.5.2 - 2021-03-24
- 3.5.1 - 2021-03-23
- 3.5.0 - 2021-03-22
- 3.4.1 - 2021-03-21
- 3.4.0 - 2021-03-20
- 3.3.1 - 2021-03-18
- 3.3.0 - 2021-03-17
- 3.2.2 - 2021-03-11
- 3.2.1 - 2021-03-10
- 3.2.0 - 2021-03-09
- 3.1.3 - 2021-03-08
- 3.1.2 - 2021-03-08
- 3.1.1 - 2021-03-05
- 3.1.0 - 2021-02-11
- 3.0.9 - 2021-02-10
- 3.0.8 - 2021-02-04
- 3.0.7 - 2021-01-31
- 3.0.6 - 2021-01-31
- 3.0.5 - 2021-01-30
- 3.0.4 - 2021-01-19
- 3.0.3 - 2021-01-18
- 3.0.2 - 2021-01-15
- 3.0.1 - 2021-01-15
- 3.0.0 - 2021-01-14
- 2.8.6 - 2022-03-29
- 2.8.5 - 2020-12-15
- 2.8.4 - 2020-11-27
- 2.8.3 - 2020-11-27
- 2.8.2 - 2020-11-25
- 2.8.1 - 2020-11-24
- 2.8.0 - 2020-11-24
- 2.7.7 - 2020-11-13
- 2.7.6 - 2020-11-12
- 2.7.5 - 2020-11-09
- 2.7.4 - 2020-11-07
- 2.7.3 - 2020-11-05
- 2.7.2 - 2020-11-05
- 2.7.1 - 2020-10-22
- 2.7.0 - 2020-10-21
- 2.6.1 - 2020-10-19
- 2.6.0 - 2020-10-06
- 2.5.1 - 2020-09-30
- 2.5.0 - 2020-09-27
- 2.4.1 - 2020-09-17
- 2.4.0 - 2020-09-15
- 2.3.4 - 2020-09-11
- 2.3.3 - 2020-08-04
- 2.3.2 - 2020-08-04
- 2.3.1 - 2020-07-28
- 2.3.0 - 2020-07-26
- 2.2.1 - 2020-07-22
- 2.2.0 - 2020-07-14
- 2.1.0 - 2020-07-06
- 2.0.0 - 2020-07-01
- 1.10.0 - 2020-06-28
- 1.9.1 - 2020-06-21
- 1.9.0 - 2020-06-20
- 1.8.0 - 2020-06-15
- 1.7.0 - 2020-05-30
- 1.6.3 - 2020-05-26
- 1.6.2 - 2020-05-15
- 1.6.1 - 2020-05-13
- 1.6.0 - 2020-05-10
- 1.5.1 - 2020-05-08
- 1.5.0 - 2020-05-06
- 1.4.0 - 2020-05-03
- 1.3.4 - 2020-04-30
- 1.3.3 - 2020-04-29
- 1.3.2 - 2020-04-27
- 1.3.1 - 2020-04-27
- 1.3.0 - 2020-04-27
- 1.2.0 - 2020-04-15
- 1.1.2 - 2020-04-14
- 1.1.1 - 2020-04-12
- 1.1.0 - 2020-04-08
- 1.0.5 - 2020-04-03
- 1.0.4 - 2020-04-03
- 1.0.3 - 2020-04-03
- 1.0.2 - 2020-04-02
- 1.0.1 - 2020-04-01
- 1.0.0 - 2020-03-31
- 0.28.0 - 2020-03-31
- 0.27.0 - 2020-03-31
- 0.26.1 - 2020-03-24
- 0.26.0 - 2020-03-19
- 0.25.1 - 2020-03-09
- 0.25.0 - 2020-02-27
- 0.24.5 - 2020-02-26
- 0.24.4 - 2020-02-22
- 0.24.3 - 2020-02-10
- 0.24.2 - 2020-02-09
- 0.24.1 - 2020-02-08
- 0.24.0 - 2020-02-07
- 0.23.7 - 2020-01-30
- 0.23.6 - 2020-01-28
- 0.23.5 - 2020-01-24
- 0.23.4 - 2020-01-22
- 0.23.3 - 2020-01-21
- 0.23.2 - 2020-01-16
- 0.23.1 - 2020-01-15
- 0.23.0 - 2020-01-15
- 0.22.0 - 2020-01-11
- 0.21.0 - 2019-12-20
- 0.20.5 - 2019-12-18
- 0.20.4 - 2019-12-17
- 0.20.3 - 2019-12-17
- 0.20.2 - 2019-12-14
- 0.20.1 - 2019-12-13
- 0.20.0 - 2019-12-12
- 0.19.1 - 2019-12-11
- 0.19.0 - 2019-12-02
- 0.18.1 - 2019-11-28
- 0.18.0 - 2019-11-27
- 0.17.0 - 2019-11-21
- 0.16.0 - 2019-11-19
- 0.15.0 - 2019-11-15
- 0.14.0 - 2019-11-09
- 0.13.2 - 2019-11-05
- 0.13.1 - 2019-11-05
- 0.13.0 - 2019-11-03
- 0.12.2 - 2019-10-30
- 0.12.1 - 2019-10-28
- 0.12.0 - 2019-10-28
- 0.11.0 - 2019-10-22
- 0.10.0 - 2019-10-14
- 0.9.0 - 2019-10-09
- 0.8.1 - 2019-10-04
- 0.8.0 - 2019-10-04
- 0.7.3 - 2019-09-30
- 0.7.2 - 2019-09-30
- 0.7.1 - 2019-09-27
- 0.7.0 - 2019-09-26
- 0.6.0 - 2019-09-24
- 0.5.0 - 2019-09-16
- 0.4.1 - 2019-09-11
- 0.4.0 - 2019-09-10
- 0.3.0 - 2019-09-06
- 0.2.0 - 2019-09-05
- 0.1.0 - 2019-06-28