Schemathesis: Property-based testing for API schemas๏
Schemathesis is a modern API testing tool for web applications built with Open API and GraphQL specifications.
It reads the application schema and generates test cases, which will ensure that your application is compliant with its schema.
The application under test could be written in any language; the only thing you need is a valid API schema in a supported format.
Simple to use and yet powerful to uncover hard-to-find errors thanks to the property-based testing approach backed by state-of-the-art Hypothesis library.
You can use Schemathesis in the command line directly:
st run https://example.schemathesis.io/openapi.json
Or via Docker:
docker run schemathesis/schemathesis:stable run https://example.schemathesis.io/openapi.json
Or in your Python tests:
import schemathesis
schema = schemathesis.from_uri("https://example.schemathesis.io/openapi.json")
@schema.parametrize()
def test_api(case):
case.call_and_validate()
Both examples above will run hundreds of requests against the API under test and report all found failures and inconsistencies along with instructions to reproduce them.
Note
You can also use our SaaS to run more comprehensive tests and visualise the outcomes!
Features๏
Content-Type, schema, headers, and status code conformance checks for Open API;
Testing of explicit examples from the input schema;
Stateful testing via Open API links;
Concurrent test execution;
Targeted testing;
Storing and replaying network requests;
Built-in ASGI / WSGI application support;
Code samples for easy failure reproduction;
Ready-to-go Docker image;
Configurable with user-defined checks, string formats, hooks, and targets.
Note
๐ Join our Discord, weโd love to hear your feedback ๐
Userโs Guide๏
- Introduction
- Command Line Interface
- Writing Python tests
- Authentication
- Stateful testing
- Why is it useful?
- How to specify connections?
- Minimal example
- Lazy schema loading
- How it works behind the scenes?
- How to customize tests
- Using pytest fixtures
- Hypothesis configuration
- How to provide initial data for test scenarios?
- Examples
- Reproducing failures
- Corner cases
- Command Line Interface
- Open API links limitations
- Schemathesis as a Service
- Data generation
- Compatibility
- Examples in API schemas
- GraphQL
- Targeted property-based testing
- Extending Schemathesis
Commercial support๏
If you are interested in the effective integration of Schemathesis to your private project, you can contact me via email or Twitter and I will help you do that.
Resources๏
Deriving Semantics-Aware Fuzzers from Web API Schemas by Zac-HD and @Stranger6667
An article about Schemathesis by @Stranger6667
Effective API schemas testing from DevConf.cz by @Stranger6667
A video from EuroPython 2020 by @hultner
Schemathesis tutorial with an accompanying video by Red Hat
Using Hypothesis and Schemathesis to Test FastAPI by @amalshaji
Additional notes๏
- Recipes
- Public API reference
- Frequently Asked Questions
- Changelog
- Unreleased - TBD
- 3.15.2 - 2022-05-09
- 3.15.1 - 2022-05-03
- 3.15.0 - 2022-05-01
- 3.14.2 - 2022-04-21
- 3.14.1 - 2022-04-18
- 3.14.0 - 2022-04-17
- 3.13.9 - 2022-04-14
- 3.13.8 - 2022-04-05
- 3.13.7 - 2022-04-02
- 3.13.6 - 2022-03-31
- 3.13.5 - 2022-03-31
- 3.13.4 - 2022-03-29
- 3.13.3 - 2022-02-20
- 3.13.2 - 2022-02-16
- 3.13.1 - 2022-02-10
- 3.13.0 - 2022-02-09
- 3.12.3 - 2022-01-13
- 3.12.2 - 2022-01-12
- 3.12.1 - 2021-12-31
- 3.12.0 - 2021-12-29
- 3.11.7 - 2021-12-23
- 3.11.6 - 2021-12-20
- 3.11.5 - 2021-12-04
- 3.11.4 - 2021-12-03
- 3.11.3 - 2021-12-02
- 3.11.2 - 2021-11-30
- 3.11.1 - 2021-11-20
- 3.11.0 - 2021-11-03
- 3.10.1 - 2021-10-04
- 3.10.0 - 2021-09-13
- 3.9.7 - 2021-07-26
- 3.9.6 - 2021-07-15
- 3.9.5 - 2021-07-14
- 3.9.4 - 2021-07-09
- 3.9.3 - 2021-06-22
- 3.9.2 - 2021-06-16
- 3.9.1 - 2021-06-13
- 3.9.0 - 2021-06-07
- 3.8.0 - 2021-06-03
- 3.7.8 - 2021-06-02
- 3.7.7 - 2021-06-01
- 3.7.6 - 2021-05-31
- 3.7.5 - 2021-05-31
- 3.7.4 - 2021-05-28
- 3.7.3 - 2021-05-28
- 3.7.2 - 2021-05-27
- 3.7.1 - 2021-05-23
- 3.7.0 - 2021-05-23
- 3.6.11 - 2021-05-20
- 3.6.10 - 2021-05-17
- 3.6.9 - 2021-05-14
- 3.6.8 - 2021-05-13
- 3.6.7 - 2021-05-12
- 3.6.6 - 2021-05-07
- 3.6.5 - 2021-05-07
- 3.6.4 - 2021-04-30
- 3.6.3 - 2021-04-20
- 3.6.2 - 2021-04-15
- 3.6.1 - 2021-04-09
- 3.6.0 - 2021-04-04
- 3.5.3 - 2021-03-27
- 3.5.2 - 2021-03-24
- 3.5.1 - 2021-03-23
- 3.5.0 - 2021-03-22
- 3.4.1 - 2021-03-21
- 3.4.0 - 2021-03-20
- 3.3.1 - 2021-03-18
- 3.3.0 - 2021-03-17
- 3.2.2 - 2021-03-11
- 3.2.1 - 2021-03-10
- 3.2.0 - 2021-03-09
- 3.1.3 - 2021-03-08
- 3.1.2 - 2021-03-08
- 3.1.1 - 2021-03-05
- 3.1.0 - 2021-02-11
- 3.0.9 - 2021-02-10
- 3.0.8 - 2021-02-04
- 3.0.7 - 2021-01-31
- 3.0.6 - 2021-01-31
- 3.0.5 - 2021-01-30
- 3.0.4 - 2021-01-19
- 3.0.3 - 2021-01-18
- 3.0.2 - 2021-01-15
- 3.0.1 - 2021-01-15
- 3.0.0 - 2021-01-14
- 2.8.6 - 2022-03-29
- 2.8.5 - 2020-12-15
- 2.8.4 - 2020-11-27
- 2.8.3 - 2020-11-27
- 2.8.2 - 2020-11-25
- 2.8.1 - 2020-11-24
- 2.8.0 - 2020-11-24
- 2.7.7 - 2020-11-13
- 2.7.6 - 2020-11-12
- 2.7.5 - 2020-11-09
- 2.7.4 - 2020-11-07
- 2.7.3 - 2020-11-05
- 2.7.2 - 2020-11-05
- 2.7.1 - 2020-10-22
- 2.7.0 - 2020-10-21
- 2.6.1 - 2020-10-19
- 2.6.0 - 2020-10-06
- 2.5.1 - 2020-09-30
- 2.5.0 - 2020-09-27
- 2.4.1 - 2020-09-17
- 2.4.0 - 2020-09-15
- 2.3.4 - 2020-09-11
- 2.3.3 - 2020-08-04
- 2.3.2 - 2020-08-04
- 2.3.1 - 2020-07-28
- 2.3.0 - 2020-07-26
- 2.2.1 - 2020-07-22
- 2.2.0 - 2020-07-14
- 2.1.0 - 2020-07-06
- 2.0.0 - 2020-07-01
- 1.10.0 - 2020-06-28
- 1.9.1 - 2020-06-21
- 1.9.0 - 2020-06-20
- 1.8.0 - 2020-06-15
- 1.7.0 - 2020-05-30
- 1.6.3 - 2020-05-26
- 1.6.2 - 2020-05-15
- 1.6.1 - 2020-05-13
- 1.6.0 - 2020-05-10
- 1.5.1 - 2020-05-08
- 1.5.0 - 2020-05-06
- 1.4.0 - 2020-05-03
- 1.3.4 - 2020-04-30
- 1.3.3 - 2020-04-29
- 1.3.2 - 2020-04-27
- 1.3.1 - 2020-04-27
- 1.3.0 - 2020-04-27
- 1.2.0 - 2020-04-15
- 1.1.2 - 2020-04-14
- 1.1.1 - 2020-04-12
- 1.1.0 - 2020-04-08
- 1.0.5 - 2020-04-03
- 1.0.4 - 2020-04-03
- 1.0.3 - 2020-04-03
- 1.0.2 - 2020-04-02
- 1.0.1 - 2020-04-01
- 1.0.0 - 2020-03-31
- 0.28.0 - 2020-03-31
- 0.27.0 - 2020-03-31
- 0.26.1 - 2020-03-24
- 0.26.0 - 2020-03-19
- 0.25.1 - 2020-03-09
- 0.25.0 - 2020-02-27
- 0.24.5 - 2020-02-26
- 0.24.4 - 2020-02-22
- 0.24.3 - 2020-02-10
- 0.24.2 - 2020-02-09
- 0.24.1 - 2020-02-08
- 0.24.0 - 2020-02-07
- 0.23.7 - 2020-01-30
- 0.23.6 - 2020-01-28
- 0.23.5 - 2020-01-24
- 0.23.4 - 2020-01-22
- 0.23.3 - 2020-01-21
- 0.23.2 - 2020-01-16
- 0.23.1 - 2020-01-15
- 0.23.0 - 2020-01-15
- 0.22.0 - 2020-01-11
- 0.21.0 - 2019-12-20
- 0.20.5 - 2019-12-18
- 0.20.4 - 2019-12-17
- 0.20.3 - 2019-12-17
- 0.20.2 - 2019-12-14
- 0.20.1 - 2019-12-13
- 0.20.0 - 2019-12-12
- 0.19.1 - 2019-12-11
- 0.19.0 - 2019-12-02
- 0.18.1 - 2019-11-28
- 0.18.0 - 2019-11-27
- 0.17.0 - 2019-11-21
- 0.16.0 - 2019-11-19
- 0.15.0 - 2019-11-15
- 0.14.0 - 2019-11-09
- 0.13.2 - 2019-11-05
- 0.13.1 - 2019-11-05
- 0.13.0 - 2019-11-03
- 0.12.2 - 2019-10-30
- 0.12.1 - 2019-10-28
- 0.12.0 - 2019-10-28
- 0.11.0 - 2019-10-22
- 0.10.0 - 2019-10-14
- 0.9.0 - 2019-10-09
- 0.8.1 - 2019-10-04
- 0.8.0 - 2019-10-04
- 0.7.3 - 2019-09-30
- 0.7.2 - 2019-09-30
- 0.7.1 - 2019-09-27
- 0.7.0 - 2019-09-26
- 0.6.0 - 2019-09-24
- 0.5.0 - 2019-09-16
- 0.4.1 - 2019-09-11
- 0.4.0 - 2019-09-10
- 0.3.0 - 2019-09-06
- 0.2.0 - 2019-09-05
- 0.1.0 - 2019-06-28