Command Line Interface

Installing Schemathesis installs the schemathesis script to your virtualenv, which you can use to test your APIs


To see the full list of CLI options & commands use the --help option or check the Full list of CLI options.

Basic usage

To execute tests, use the st run command:

$ st run

With this command, Schemathesis will load the schema from and generate separate test sets for each operation in this schema. Each test set includes up to 100 test cases by default, depending on the operation definition.

For example, if your API schema has three operations, then you will see a similar output:

================ Schemathesis test session starts ===============
Schema location:
Base URL:
Specification version: Swagger 2.0
Workers: 1
Collected API operations: 3

GET /api/path_variable/{key} .                             [ 33%]
GET /api/success .                                         [ 66%]
POST /api/users/ .                                         [100%]

============================ SUMMARY ============================

Performed checks:
    not_a_server_error              201 / 201 passed       PASSED

======================= 3 passed in 1.77s =======================

The output style is inspired by pytest and provides necessary information about the loaded API schema, processed operations, found errors, and used checks.

By default, Schemathesis works with schemas that do not conform to the Open API spec, but you can enable schema validation with --validate-schema=true.


Schemathesis supports colorless output via the NO_COLOR <> environment variable or the --no-color CLI option.

Testing specific operations

By default, Schemathesis runs tests for all operations, but you can select specific operations with the following CLI options:

  • --endpoint / -E. Operation path;

  • --method / -M. HTTP method;

  • --tag / -T. Open API tag;

  • --operation-id / -O. operationId field value;

Each option accepts a case-insensitive regex string and could be used multiple times in a single command. For example, the following command will select all operations which paths start with /api/users:

$ st run -E ^/api/users


As filters are treated as regular expressions, ensure that they contain proper anchors. For example, /users/ will match /v1/users/orders/, but ^/users/$ will match only /users/.

If your API contains deprecated operations (that have deprecated: true in their definition), then you can skip them by passing --skip-deprecated-operations:

$ st run --skip-deprecated-operations ...

Overriding test data

You can set specific values for Open API parameters in test cases, such as query parameters, headers and cookies.

This is particularly useful for scenarios where specific parameter values are required for deeper testing. For instance, when dealing with values that represent data in a database, which Schemathesis might not automatically know or generate.

Each override follows the general form of --set-[part] name=value. For Open API, the [part] corresponds to the in value of a parameter which is query, header, cookie, or path. You can specify multiple overrides in a single command and each of them will be applied only to API operations that use such a parameter.

For example, to override a query parameter and path:

$ st run --set-query apiKey=secret --set-path user_id=42 ...

This command overrides the apiKey query parameter and user_id path parameter, using secret and 42 as their respective values in all applicable test cases.

Tests configuration

Schemathesis is built on top of the Hypothesis library and allows you to configure testing process in the same way.

We support all configuration options accepted by the hypothesis.settings decorator. All of them are prefixed with --hypothesis- and underscores are replaced with dashes, for example:

  • --hypothesis-max-examples=1000. Generate up to 1000 test cases per API operation;

  • --hypothesis-phases=explicit. Run only examples, specified explicitly in the API schema;

  • --hypothesis-suppress-health-check=too_slow. Disables the too_slow health check and makes Schemathesis continue testing even if it is considered too slow.

See the whole list of available options via the st run --help command and in the Hypothesis documentation.

How are responses checked?

For each API response received during the test, Schemathesis runs several checks to verify response conformance. By default, it runs only one check that raises an error if the checked response has a 5xx HTTP status code.

There are four built-in checks you can use via the –checks / -c CLI option:

  • not_a_server_error. The response has 5xx HTTP status;

  • status_code_conformance. The response status is not defined in the API schema;

  • content_type_conformance. The response content type is not defined in the API schema;

  • response_schema_conformance. The response content does not conform to the schema defined for this specific response;

  • response_headers_conformance. The response headers does not contain all defined headers.

To make Schemathesis perform all built-in checks use --checks all CLI option:

$ st run --checks all
================ Schemathesis test session starts ===============
Schema location:
Base URL:
Specification version: Swagger 2.0
Workers: 1
Collected API operations: 3

GET /api/path_variable/{key} .                             [ 33%]
GET /api/success .                                         [ 66%]
POST /api/users/ .                                         [100%]

============================ SUMMARY ============================

Performed checks:
    not_a_server_error              201 / 201 passed       PASSED
    status_code_conformance         201 / 201 passed       PASSED
    content_type_conformance        201 / 201 passed       PASSED
    response_schema_conformance     201 / 201 passed       PASSED

======================= 3 passed in 1.69s =======================

You can also define a list of checks to exclude using the --exclude-checks CLI option:

$ st run --checks all --exclude-checks not_a_server_error
================ Schemathesis test session starts ===============
Schema location:
Base URL:
Specification version: Swagger 2.0
Workers: 1
Collected API operations: 3

GET /api/path_variable/{key} .                             [ 33%]
GET /api/success .                                         [ 66%]
POST /api/users/ .                                         [100%]

============================ SUMMARY ============================

Performed checks:
    status_code_conformance         201 / 201 passed       PASSED
    content_type_conformance        201 / 201 passed       PASSED
    response_schema_conformance     201 / 201 passed       PASSED

======================= 3 passed in 1.69s =======================

Additionally, you can define the response time limit with --max-response-time. If any response will take longer than the provided value (in milliseconds) than it will indicate a failure:

$ st run --max-response-time=50 ...
================ Schemathesis test session starts ===============
Schema location:
Base URL:
Specification version: Swagger 2.0
Workers: 1
Collected API operations: 1

GET /api/slow F                                            [100%]

============================ FAILURES ===========================
__________________________ GET /api/slow ________________________
1. Test Case ID: 9Yjzd8

- Response time limit exceeded

    Actual: 101.92ms
    Limit: 50.00ms

[200] OK:

    `{"success": true}`

Reproduce with:

    curl -X GET

Or add this option to your command line parameters:
============================ SUMMARY ============================

Performed checks:
    not_a_server_error                  2 / 2 passed       PASSED
    max_response_time                   0 / 2 passed       FAILED

======================= 1 failed in 0.29s =======================

Concurrent testing

In some cases, you can speed up the testing process by distributing all tests among multiple threads via the -w / --workers option:

st run --workers 8

In the example above, all tests will be distributed among eight worker threads. Note that it is not guaranteed to improve performance because it depends on your application behavior.

Code samples style

To reproduce test failures Schemathesis generates code samples:


You can control these samples via the --code-sample-style CLI option. For example, passing curl will generate a cURL command like this:

curl -X GET

ASGI / WSGI support

Schemathesis natively supports testing of ASGI and WSGI compatible apps (e.g., Flask or FastAPI), which is significantly faster since it doesn’t involve the network.

To test your app with this approach, you need to pass a special “path” to your application instance via the --app CLI option. This path consists of two parts, separated by :. The first one is an importable path to the module with your app. The second one is the variable name that points to your app. Example: --app=project.wsgi:app.

Then your schema location could be:

  • A full URL;

  • An existing filesystem path;

  • In-app path with the schema.

For example:

st run --app=src.wsgi:app /swagger.json

NOTE. Depending on your setup, you might need to run this command with a custom PYTHONPATH environment variable like this:

$ PYTHONPATH=$(pwd) st run --app=src.wsgi:app /swagger.json

Storing and replaying test cases

It can be useful for debugging purposes to store all requests generated by Schemathesis and all responses from the app into a separate file. Schemathesis allows you to do this with the --cassette-path command-line option:

$ st run --cassette-path cassette.yaml

This command will create a new YAML file that will network interactions in VCR format. It might look like this:

command: 'st run --cassette-path=cassette.yaml'
recorded_with: 'Schemathesis 1.2.0'
- id: '0'
  status: 'FAILURE'
  seed: '1'
  elapsed: '0.00123'
  recorded_at: '2020-04-22T17:52:51.275318'
    - name: 'not_a_server_error'
      status: 'FAILURE'
      message: 'Received a response with 5xx status code: 500'
    uri: ''
    method: 'GET'
      encoding: 'utf-8'
      string: ''
      code: '500'
      message: 'Internal Server Error'
      encoding: 'utf-8'
      string: '500: Internal Server Error'
    http_version: '1.1'

Schemathesis provides the following extra fields:

  • command. Full CLI command used to run Schemathesis.

  • A numeric interaction ID within the current cassette.

  • http_interactions.status. Type of test outcome is one of SUCCESS, FAILURE. The status value is calculated from individual checks statuses - if any check failed, then the final status is FAILURE.

  • http_interactions.seed. The Hypothesis seed used in that particular case could be used as an argument to --hypothesis-seed CLI option to reproduce this request.

  • http_interactions.elapsed. Time in seconds that a request took.

  • http_interactions.checks. A list of executed checks and and their status.

  • http_interactions.data_generation_method. The way test case was generated - positive or negative.

  • http_interactions.thread_id. Unique integer that identifies the thread where a test case was used.

  • http_interactions.correlation_id. A unique ID which connects events that happen during testing of the same API operation

By default, payloads are converted to strings, but similar to the original Ruby’s VCR, Schemathesis supports preserving exact body bytes via the --cassette-preserve-exact-body-bytes option.

To work with the cassette, you could use yq or any similar tool. Show response body content of first failed interaction:

$ yq '.http_interactions.[] | select(.status == "FAILURE") | .response.body.string' foo.yaml | head -n 1
500: Internal Server Error

Check payload in requests to /api/upload_file:

$ yq '.http_interactions.[] | select(.request.uri == "").request.body.string'
Content-Disposition: form-data; name="data"; filename="data"


If you use --cassette-preserve-exact-body-bytes then you need to look for the base64_string field and decode it:

$ yq '.http_interactions.[] | select(.status == "FAILURE") | .response.body.base64_string' foo.yaml | head -n 1 | base64 -d
500: Internal Server Error

Saved cassettes can be replayed with st replay command. Additionally, you may filter what interactions to replay by these parameters:

  • id. Specific, unique ID;

  • status. Replay only interactions with this status (SUCCESS or FAILURE);

  • uri. A regular expression for request URI;

  • method. A regular expression for request method;

During replaying, Schemathesis will output interactions being replayed together with the response codes from the initial and current execution:

$ st replay foo.yaml --status=FAILURE
Replaying cassette: foo.yaml
Total interactions: 4005

  ID              : 0
  URI             :
  Old status code : 500
  New status code : 500

  ID              : 1
  URI             :
  Old status code : 500
  New status code : 500

JUnit support

It is possible to export test results to format, acceptable by such tools as Jenkins.

$ st run --junit-xml=/path/junit.xml

This command will create an XML at a given path, as in the example below.

<?xml version="1.0" ?>
<testsuites disabled="0" errors="0" failures="4" tests="4" time="1.7481054730014876">
        <testsuite disabled="0" errors="0" failures="4" name="schemathesis" skipped="0" tests="4" time="1.7481054730014876" hostname="midgard">
                <testcase name="GET /response-conformance/missing-field" time="0.859204">
                        <failure type="failure" message="1. Test Case ID: JA63GZ

- Response violates schema

    'age' is a required property


            &quot;type&quot;: &quot;object&quot;,
            &quot;properties&quot;: {
                &quot;id&quot;: {
                    &quot;type&quot;: &quot;string&quot;
                &quot;name&quot;: {
                    &quot;type&quot;: &quot;string&quot;
                &quot;age&quot;: {
                    &quot;type&quot;: &quot;integer&quot;
            &quot;required&quot;: [


            &quot;id&quot;: &quot;123&quot;,
            &quot;name&quot;: &quot;Alice&quot;

[200] OK:


Reproduce with:

    curl -X GET"/>
                <testcase name="GET /response-conformance/malformed-json" time="0.068179">
                        <failure type="failure" message="1. Test Case ID: Vn5hfI

- JSON deserialization error

    Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

[200] OK:

    `{success: true}`

Reproduce with:

    curl -X GET"/>
                <testcase name="GET /response-conformance/undocumented-status-code" time="0.756355">
                        <failure type="failure" message="1. Test Case ID: jm2nOs

- Undocumented HTTP status code

    Received: 404
    Documented: 200, 400

[404] Not Found:

    `{&quot;error&quot;:&quot;Not Found&quot;}`

Reproduce with:

    curl -X GET ''"/>
                <testcase name="GET /response-conformance/incorrect-content-type" time="0.064367">
                        <failure type="failure" message="1. Test Case ID: Sveexo

- Undocumented Content-Type

    Received: text/plain
    Documented: application/json

[200] OK:


Reproduce with:

    curl -X GET"/>

Base URL configuration

If your Open API schema defines servers (or basePath in Open API 2.0), these values will be used to construct a full operation URL during testing. In the case of Open API 3, the first value from servers will be used.

However, you may want to run tests against a different base URL. To do this, you need to pass the --base-url option in CLI or provide base_url argument to a loader/runner if you use Schemathesis in your code:

st run --base-url=

And if your schema defines servers like this:

  - url:{basePath}
        default: v1

Then the tests will be executed against /api/v2 base path.

The --base-url argument is also used if you wish to load the OpenAPI specification from a local file.

st run --base-url= path/to/openapi.json

Extending CLI

To fit Schemathesis to your workflows, you might want to extend it with your custom checks or setup environment before the test run.

Extensions should be placed in a separate Python module. Then, Schemathesis should be informed about this module via the SCHEMATHESIS_HOOKS environment variable:

export SCHEMATHESIS_HOOKS=myproject.tests.hooks
st run

Also, depending on your setup, you might need to run this command with a custom PYTHONPATH environment variable like this:

export PYTHONPATH=$(pwd)
export SCHEMATHESIS_HOOKS=myproject.tests.hooks
st run

The passed value will be treated as an importable Python path and imported before the test run.


You can find more details on how to extend Schemathesis in the Extending Schemathesis section.

Registering custom checks

To use your custom checks with Schemathesis CLI, you need to register them via the schemathesis.check decorator:

import schemathesis

def new_check(response, case):
    # some awesome assertions!

The registered check should accept a response with requests.Response / schemathesis.utils.WSGIResponse type and case with schemathesis.models.Case type. This code should be placed in the module you pass to the SCHEMATHESIS_HOOKS environment variable.

Then your checks will be available in Schemathesis CLI, and you can use them via the -c command-line option.

$ SCHEMATHESIS_HOOKS=module.with.checks
$ st run -c new_check

Additionally, checks may return True to skip the check under certain conditions. For example, you may only want to run checks when the response code is 200.

import schemathesis

def conditional_check(response, case):
    if response.status_code == 200:
        ...  # some awesome assertions!
        # check not relevant to this response, skip test
        return True

Skipped check calls will not be reported in the run summary.


Learn more about writing custom checks here.

Rate limiting

APIs implement rate limiting to prevent misuse of their resources. Schemathesis CLI’s --rate-limit option can be used to set the maximum number of requests per second, minute, hour, or day during testing to avoid hitting these limits.

# 3 requests per second
st run --rate-limit=3/s
# 100 requests per minute
st run --rate-limit=100/m
# 1000 requests per hour
st run --rate-limit=1000/h
# 10000 requests per day
st run --rate-limit=10000/d


If Schemathesis produces an internal error, its traceback is hidden. To show error tracebacks in the CLI output, use the --show-trace option.

Additionally you can dump all internal events to a JSON Lines file with the --debug-output-file CLI option.

Running CLI via Docker

Schemathesis CLI is also available as a Docker image:

docker run schemathesis/schemathesis:stable \

To run it against the localhost server, add --network=host parameter:

docker run --network="host" schemathesis/schemathesis:stable \

If your API spec is stored in a file, you could use it too by specifying a Docker volume:

docker run -v $(pwd):/mnt schemathesis/schemathesis:stable \
    run /mnt/spec.json

In the example above, the spec.json file from the current working directory is shared with the Schemathesis container. Note, that $(pwd) is shell-specific and works in sh / bash / zsh, but could be different in e.g. PowerShell.

When running from Docker, by default color output is not present. You can use --force-color if you know that the host’s terminal supports colors. Note that --force-color and --no-color are not compatible with each other.


See Docker volumes documentation for more information.

Docker on MacOS

Due to the networking behavior of Docker on MacOS, the containerized application cannot directly reach localhost of the host machine. To address this, MacOS users should use the special DNS name host.docker.internal when referring to the host within Docker.

docker run schemathesis/schemathesis:stable \
    run http://host.docker.internal:8080/swagger.json


See Docker on MacOS documentation for more details

Full list of CLI options


Automated API testing employing fuzzing techniques for OpenAPI and GraphQL.

schemathesis [OPTIONS] COMMAND [ARGS]...



Show the version and exit.


Run tests against an API using a specified SCHEMA.

[Required] SCHEMA: Path to an OpenAPI (.json, .yml) or GraphQL SDL file, or a URL pointing to such specifications.

[Optional] API_NAME: Identifier for uploading test data to

schemathesis run [OPTIONS] SCHEMA [API_NAME]


-c, --checks <checks>

Specifies the validation checks to apply to API responses. Provide a comma-separated list of checks such as ‘not_a_server_error,status_code_conformance’, etc. Default is ‘not_a_server_error’.




not_a_server_error | status_code_conformance | content_type_conformance | response_headers_conformance | response_schema_conformance | all

--exclude-checks <exclude_checks>

Specifies the validation checks to skip during testing. Provide a comma-separated list of checks you wish to bypass.



not_a_server_error | status_code_conformance | content_type_conformance | response_headers_conformance | response_schema_conformance

-D, --data-generation-method <data_generation_methods>

Specifies the approach Schemathesis uses to generate test data. Use ‘positive’ for valid data, ‘negative’ for invalid data, or ‘all’ for both. Default is ‘positive’.




positive | negative | all

--max-response-time <max_response_time>

Sets a custom time limit for API response times. The test will fail if a response time exceeds this limit. Provide the time in milliseconds.

-t, --target <targets>

Guides input generation to values more likely to expose bugs via targeted property-based testing.



response_time | all

-x, --exitfirst

Terminates the test suite immediately upon the first failure or error encountered.



--max-failures <max_failures>

Terminates the test suite after reaching a specified number of failures or errors.


Simulates test execution without making any actual requests, useful for validating data generation.

-a, --auth <auth>

Provides the server authentication details in the ‘USER:PASSWORD’ format.

-A, --auth-type <auth_type>

Specifies the authentication method. Default is ‘basic’.




basic | digest

--set-query <set_query>

OpenAPI: Override a specific query parameter by specifying ‘parameter=value’

--set-header <set_header>

OpenAPI: Override a specific header parameter by specifying ‘parameter=value’

OpenAPI: Override a specific cookie parameter by specifying ‘parameter=value’

--set-path <set_path>

OpenAPI: Override a specific path parameter by specifying ‘parameter=value’

-H, --header <headers>

Adds a custom HTTP header to all API requests. Format: ‘Header-Name: Value’.

-E, --endpoint <endpoints>

API operation path pattern (e.g., users/d+).

-M, --method <methods>

HTTP method (e.g., GET, POST).

-T, --tag <tags>

Schema tag pattern.

-O, --operation-id <operation_ids>

OpenAPI operationId pattern.

-w, --workers <workers_num>

Sets the number of concurrent workers for testing. Auto-adjusts if ‘auto’ is specified.




auto | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64

-b, --base-url <base_url>

Provides the base URL of the API, required when schema is provided as a file.

--app <app>

Specifies the WSGI/ASGI application under test, provided as an importable Python path.

--wait-for-schema <wait_for_schema>

Maximum duration, in seconds, to wait for the API schema to become available.

--request-timeout <request_timeout>

Sets a timeout limit, in milliseconds, for each network request during tests.

--request-proxy <request_proxy>

Set the proxy for all network requests.

--request-tls-verify <request_tls_verify>

Configures TLS certificate verification for server requests. Can specify path to CA_BUNDLE for custom certs.



--request-cert <request_cert>

File path of unencrypted client certificate for authentication. The certificate can be bundled with a private key (e.g. PEM) or the private key can be provided with the –request-cert-key argument.

--request-cert-key <request_cert_key>

Specifies the file path of the private key for the client certificate.

--validate-schema <validate_schema>

Toggles validation of incoming payloads against the defined API schema. Set to ‘True’ to enable or ‘False’ to disable. Default is ‘False’.




Exclude deprecated API operations from testing.



--junit-xml <junit_xml>

Outputs a JUnit-XML style report at the specified file path.

--report <report_value>

Specifies how the generated report should be handled. If used without an argument, the report data will automatically be uploaded to If a file name is provided, the report will be stored in that file. The report data, consisting of a tar gz file with multiple JSON files, is subject to change.

--debug-output-file <debug_output_file>

Saves debugging information in a JSONL format at the specified file path.


Displays complete traceback information for internal errors.



--code-sample-style <code_sample_style>

Selects the code sample style for reproducing failures.


python | curl

--cassette-path <cassette_path>

Saves the test outcomes in a VCR-compatible format.


Retains exact byte sequence of payloads in cassettes, encoded as base64.

--fixups <fixups>

Applies compatibility adjustments like ‘fast_api’, ‘utf8_bom’.


fast_api | utf8_bom | all

--rate-limit <rate_limit>

Specifies a rate limit for test requests in ‘<limit>/<duration>’ format. Example - 100/m for 100 requests per minute.

--stateful <stateful>

Enables or disables stateful testing features.


none | links

--force-schema-version <force_schema_version>

Forces the schema to be interpreted as a particular OpenAPI version.


20 | 30

--sanitize-output <sanitize_output>

Enable or disable automatic output sanitization to obscure sensitive data.




Forces the generation of unique test cases.




Enables support for the ‘uuid’ string format in OpenAPI.




Enables generation of random examples for API operations that do not have explicit examples defined.



--hypothesis-database <hypothesis_database>

Configures storage for examples discovered by Hypothesis. Use ‘none’ to disable, ‘:memory:’ for temporary storage, or specify a file path for persistent storage.

--hypothesis-deadline <hypothesis_deadline>

Sets a time limit for each test case generated by Hypothesis, in milliseconds. Exceeding this limit will cause the test to fail.


Enables deterministic mode in Hypothesis, which eliminates random variation between test runs.

--hypothesis-max-examples <hypothesis_max_examples>

Sets the cap on the number of examples generated by Hypothesis for each API method/path pair.

--hypothesis-phases <hypothesis_phases>

Specifies which testing phases to execute.


explicit | reuse | generate | target | shrink

--hypothesis-no-phases <hypothesis_no_phases>

Specifies which testing phases to exclude from execution.


explicit | reuse | generate | target | shrink

--hypothesis-report-multiple-bugs <hypothesis_report_multiple_bugs>

If set, only the most easily reproducible exception will be reported when multiple issues are found.

--hypothesis-seed <hypothesis_seed>

Sets a seed value for Hypothesis, ensuring reproducibility across test runs.

--hypothesis-suppress-health-check <hypothesis_suppress_health_check>

Disables specified health checks from Hypothesis like ‘data_too_large’, ‘filter_too_much’, etc. Provide a comma-separated list


data_too_large | filter_too_much | too_slow | large_base_example

--hypothesis-verbosity <hypothesis_verbosity>

Controls the verbosity level of Hypothesis output.


quiet | normal | verbose | debug


Disable ANSI color escape codes.


Explicitly tells to enable ANSI color escape codes.

--experimental <experiments>

Enable experimental support for specific features.


openapi-3.1 | schema-analysis

--generation-allow-x00 <generation_allow_x00>

Determines whether to allow the generation of `` bytes within strings.



--generation-codec <generation_codec>

Specifies the codec used for generating strings.

--schemathesis-io-token <schemathesis_io_token> authentication token.

--schemathesis-io-url <schemathesis_io_url> base URL.

--schemathesis-io-telemetry <schemathesis_io_telemetry>

Controls whether you send anonymized CLI usage data to along with your report.



--hosts-file <hosts_file>

Path to a file to store the auth configuration.

-v, --verbosity

Increase verbosity of the output.



Required argument


Optional argument

Environment variables


Provide a default for API_NAME


Provide a default for --base-url


Provide a default for --wait-for-schema


Provide a default for --report


Provide a default for --schemathesis-io-token


Provide a default for --schemathesis-io-url


Provide a default for --schemathesis-io-telemetry


Provide a default for --hosts-file