Command Line Interface

Installing Schemathesis installs the schemathesis script to your virtualenv, which you can use to test your APIs

Note

To see the full list of CLI options & commands use the --help option or check the Full list of CLI options.

Basic usage

To execute tests, use the st run command:

$ st run https://example.schemathesis.io/openapi.json

With this command, Schemathesis will load the schema from https://example.schemathesis.io/openapi.json and generate separate test sets for each operation in this schema. Each test set includes up to 100 test cases by default, depending on the operation definition.

For example, if your API schema has three operations, then you will see a similar output:

================ Schemathesis test session starts ===============
Schema location: http://127.0.0.1:8081/schema.yaml
Base URL: http://127.0.0.1:8081/api
Specification version: Swagger 2.0
Workers: 1
Collected API operations: 3

GET /api/path_variable/{key} .                             [ 33%]
GET /api/success .                                         [ 66%]
POST /api/users/ .                                         [100%]

============================ SUMMARY ============================

Performed checks:
    not_a_server_error              201 / 201 passed       PASSED

======================= 3 passed in 1.77s =======================

The output style is inspired by pytest and provides necessary information about the loaded API schema, processed operations, found errors, and used checks.

By default, Schemathesis works with schemas that do not conform to the Open API spec, but you can enable schema validation with --validate-schema=true.

Note

Schemathesis supports colorless output via the NO_COLOR <https://no-color.org/> environment variable or the --no-color CLI option.

Narrowing the testing scope

By default, Schemathesis tests all operations in your API. However, you can fine-tune your test scope with various CLI options to include or exclude specific operations based on paths, methods, names, tags, and operation IDs.

Include and Exclude Options

Use the following format to include or exclude specific operations in your tests:

--{include,exclude}-{path,method,name,tag,operation-id} TEXT
--{include,exclude}-{path,method,name,tag,operation-id}-regex TEXT

The -regex suffix enables regular expression matching for the specified criteria. For example, --include-path-regex '^/users' matches any path starting with /users. Without this suffix (e.g., --include-path '/users'), the option performs an exact match. Use regex for flexible pattern matching and the non-regex version for precise, literal matching.

Additionally, you can exclude deprecated operations with:

--exclude-deprecated

Note

The name property in Schemathesis refers to the full operation name. For Open API, it is formatted as HTTP_METHOD PATH (e.g., GET /users). For GraphQL, it follows the pattern OperationType.field (e.g., Query.getBookings or Mutation.updateOrder).

Important

For GraphQL schemas, Schemathesis only supports filtration by the name property.

You also can filter API operations by an expression over operation’s definition:

st run --include-by="/x-property == 42" https://example.schemathesis.io/openapi.json

The expression above will select only operations with the x-property field equal to 42. Expressions have the following form:

"<pointer> <operator> <value>"

<pointer> is a JSON Pointer to the value in the operation definition.
<operator> is one of the following: ==, !=.
<value> is a JSON value to compare with. If it is not a valid JSON value, it is treated as a string.

Examples

Include operations with paths starting with /api/users:

$ st run --include-path-regex '^/api/users' https://example.schemathesis.io/openapi.json

Exclude POST method operations:

$ st run --exclude-method 'POST' https://example.schemathesis.io/openapi.json

Include operations with the admin tag:

$ st run --include-tag 'admin' https://example.schemathesis.io/openapi.json

Exclude deprecated operations:

$ st run --exclude-deprecated https://example.schemathesis.io/openapi.json

Include GET /users and POST /orders:

$ st run \
  --include-name 'GET /users' \
  --include-name 'POST /orders' \
  https://example.schemathesis.io/openapi.json

Include queries for getBook and updateBook operations in GraphQL:

$ st run \
  --include-name 'Query.getBook' \
  --include-name 'Mutation.updateBook' \
  https://example.schemathesis.io/graphql

Overriding test data

You can set specific values for Open API parameters in test cases, such as query parameters, headers and cookies.

This is particularly useful for scenarios where specific parameter values are required for deeper testing. For instance, when dealing with values that represent data in a database, which Schemathesis might not automatically know or generate.

Each override follows the general form of --set-[part] name=value. For Open API, the [part] corresponds to the in value of a parameter which is query, header, cookie, or path. You can specify multiple overrides in a single command and each of them will be applied only to API operations that use such a parameter.

For example, to override a query parameter and path:

$ st run --set-query apiKey=secret --set-path user_id=42 ...

This command overrides the apiKey query parameter and user_id path parameter, using secret and 42 as their respective values in all applicable test cases.

Tests configuration

Schemathesis is built on top of the Hypothesis library and allows you to configure testing process in the same way.

We support all configuration options accepted by the hypothesis.settings decorator. All of them are prefixed with --hypothesis- and underscores are replaced with dashes, for example:

--hypothesis-max-examples=1000. Generate up to 1000 test cases per API operation;
--hypothesis-phases=explicit. Run only examples, specified explicitly in the API schema;
--hypothesis-suppress-health-check=too_slow. Disables the too_slow health check and makes Schemathesis continue testing even if it is considered too slow.

See the whole list of available options via the st run --help command and in the Hypothesis documentation.

How are responses checked?

For each API response received during the test, Schemathesis runs several checks to verify response conformance. By default, it runs only one check that raises an error if the checked response has a 5xx HTTP status code.

There are four built-in checks you can use via the –checks / -c CLI option:

not_a_server_error. The response has 5xx HTTP status;
status_code_conformance. The response status is not defined in the API schema;
content_type_conformance. The response content type is not defined in the API schema;
response_schema_conformance. The response content does not conform to the schema defined for this specific response;
negative_data_rejection. The API accepts data that is invalid according to the schema;
response_headers_conformance. The response headers do not contain all defined headers or do not conform to their respective schemas.
use_after_free. The API returned a non-404 response a successful DELETE operation on a resource. NOTE: Only enabled for new-style stateful testing.
ensure_resource_availability. Freshly created resource is not available in related API operations. NOTE: Only enabled for new-style stateful testing.
ignored_auth. The API operation does not check the specified authentication.

To make Schemathesis perform all built-in checks use --checks all CLI option:

$ st run --checks all https://example.schemathesis.io/openapi.json
================ Schemathesis test session starts ===============
Schema location: https://example.schemathesis.io/openapi.json
Base URL: http://api.com/
Specification version: Swagger 2.0
Workers: 1
Collected API operations: 3

GET /api/path_variable/{key} .                             [ 33%]
GET /api/success .                                         [ 66%]
POST /api/users/ .                                         [100%]

============================ SUMMARY ============================

Performed checks:
    not_a_server_error              201 / 201 passed       PASSED
    status_code_conformance         201 / 201 passed       PASSED
    content_type_conformance        201 / 201 passed       PASSED
    response_schema_conformance     201 / 201 passed       PASSED

======================= 3 passed in 1.69s =======================

You can also define a list of checks to exclude using the --exclude-checks CLI option:

$ st run --checks all --exclude-checks not_a_server_error https://example.schemathesis.io/openapi.json
================ Schemathesis test session starts ===============
Schema location: https://example.schemathesis.io/openapi.json
Base URL: http://api.com/
Specification version: Swagger 2.0
Workers: 1
Collected API operations: 3

GET /api/path_variable/{key} .                             [ 33%]
GET /api/success .                                         [ 66%]
POST /api/users/ .                                         [100%]

============================ SUMMARY ============================

Performed checks:
    status_code_conformance         201 / 201 passed       PASSED
    content_type_conformance        201 / 201 passed       PASSED
    response_schema_conformance     201 / 201 passed       PASSED

======================= 3 passed in 1.69s =======================

Additionally, you can define the response time limit with --max-response-time. If any response will take longer than the provided value (in milliseconds) than it will indicate a failure:

$ st run --max-response-time=50 ...
================ Schemathesis test session starts ===============
Schema location: https://example.schemathesis.io/openapi.json
Base URL: https://example.schemathesis.io/api
Specification version: Swagger 2.0
Workers: 1
Collected API operations: 1

GET /api/slow F                                            [100%]

============================ FAILURES ===========================
__________________________ GET /api/slow ________________________
1. Test Case ID: 9Yjzd8

- Response time limit exceeded

    Actual: 101.92ms
    Limit: 50.00ms

[200] OK:

    `{"success": true}`

Reproduce with:

    curl -X GET http://127.0.0.1:8081/api/slow

Or add this option to your command line parameters:
    --hypothesis-seed=103697217851787640556597810346466192664
============================ SUMMARY ============================

Performed checks:
    not_a_server_error                  2 / 2 passed       PASSED
    max_response_time                   0 / 2 passed       FAILED

======================= 1 failed in 0.29s =======================

Concurrent testing

In some cases, you can speed up the testing process by distributing all tests among multiple threads via the -w / --workers option:

st run --workers 8 https://example.com/api/swagger.json

In the example above, all tests will be distributed among eight worker threads. Note that it is not guaranteed to improve performance because it depends on your application behavior.

Code samples style

To reproduce test failures Schemathesis generates cURL commands:

curl -X GET http://127.0.0.1:8081/api/failure

You can control these samples via the --code-sample-style CLI option. For example, passing python will generate a Python snippet like this:

requests.get("http://127.0.0.1:8081/api/failure")

Output verbosity

Sometimes the output contains parts of your API schema or responses in order to provide more context. By default, Schemathesis truncates these parts to make the output more readable. However, you can control this behavior with:

--output-truncate=false. Disables schema and response truncation in error messages.

ASGI / WSGI support

Schemathesis natively supports testing of ASGI and WSGI compatible apps (e.g., Flask or FastAPI), which is significantly faster since it doesn’t involve the network.

To test your app with this approach, you need to pass a special “path” to your application instance via the --app CLI option. This path consists of two parts, separated by :. The first one is an importable path to the module with your app. The second one is the variable name that points to your app. Example: --app=project.wsgi:app.

Then your schema location could be:

A full URL;
An existing filesystem path;
In-app path with the schema.

For example:

st run --app=src.wsgi:app /swagger.json

NOTE. Depending on your setup, you might need to run this command with a custom PYTHONPATH environment variable like this:

$ PYTHONPATH=$(pwd) st run --app=src.wsgi:app /swagger.json

Storing and replaying test cases

It can be useful for debugging purposes to store all requests generated by Schemathesis and all responses from the app into a separate file. Schemathesis allows you to do this with the --cassette-path command-line option:

$ st run --cassette-path cassette.yaml http://127.0.0.1/schema.yaml

Schemathesis supports VCR and HAR formats and stores all network interactions in a YAML file.

HAR format

HTTP Archive (HAR) is a JSON-based format used for tracking HTTP requests and responses. Schemathesis uses a simplified version of this format that does not include page-related information:

{
    "log": {
        "version": "1.2",
        "creator": {
            "name": "harfile",
            "version": "0.2.0"
        },
        "browser": {
            "name": "",
            "version": ""
        },
        "entries": [
            {
                "startedDateTime": "2024-06-29T20:10:29.254107+02:00",
                "time": 0.88,
                "request": {"method": "GET", "url": "http://127.0.0.1:8081/api/basic", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "User-Agent", "value": "schemathesis/3.30.4"}, {"name": "Accept-Encoding", "value": "gzip, deflate"}, {"name": "Accept", "value": "*/*"}, {"name": "Connection", "value": "keep-alive"}, {"name": "Authorization", "value": "[Filtered]"}, {"name": "X-Schemathesis-TestCaseId", "value": "ScU88H"}], "queryString": [], "headersSize": 164, "bodySize": 0},
                "response": {"status": 401, "statusText": "Unauthorized", "httpVersion": "HTTP/1.1", "cookies": [], "headers": [{"name": "Content-Type", "value": "application/json; charset=utf-8"}, {"name": "Content-Length", "value": "26"}, {"name": "Date", "value": "Sat, 29 Jun 2024 18:10:29 GMT"}, {"name": "Server", "value": "Python/3.11 aiohttp/3.9.3"}], "content": {"size": 26, "mimeType": "application/json; charset=utf-8", "text": "{\"detail\": \"Unauthorized\"}"}, "redirectURL": "", "headersSize": 139, "bodySize": 26},
                "timings": {"send": 0, "wait": 0, "receive": 0.88, "blocked": 0, "dns": 0, "connect": 0, "ssl": 0},
                "cache": {}
            },
            {

To view the content of a HAR file, you can use this HAR viewer.

VCR format

The content of a VCR cassette looks like this:

command: 'st run --cassette-path=cassette.yaml http://127.0.0.1/schema.yaml'
recorded_with: 'Schemathesis 1.2.0'
http_interactions:
- id: '0'
  status: 'FAILURE'
  seed: '1'
  elapsed: '0.00123'
  recorded_at: '2020-04-22T17:52:51.275318'
  checks:
    - name: 'not_a_server_error'
      status: 'FAILURE'
      message: 'Received a response with 5xx status code: 500'
  request:
    uri: 'http://127.0.0.1/api/failure'
    method: 'GET'
    headers:
      ...
    body:
      encoding: 'utf-8'
      string: ''
  response:
    status:
      code: '500'
      message: 'Internal Server Error'
    headers:
      ...
    body:
      encoding: 'utf-8'
      string: '500: Internal Server Error'
    http_version: '1.1'

Schemathesis provides the following extra fields:

command. Full CLI command used to run Schemathesis.
http_interactions.id. A numeric interaction ID within the current cassette.
http_interactions.status. Type of test outcome is one of SUCCESS, FAILURE. The status value is calculated from individual checks statuses - if any check failed, then the final status is FAILURE.
http_interactions.seed. The Hypothesis seed used in that particular case could be used as an argument to --hypothesis-seed CLI option to reproduce this request.
http_interactions.elapsed. Time in seconds that a request took.
http_interactions.checks. A list of executed checks and and their status.
http_interactions.data_generation_method. The way test case was generated - positive or negative.
http_interactions.thread_id. Unique integer that identifies the thread where a test case was used.
http_interactions.correlation_id. A unique ID which connects events that happen during testing of the same API operation

By default, payloads are converted to strings, but similar to the original Ruby’s VCR, Schemathesis supports preserving exact body bytes via the --cassette-preserve-exact-body-bytes option.

To work with the cassette, you could use yq or any similar tool. Show response body content of first failed interaction:

$ yq '.http_interactions.[] | select(.status == "FAILURE") | .response.body.string' foo.yaml | head -n 1
500: Internal Server Error

Check payload in requests to /api/upload_file:

$ yq '.http_interactions.[] | select(.request.uri == "http://127.0.0.1:8081/api/upload_file").request.body.string'
--7d4db38ad065994d913cb02b2982e3ba
Content-Disposition: form-data; name="data"; filename="data"


--7d4db38ad065994d913cb02b2982e3ba--

If you use --cassette-preserve-exact-body-bytes then you need to look for the base64_string field and decode it:

$ yq '.http_interactions.[] | select(.status == "FAILURE") | .response.body.base64_string' foo.yaml | head -n 1 | base64 -d
500: Internal Server Error

Saved cassettes can be replayed with st replay command. Additionally, you may filter what interactions to replay by these parameters:

id. Specific, unique ID;
status. Replay only interactions with this status (SUCCESS or FAILURE);
uri. A regular expression for request URI;
method. A regular expression for request method;

During replaying, Schemathesis will output interactions being replayed together with the response codes from the initial and current execution:

$ st replay foo.yaml --status=FAILURE
Replaying cassette: foo.yaml
Total interactions: 4005

  ID              : 0
  URI             : http://127.0.0.1:8081/api/failure
  Old status code : 500
  New status code : 500

  ID              : 1
  URI             : http://127.0.0.1:8081/api/failure
  Old status code : 500
  New status code : 500

JUnit support

It is possible to export test results to format, acceptable by such tools as Jenkins.

$ st run --junit-xml=/path/junit.xml http://127.0.0.1/schema.yaml

This command will create an XML at a given path, as in the example below.

<?xml version="1.0" ?>
<testsuites disabled="0" errors="0" failures="4" tests="4" time="1.7481054730014876">
        <testsuite disabled="0" errors="0" failures="4" name="schemathesis" skipped="0" tests="4" time="1.7481054730014876" hostname="midgard">
                <testcase name="GET /response-conformance/missing-field" time="0.859204">
                        <failure type="failure" message="1. Test Case ID: JA63GZ

- Response violates schema

    'age' is a required property

    Schema:

        {
            &quot;type&quot;: &quot;object&quot;,
            &quot;properties&quot;: {
                &quot;id&quot;: {
                    &quot;type&quot;: &quot;string&quot;
                },
                &quot;name&quot;: {
                    &quot;type&quot;: &quot;string&quot;
                },
                &quot;age&quot;: {
                    &quot;type&quot;: &quot;integer&quot;
                }
            },
            &quot;required&quot;: [
                &quot;id&quot;,
                &quot;name&quot;,
                &quot;age&quot;
            ]
        }

    Value:

        {
            &quot;id&quot;: &quot;123&quot;,
            &quot;name&quot;: &quot;Alice&quot;
        }

[200] OK:

    `{&quot;id&quot;:&quot;123&quot;,&quot;name&quot;:&quot;Alice&quot;}`

Reproduce with:

    curl -X GET https://example.schemathesis.io/response-conformance/missing-field"/>
                </testcase>
                <testcase name="GET /response-conformance/malformed-json" time="0.068179">
                        <failure type="failure" message="1. Test Case ID: Vn5hfI

- JSON deserialization error

    Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

[200] OK:

    `{success: true}`

Reproduce with:

    curl -X GET https://example.schemathesis.io/response-conformance/malformed-json"/>
                </testcase>
                <testcase name="GET /response-conformance/undocumented-status-code" time="0.756355">
                        <failure type="failure" message="1. Test Case ID: jm2nOs

- Undocumented HTTP status code

    Received: 404
    Documented: 200, 400

[404] Not Found:

    `{&quot;error&quot;:&quot;Not Found&quot;}`

Reproduce with:

    curl -X GET 'https://example.schemathesis.io/response-conformance/undocumented-status-code?id=1'"/>
                </testcase>
                <testcase name="GET /response-conformance/incorrect-content-type" time="0.064367">
                        <failure type="failure" message="1. Test Case ID: Sveexo

- Undocumented Content-Type

    Received: text/plain
    Documented: application/json

[200] OK:

    `Success!`

Reproduce with:

    curl -X GET https://example.schemathesis.io/response-conformance/incorrect-content-type"/>
                </testcase>
        </testsuite>
</testsuites>

Base URL configuration

If your Open API schema defines servers (or basePath in Open API 2.0), these values will be used to construct a full operation URL during testing. In the case of Open API 3, the first value from servers will be used.

However, you may want to run tests against a different base URL. To do this, you need to pass the --base-url option in CLI or provide base_url argument to a loader/runner if you use Schemathesis in your code:

st run --base-url=http://127.0.0.1:8080/api/v2 http://production.com/api/openapi.json

And if your schema defines servers like this:

servers:
  - url: https://production.com/api/{basePath}
    variables:
      basePath:
        default: v1

Then the tests will be executed against /api/v2 base path.

The --base-url argument is also used if you wish to load the OpenAPI specification from a local file.

st run --base-url=http://127.0.0.1:8080/api/v1 path/to/openapi.json

Extending CLI

To fit Schemathesis to your workflows, you might want to extend it with your custom checks or setup environment before the test run.

Extensions should be placed in a separate Python module. Then, Schemathesis should be informed about this module via the SCHEMATHESIS_HOOKS environment variable:

export SCHEMATHESIS_HOOKS=myproject.tests.hooks
st run http://127.0.0.1/openapi.yaml

Also, depending on your setup, you might need to run this command with a custom PYTHONPATH environment variable like this:

export PYTHONPATH=$(pwd)
export SCHEMATHESIS_HOOKS=myproject.tests.hooks
st run https://example.com/api/swagger.json

The passed value will be treated as an importable Python path and imported before the test run.

Note

You can find more details on how to extend Schemathesis in the Extending Schemathesis section.

Registering custom checks

To use your custom checks with Schemathesis CLI, you need to register them via the schemathesis.check decorator:

import schemathesis


@schemathesis.check
def new_check(ctx, response, case):
    # some awesome assertions!
    pass

The registered check should accept ctx, a response with requests.Response / schemathesis.utils.WSGIResponse type and case with schemathesis.models.Case type. This code should be placed in the module you pass to the SCHEMATHESIS_HOOKS environment variable.

Then your checks will be available in Schemathesis CLI, and you can use them via the -c command-line option.

$ SCHEMATHESIS_HOOKS=module.with.checks
$ st run -c new_check https://example.com/api/swagger.json

Additionally, checks may return True to skip the check under certain conditions. For example, you may only want to run checks when the response code is 200.

import schemathesis


@schemathesis.check
def conditional_check(ctx, response, case):
    if response.status_code == 200:
        ...  # some awesome assertions!
    else:
        # check not relevant to this response, skip test
        return True

Skipped check calls will not be reported in the run summary.

Note

Learn more about writing custom checks here.

Rate limiting

APIs implement rate limiting to prevent misuse of their resources. Schemathesis CLI’s --rate-limit option can be used to set the maximum number of requests per second, minute, hour, or day during testing to avoid hitting these limits.

# 3 requests per second
st run --rate-limit=3/s
# 100 requests per minute
st run --rate-limit=100/m
# 1000 requests per hour
st run --rate-limit=1000/h
# 10000 requests per day
st run --rate-limit=10000/d

Debugging

If Schemathesis produces an internal error, its traceback is hidden. To show error tracebacks in the CLI output, use the --show-trace option.

Additionally you can dump all internal events to a JSON Lines file with the --debug-output-file CLI option.

Running CLI via Docker

Schemathesis CLI is also available as a Docker image:

docker run schemathesis/schemathesis:stable \
    run http://api.com/schema.json

To run it against the localhost server, add --network=host parameter:

docker run --network="host" schemathesis/schemathesis:stable \
    run http://127.0.0.1/schema.json

If your API spec is stored in a file, you could use it too by specifying a Docker volume:

docker run -v $(pwd):/app schemathesis/schemathesis:stable \
    run /app/spec.json

In the example above, the spec.json file from the current working directory is shared with the Schemathesis container. Note, that $(pwd) is shell-specific and works in sh / bash / zsh, but could be different in e.g. PowerShell.

When running from Docker, by default color output is not present. You can use --force-color if you know that the host’s terminal supports colors. Note that --force-color and --no-color are not compatible with each other.

Note

See Docker volumes documentation for more information.

Docker on MacOS

Due to the networking behavior of Docker on MacOS, the containerized application cannot directly reach localhost of the host machine. To address this, MacOS users should use the special DNS name host.docker.internal when referring to the host within Docker.

docker run schemathesis/schemathesis:stable \
    run http://host.docker.internal:8080/swagger.json

Note

See Docker on MacOS documentation for more details

Full list of CLI options

schemathesis

Property-based API testing for OpenAPI and GraphQL.

schemathesis [OPTIONS] COMMAND [ARGS]...

Options

--version: Show the version and exit.

run

Run tests against an API using a specified SCHEMA.

[Required] SCHEMA: Path to an OpenAPI (.json, .yml) or GraphQL SDL file, or a URL pointing to such specifications

[Optional] API_NAME: Identifier for uploading test data to Schemathesis.io

schemathesis run [OPTIONS] SCHEMA [API_NAME]

Options

-w, --workers <workers_num>

Number of concurrent workers for testing. Auto-adjusts if ‘auto’ is specified

Default:: 1
Options:: auto | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 | 26 | 27 | 28 | 29 | 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | 38 | 39 | 40 | 41 | 42 | 43 | 44 | 45 | 46 | 47 | 48 | 49 | 50 | 51 | 52 | 53 | 54 | 55 | 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | 64

--dry-run: Simulate test execution without making any actual requests, useful for validating data generation

--fixups <fixups>

Apply compatibility adjustments

Options:: fast_api | utf8_bom | all

--experimental <experiments>

Enable experimental features

Options:: openapi-3.1 | schema-analysis | stateful-test-runner | stateful-only | coverage-phase | positive_data_acceptance

--experimental-missing-required-header-allowed-statuses <missing_required_header_allowed_statuses>: Comma-separated list of status codes expected for test cases with a missing required header

--experimental-positive-data-acceptance-allowed-statuses <positive_data_acceptance_allowed_statuses>: Comma-separated list of status codes considered as successful responses

--experimental-negative-data-rejection-allowed-statuses <negative_data_rejection_allowed_statuses>: Comma-separated list of status codes expected for rejected negative data

-c, --checks <checks>

Comma-separated list of checks to run against API responses

Default:: not_a_server_error
Options:: not_a_server_error | status_code_conformance | content_type_conformance | response_headers_conformance | response_schema_conformance | negative_data_rejection | ignored_auth | all

--exclude-checks <exclude_checks>

Comma-separated list of checks to skip during testing

Default:
Options:: not_a_server_error | status_code_conformance | content_type_conformance | response_headers_conformance | response_schema_conformance | negative_data_rejection | ignored_auth

--max-response-time <max_response_time>: Time limit in milliseconds for API response times. The test will fail if a response time exceeds this limit.

-x, --exitfirst

Terminate the test suite immediately upon the first failure or error encountered

Default:: False

--max-failures <max_failures>: Terminate the test suite after reaching a specified number of failures or errors

--app <app>: Specify the WSGI/ASGI application under test, provided as an importable Python path

--wait-for-schema <wait_for_schema>: Maximum duration, in seconds, to wait for the API schema to become available. Disabled by default

--validate-schema <validate_schema>

Validate input API schema. Set to ‘true’ to enable or ‘false’ to disable

Default:: False

-b, --base-url <base_url>: Base URL of the API, required when schema is provided as a file

--request-timeout <request_timeout>: Timeout limit, in milliseconds, for each network request during tests

--request-proxy <request_proxy>: Set the proxy for all network requests

--request-tls-verify <request_tls_verify>

Configures TLS certificate verification for server requests. Can specify path to CA_BUNDLE for custom certs

Default:: true

--request-cert <request_cert>: File path of unencrypted client certificate for authentication. The certificate can be bundled with a private key (e.g. PEM) or the private key can be provided with the –request-cert-key argument

--request-cert-key <request_cert_key>: Specify the file path of the private key for the client certificate

--rate-limit <rate_limit>: Specify a rate limit for test requests in ‘<limit>/<duration>’ format. Example - 100/m for 100 requests per minute

-H, --header <headers>: Add a custom HTTP header to all API requests. Format: ‘Header-Name: Value’

-a, --auth <auth>: Provide the server authentication details in the ‘USER:PASSWORD’ format

-A, --auth-type <auth_type>

Specify the authentication method. For custom authentication methods, see our Authentication documentation: https://schemathesis.readthedocs.io/en/stable/auth.html#custom-auth

Default:: basic
Options:: basic | digest

--include-path <include_path>: Path to include in testing.

--include-path-regex <include_path_regex>: Path pattern to include in testing.

--exclude-path <exclude_path>: Path to exclude from testing.

--exclude-path-regex <exclude_path_regex>: Path pattern to exclude from testing.

--include-method <include_method>: Method to include in testing.

--include-method-regex <include_method_regex>: Method pattern to include in testing.

--exclude-method <exclude_method>: Method to exclude from testing.

--exclude-method-regex <exclude_method_regex>: Method pattern to exclude from testing.

--include-name <include_name>: Operation name to include in testing.

--include-name-regex <include_name_regex>: Operation name pattern to include in testing.

--exclude-name <exclude_name>: Operation name to exclude from testing.

--exclude-name-regex <exclude_name_regex>: Operation name pattern to exclude from testing.

--include-tag <include_tag>: Tag to include in testing.

--include-tag-regex <include_tag_regex>: Tag pattern to include in testing.

--exclude-tag <exclude_tag>: Tag to exclude from testing.

--exclude-tag-regex <exclude_tag_regex>: Tag pattern to exclude from testing.

--include-operation-id <include_operation_id>: ID to include in testing.

--include-operation-id-regex <include_operation_id_regex>: ID pattern to include in testing.

--exclude-operation-id <exclude_operation_id>: ID to exclude from testing.

--exclude-operation-id-regex <exclude_operation_id_regex>: ID pattern to exclude from testing.

--include-by <include_by>: Include API operations by expression

--exclude-by <exclude_by>: Exclude API operations by expression

--exclude-deprecated

Exclude deprecated API operations from testing

Default:: False

--junit-xml <junit_xml>: Output a JUnit-XML style report at the specified file path

--cassette-path <cassette_path>: Save the test outcomes in a VCR-compatible format

--cassette-format <cassette_format>

Format of the saved cassettes

Options:: vcr | har

--cassette-preserve-exact-body-bytes: Retain exact byte sequence of payloads in cassettes, encoded as base64

--code-sample-style <code_sample_style>

Code sample style for reproducing failures

Options:: python | curl

--sanitize-output <sanitize_output>

Enable or disable automatic output sanitization to obscure sensitive data

Default:: True

--output-truncate <output_truncate>

Truncate schemas and responses in error messages

Default:: true

--show-trace

Display complete traceback information for internal errors

Default:: False

--debug-output-file <debug_output_file>: Save debugging information in a JSONL format at the specified file path

-D, --data-generation-method <data_generation_methods>

Specify the approach Schemathesis uses to generate test data. Use ‘positive’ for valid data, ‘negative’ for invalid data, or ‘all’ for both

Default:: positive
Options:: positive | negative | all

--stateful <stateful>

Enable or disable stateful testing

Options:: none | links

--generation-allow-x00 <generation_allow_x00>

Whether to allow the generation of `` bytes within strings

Default:: true

--generation-codec <generation_codec>: The codec used for generating strings

--generation-with-security-parameters <generation_with_security_parameters>

Whether to generate security parameters

Default:: true

--generation-graphql-allow-null <generation_graphql_allow_null>

Whether to use null values for optional arguments in GraphQL queries

Default:: true

--contrib-unique-data

Force the generation of unique test cases

Default:: False

--contrib-openapi-formats-uuid

Enable support for the ‘uuid’ string format in OpenAPI

Default:: False

--contrib-openapi-fill-missing-examples

Enable generation of random examples for API operations that do not have explicit examples

Default:: False

-t, --target <targets>

Guide input generation to values more likely to expose bugs via targeted property-based testing

Default:
Options:: response_time | all

--force-schema-version <force_schema_version>

Force the schema to be interpreted as a particular OpenAPI version

Options:: 20 | 30

--set-query <set_query>: OpenAPI: Override a specific query parameter by specifying ‘parameter=value’

--set-header <set_header>: OpenAPI: Override a specific header parameter by specifying ‘parameter=value’

--set-cookie <set_cookie>: OpenAPI: Override a specific cookie parameter by specifying ‘parameter=value’

--set-path <set_path>: OpenAPI: Override a specific path parameter by specifying ‘parameter=value’

--hypothesis-database <hypothesis_database>: Storage for examples discovered by Hypothesis. Use ‘none’ to disable, ‘:memory:’ for temporary storage, or specify a file path for persistent storage

--hypothesis-deadline <hypothesis_deadline>: Time limit for each test case generated by Hypothesis, in milliseconds. Exceeding this limit will cause the test to fail

--hypothesis-derandomize: Enables deterministic mode in Hypothesis, which eliminates random variation between tests

--hypothesis-max-examples <hypothesis_max_examples>: The cap on the number of examples generated by Hypothesis for each API operation

--hypothesis-phases <hypothesis_phases>

Testing phases to execute

Options:: explicit | reuse | generate | target | shrink

--hypothesis-no-phases <hypothesis_no_phases>

Testing phases to exclude from execution

Options:: explicit | reuse | generate | target | shrink

--hypothesis-report-multiple-bugs <hypothesis_report_multiple_bugs>: Report only the most easily reproducible error when multiple issues are found

--hypothesis-seed <hypothesis_seed>: Seed value for Hypothesis, ensuring reproducibility across test runs

--hypothesis-suppress-health-check <hypothesis_suppress_health_check>

A comma-separated list of Hypothesis health checks to disable

Options:: data_too_large | filter_too_much | too_slow | large_base_example | all

--hypothesis-verbosity <hypothesis_verbosity>

Verbosity level of Hypothesis output

Options:: quiet | normal | verbose | debug

--report <report_value>: Specify how the generated report should be handled. If used without an argument, the report data will automatically be uploaded to Schemathesis.io. If a file name is provided, the report will be stored in that file. The report data, consisting of a tar gz file with multiple JSON files, is subject to change

--schemathesis-io-token <schemathesis_io_token>: Schemathesis.io authentication token

--schemathesis-io-url <schemathesis_io_url>: Schemathesis.io base URL

--schemathesis-io-telemetry <schemathesis_io_telemetry>

Whether to send anonymized usage data to Schemathesis.io along with your report

Default:: true

--hosts-file <hosts_file>: Path to a file to store the Schemathesis.io auth configuration

-v, --verbosity: Increase verbosity of the output

--no-color: Disable ANSI color escape codes

--force-color: Explicitly tells to enable ANSI color escape codes

Arguments

SCHEMA: Required argument

API_NAME: Optional argument

Environment variables

SCHEMATHESIS_API_NAME: Provide a default for API_NAME

SCHEMATHESIS_EXPERIMENTAL_MISSING_REQUIRED_HEADER_ALLOWED_STATUSES: Provide a default for --experimental-missing-required-header-allowed-statuses

SCHEMATHESIS_EXPERIMENTAL_POSITIVE_DATA_ACCEPTANCE_ALLOWED_STATUSES: Provide a default for --experimental-positive-data-acceptance-allowed-statuses

SCHEMATHESIS_EXPERIMENTAL_NEGATIVE_DATA_REJECTION_ALLOWED_STATUSES: Provide a default for --experimental-negative-data-rejection-allowed-statuses

SCHEMATHESIS_WAIT_FOR_SCHEMA: Provide a default for --wait-for-schema

SCHEMATHESIS_BASE_URL: Provide a default for --base-url

SCHEMATHESIS_REPORT: Provide a default for --report

SCHEMATHESIS_TOKEN: Provide a default for --schemathesis-io-token

SCHEMATHESIS_URL: Provide a default for --schemathesis-io-url

SCHEMATHESIS_TELEMETRY: Provide a default for --schemathesis-io-telemetry

SCHEMATHESIS_HOSTS_PATH: Provide a default for --hosts-file