Skip to content

Python Test Frameworks

Support Ukraine Badge plugin for pytest

uses API:


Terminal window
pip install pytestomatio


Synchronize tests to and get back test id. Tests will not be executed

Terminal window
pytest --testomatio sync

Remove all test ids from source code. Tests will not be executed

Terminal window
pytest --testomatio remove

Run pytest and send test results into
Test can be sent to without ids in your test code. If failed to match tests by title, it will create new tests for the run

Terminal window
pytest --testomatio report

Run pytest with debug parameter to get test data collected in metadata.json file

Terminal window
pytest --testomatio debug


Create environment variable TESTOMATIO and set your API key.

Terminal window

Select code style by set environment variable TESTOMATIO_CODE_STYLE. Available options are ‘default’ and ‘pep8’
If you are not sure, don’t set this variable. Default value is ‘default’

Terminal window

Set test run name in

Terminal window

Set test run environment in

Terminal window

Set test run labels in TESTOMATIO_LABELS=smoke,regression

Terminal window
### Run groups parameter
There is environment variable `TESTOMATIO_RUNGROUP_TITLE` that can be used to specify run tests with specific group title.
### pytest.ini
In case you are using private service, create `pytest.ini` file in your project root directory. Specify url in it
testomatio_url =

Submitting Test Run Environment

to configure test environment, you can use additional option:

Terminal window
pytest --testomatio report --testRunEnv "windows11,chrome,1920x1080"

Environment values are comma separated, please use double quotation.

Submitting Test Artifacts does not store any screenshots, logs or other artifacts.

In order to manage them it is advised to use S3 Buckets (GCP Storage).

Analyser needs to be aware of the cloud storage credentials. There are two options:

  1. Enable Share credentials with Reporter option in Settings -> Artifacts.

You would need to decide when you want to upload your test artifacts to cloud storage

  1. Upload page screenshot when test fails, using fixtures reference
# content of
import pytest
import random
import os
from typing import Dict
from pytest import StashKey, CollectReport
from playwright.sync_api import Page
phase_report_key = StashKey[Dict[str, CollectReport]]()
@pytest.hookimpl(wrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
rep = yield
item.stash.setdefault(phase_report_key, {})[rep.when] = rep
return rep
def handle_artifacts(page: Page, request):
report = request.node.stash[phase_report_key]
if ("call" not in report) or report["setup"].failed or report["call"].failed:
random_string = ''.join(random.choices(string.ascii_letters + string.digits, k=8))
filename = f"{random_string}.png"
screenshot_path = os.path.join(artifacts_dir, filename)
# file_path - required, path to file to be uploaded
# file_bytes - required, bytes of the file to be uploaded
# key - required, file name in the s3 bucket
# bucket_name - optional,name of the bucket to upload file to. Default value is taken from
artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)
# or
# artifact_url = pytest.testomatio.upload_file_object(file_bytes, key, bucket_name)
pytest.testomatio.add_artifacts(request.node, [artifact_url])

⚠️ Please take into account s3_connector available only after pytest_collection_modifyitems() hook is executed.

  1. If you prefer to use pytest hooks - add pytest_runtest_makereport hook in your file.
def pytest_runtest_makereport(item, call):
artifact_url = pytest.testomatio.upload_file(screenshot_path, filename)


  • tests can be synced even without @patest.mark.testomatio('@T96c700e6') decorator.
  • test title in == test name in pytest
  • test suit title in == test file name in pytest

Example of test

To make the experience more consistent, it uses standard pytest markers. test id is a string value that starts with @T and has 8 symbols after.

from pytest import mark
def test_example():
assert 2 + 2 == 4

Compatibility table with Testomatio check-tests

Importing test into testomat.iocompletepytest --testomatio sync
Exclude hook code of a testN/AN/A
Include line number code of a testN/AN/A
Import Parametrized Testscompletedefault behaviour
Disable Detached Testscompletepytest --testomatio sync --no-detached
Synchronous Importcompletedefault behaviour
Auto-assign Test IDs in Source Codecompletedefault behaviour
Keep Test IDs Between Projectscompletepytest --testomatio sync --create
Clean Test IDscompletepytest --testomatio remove
Import Into a BranchN/AN/A
Keep Structure of Source Codecompletepytest --testomatio sync --keep-structure
Delete Empty Suitescompletepytest --testomatio sync --no-empty
Import Into a Specific Foldercompletepytest --testomatio --directory "Windows\smoke"
Debuggingparitypytest --testomatio debug


  • import into empty project
  • updated test - (resync)
  • test run
  • test run into a folder
  • test run labels, tags


  • Fix test duration
  • Require more back references from testomatio
  • pytest.skip should behave as @pytest.mark.skip
  • Refactor test run and make run url available


  1. pip install -e .
  2. cz commit
  3. cz bump
  4. git push remoteName branchName --tags