Testomat.io allows sending notifications for finished runs:
- Send brief reports to stakeholders
- Notify team members of failed tests
- Configure on which condition notification should be sent
Testomat.io allows sending notifications for finished runs via Email. Let's see how it works! First, you need to set up Email notifications in the Settings tab. Click on Report Notifications and click on Add Notification Rule
At this point your next steps are:
- Enter a title for Notification Rule
- Choose Email from the list
- Enter Email or multiple Emails you want to response
- Customize these fields in BASIC RULES or use ADVANCED RULES ENGINE to enter your rule expression
- Click on Save button
Now you have Email Notification enabled for the project.
How does it work? Each time Testomat.io creates Run Report which corresponds to your Email Notification Rule it will be sent to email.
Please note, that you can set up multiple Email Notifications for different Run reports.
Testomatio can send notifications to a specific Slack channel. Prepare a channel inside Slack workspace to which notifications will be sent:
To enable Slack notification create an incoming webhook by opening this link. Create a new Slack App:
Activate webhooks for this app:
Add a new webhook for app:
Select a channel to which notification will be sent:
Copy Webhook URL:
Create a new notification in Testomatio, select "Slack" and paste webhook URL into Url:
To send noitifcations in MS Teams you need to set up incoming webhooks for your channel. Steps to configure:
- Navigate to "Apps" panel
- Search for "Incoming Webhook" and add it
- Configure it and copy webhook url
- Create a new notification in Testomatio, select "ms_teams" and paste webhook URL into Url:
Testomat.io allows to create Jira issue for failed test runs automatically. This option can be enabled in settings. To do this, you need to connect Jira project with Testomat.io. Please see dedicated documentation.
- Enter a name to your Notification rule
- Pick Jira from Notification Type drop-down
- Pick your dedicated Jira project from Jira Project drop-down
- Pick needed issue type from Issue Type drop-down
- Configure rules to define on which conditions this notification should be sent
- Click on Save button
Now Testomat.io will create an issue with detailed information on Test Run results within your Jira project for failed Test Runs. So you don't need to put all the data on each Test Run manually. This helps to save time and notify all contributors in a convenient way.
Testomatio has powerful rule engine which can be used to define on which conditions a notification should be sent. You can have multiple notification types with different notification channels in use for a single project.
There is a basic and advanced rules engine:
Inside Basic rules you can define simple conditions on which notifications should be sent. For instance, here is the rule for all manual with "Release" word to be reported:
This is the same rule we defined previously in Basic mode written in the format of Advanced mode. A notification will be sent for all manual runs that contain the word "Release":
manual and contains(run, "Release")
A complete list of allowed variables:
automated- boolean. True if a run is automated
manual- boolean. True if a run is manual
has_failed- boolean. True if a run has failed
has_passed- boolean. True if a run has passed
was_terminated- boolean. True if a run was terminated
run- string. Title of a run
rungroup- title. Title of rungroup a run belongs to
status- string. Status of run, 'passed' or 'failed' as a string.
started_at- datetime. Time when the run was started.
finished_at- datetime. Time when the run was finished
passed_tests- collection. A list of all passed tests in a run.
failed_tests- collection. A list of all failed tests in a run.
skipped_tests- collection. A list of all skipped tests in a run.
An expression should return a boolean value. To deal with types other than boolean functions and methods can be used:
String values can be checked with equal
== or not equal
!= operators. Also there is
contains function which checks inclusion of a string in another string:
Collections contain an array of objects.
.size to check for the size of items in the collection. For instance, this rule is activated when a number of failed tests is more than 10.
failed_tests.size > 10
Collection of tests can be filtered. Tests in the collection have following properties:
test['title']- title of a test
test['suite']- title of a suite of a test
test['id']- id of a test
test['suite_id']- id of a suite
test['status']- status of a test in collection
For instance, this is how to check if a collection of failed tests contains at least one test with
@important tag in its name:
failed_tests.filter(test, contains(test["title"], "@important")).size > 0
For instance, this is how notification can be enabled for reports finished in non-business time:
(finished_at.hour > 18 or finished_at.hour < 9)
To match tests executed on CI specify a Run title with "[CI]" prefix to identify that these tests were executed on CI:
TESTOMATIO_TITLE="[CI] Automated Tests"
Then write a notification rule that will check only for failing runs with "[CI]" in their title:
contains(run, "[CI]") and has_failed
automated and was_terminated
Please note that Run Group Notifications are available for Email notification type only
To configure Notification Rule for Run Group you need to:
- pick rungroup for Notification rule context
- add your Rule Expression, for example, you can use
rungroup_finishedvariable if you want to get notification when all Run report inside the group are finished.
A list of allowed variables:
title- string. Title of a rungroup
rungroup_finished- boolean. True if all runs completed => True if rungroup contains only finished runs.
runs- collection. A list of all runs inside a rungroup
finished_runs- collection. A list of finished (passed or failed) runs inside a rungroup
ongoing_runs- collection. A list of pending runs (scheduled, in progress) runs inside a rungroup
failed_runs- collection. A list of failed runs inside a rungroup