I have set of Test scenarios (say 10) which I would like to execute against different countries (say 3).
My first recommendation would be Templates with for loops. This way you do not have to manage failures, each iterations will be independent from the other. Every data set will be executed with the template. Note that if one iteration fails the whole test case will be marked as failed, but you will be able to check which iteration has failed.
Here is the code for the above example:
*** Variables ***
@{COUNTRIES} USA UK
*** Test Cases ***
Test Scenario 1
[Template] Test Scenario 1 Template
FOR ${country} IN @{COUNTRIES}
${country}
END
Test Scenario 2
[Template] Test Scenario 2 Template
FOR ${country} IN @{COUNTRIES}
${country}
END
Test Scenario 3
[Template] Test Scenario 3 Template
FOR ${country} IN @{COUNTRIES}
${country}
END
*** Keywords ***
Test Scenario 1 Template
[Arguments] ${country}
Log ${country}
Run Keyword If $country == 'UK' Fail Simulate failure.
Test Scenario 2 Template
[Arguments] ${country}
Log ${country}
Run Keyword If $country == 'USA' Fail Simulate failure.
Test Scenario 3 Template
[Arguments] ${country}
Log ${country}
The other option is based on this answer and it generates test cases dynamically during run time. Only a small library that also acts as a listener is needed. This library can have a start_suite
method that will be invoked and it will get the suite(s) as Python object(s), robot.running.model.TestSuite
. Then you could use this object along with Robot Framework's API to create new test cases programmatically.
DynamicTestLibrary.py:
from robot.running.model import TestSuite
class DynamicTestLibrary(object):
ROBOT_LISTENER_API_VERSION = 3
ROBOT_LIBRARY_SCOPE = 'GLOBAL'
ROBOT_LIBRARY_VERSION = 0.1
def __init__(self):
self.ROBOT_LIBRARY_LISTENER = self
self.top_suite = None
def _start_suite(self, suite, result):
self.top_suite = suite
self.top_suite.tests.clear() # remove placeholder test
def add_test_case(self, name, keyword, *args):
tc = self.top_suite.tests.create(name=name)
tc.keywords.create(name=keyword, args=args)
def add_test_matrix(self, data_set, test_scenarios):
for data in data_set:
for test_scenario in test_scenarios:
self.add_test_case(f'{test_scenario} - {data}', test_scenario, data)
globals()[__name__] = DynamicTestLibrary
Due to the backward incompatible changes (Running and result models have been changed) made in the 4.0 release the add_test_case
function should be change as below if you are using version above 4.0.
def add_test_case(self, name, keyword, *args):
tc = self.top_suite.tests.create(name=name)
tc.body.create_keyword(name=keyword, args=args)
In the robot file add a Suite Setup
in which you can call the Add Test Matrix
keyword with the list of countries and test scenarios to generate a test case for each combination. This way there will be an individual test case for each country - test scenario pair while having everything in one single file.
test.robot:
*** Settings ***
Library DynamicTestLibrary
Suite Setup Generate Test Matrix
*** Variables ***
@{COUNTRIES} USA UK
*** Test Cases ***
Placeholder test
[Documentation] Placeholder test to prevent empty suite error. It will be removed from execution during the run.
No Operation
*** Keywords ***
Generate Test Matrix
${test scenarios}= Create List Test Scenario 1 Test Scenario 2 Test Scenario 3
DynamicTestLibrary.Add Test Matrix ${COUNTRIES} ${test scenarios}
Test Scenario 1
[Arguments] ${country}
Log ${country}
Run Keyword If $country == 'UK' Fail Simulate failure.
Test Scenario 2
[Arguments] ${country}
Log ${country}
Run Keyword If $country == 'USA' Fail Simulate failure.
Test Scenario 3
[Arguments] ${country}
Log ${country}
This will be the output on the console:
# robot --pythonpath . test.robot
==============================================================================
Test
==============================================================================
Test Scenario 1 - USA | PASS |
------------------------------------------------------------------------------
Test Scenario 2 - USA | FAIL |
Simulate failure.
------------------------------------------------------------------------------
Test Scenario 3 - USA | PASS |
------------------------------------------------------------------------------
Test Scenario 1 - UK | FAIL |
Simulate failure.
------------------------------------------------------------------------------
Test Scenario 2 - UK | PASS |
------------------------------------------------------------------------------
Test Scenario 3 - UK | PASS |
------------------------------------------------------------------------------
Test | FAIL |
6 critical tests, 4 passed, 2 failed
6 tests total, 4 passed, 2 failed
==============================================================================