[SOLVED] Importing Robot framework executions to Xray with test step status

Importing Robot framework executions to Xray with test step status

I've been integration my Robotframework tests with Jira and Xray. I already have all tests with the corresponding test steps (actions) in Jira, organized in Test Plan and Test Executions.

Now what I'm failing to accomplish is having the individual test step status from the output.xml import.

For better visibility:

*** Test Cases ***
TESTKEY-92: Portal Test Case
    [Documentation]    Verify portal functionality.
    [Tags]             TESTKEY-92
    Step 1: Open Portal
    Step 2: Verify Login Button

*** Keywords ***
Step 1: Open Portal
    [Documentation]    Open the portal and handle popups.
    [Tags]    TESTKEY-92-step-1
    Open PORTAL
    Run Keyword And Ignore Error    Click Element    ${POPUP_BUTTON}
    Run Keyword And Ignore Error    Click Element    ${COOKIES_BUTTON}

Step 2: Verify Login Button
    [Documentation]    Verify the login button is present.
    [Tags]    TESTKEY-92-step-2
    Page Should Contain    ${LOGIN_BUTTON}

(I made it so the Step 2 would fail for integration testing purposes)

I then send via REST API with a curl command to the Jira test:

curl -v -X POST -H "Authorization: Bearer <my PAT>" -H "Content-Type: multipart/form-data" -F "file=@C:/Users/user/Downloads/test_reports/output.xml" "https://<my url>/rest/raven/2.0/import/execution/robot?testExecKey=TESTKEY-111"

If the test PASSED all the steps go automatically to PASS, however forcing the FAIL on one step to test the integration I'll have the TESTKEY-92 test failing and both test steps are as TODO.

Note both test steps in TESTKEY-92 have exactly the same string as the Keywords:

Step 1: Open Portal
Step 2: Verify Login Button

So how can I declare the keywords to have full match between the robot output and Jira test steps?

Solution

Whenever you import test results from Robot Framework they're usually mapped to Generic (i.e., unstructured test cases, without the typical steps). The process is detailed in Xray documentation. You may also report results against an exist manual, step-based, test case; however, you should have only the overall result about the test.

Xray doesn't provide itself a way to map keywords from RF to steps in manual/step-based Test issues in Xray. You can have visibility of RF keywords but in testruns of Generic (unstructutured) Test issues, and not on runs of manual/step-based tests.