I'm trying to use Import Execution Results - REST to import Behave JSON results with Multipart. I can get the import to work, but the "The execution details displays the result of the Cucumber Scenario." does not work.
Here's what I'm doing:
I'm using the following feature file with Behave:
Scenario Outline: Verify the UI content of SOME page
Given I access to SOME in "PAGE" as "DEFAULT_USER"
When I set "<toggle_type>" toggle "with" page refresh
Then I can see page displayed with the title "SOME | PAGE"
And I compare "SOME IMAGE" image with a "98" similarity with "<toggle_type>"
@SOME-TAG
Examples:
| toggle_type |
| spring |
| default |
I'm running behave with:
behave --no-skipped --tags=SOME-TAG -f json.pretty -o='reports/behave/Xray_behave_result.json'
I'm importing the report with:
POST /rest/raven/1.0/import/execution/behave/multipart
The server reply is:
{"testExecIssue":{"id":"4678506","key":"SOMETHING-55744","self":"https://stage-myjira.somedomain.com/rest/api/2/issue/4678506%22%7D,%22testIssues%22:%7B%22success%22:%5B%7B%22id%22:%224678305%22,%22key%22:%22SOME-TAG%22,%22self%22:%22https://stage-myjira.somedomain.com/rest/api/2/issue/4678305%22%7D%5D%7D,%22infoMessages":\[\]}
But when I look at the Test Details for the test excution:SOMETHING-55744
of the test: SOME-TAG
, it's empty:
This is the resulting json: "Xray_behave_result.json"
[
{
"elements": [
{
"keyword": "Scenario Outline",
"location": "something_features/features/something_web.feature:74",
"name": "Verify the UI content of SOME page -- @2.1 ",
"status": "passed",
"steps": [
{
"keyword": "Given",
"location": "something_features/features/something_web.feature:60",
"match": {
"arguments": [
{
"name": "page",
"value": "some_page"
},
{
"name": "something",
"value": "other thing"
}
],
"location": "general_common_steps/common_something_steps.py:70"
},
"name": "I access to SOME \"page\" as \"DEFAULT USER\"",
"result": {
"duration": 11.369250059127808,
"status": "passed"
},
"step_type": "given"
},
{
"keyword": "When",
"location": "something_features/features/something_web.feature:61",
"match": {
"arguments": [
{
"name": "toggle_type",
"value": "spring"
},
{
"name": "refresh_condition",
"value": "with"
}
],
"location": "general_common_steps/common_something_steps.py:362"
},
"name": "I set \"spring\" toggle \"with\" page refresh",
"result": {
"duration": 3.7522451877593994,
"status": "passed"
},
"step_type": "when"
},
{
"keyword": "Then",
"location": "something_features/features/something_web.feature:62",
"match": {
"arguments": [
{
"name": "page_title",
"value": "SOME | PAGE"
}
],
"location": "general_common_steps/common_something_steps.py:261"
},
"name": "I can see page displayed with the title \"SOME | PAGE\"",
"result": {
"duration": 9.731576442718506,
"status": "passed"
},
"step_type": "then"
},
{
"keyword": "And",
"location": "something_features/features/something_web.feature:63",
"match": {
"arguments": [
{
"name": "banner",
"value": "SOME IMAGE"
},
{
"name": "similarity",
"value": "98"
}
],
"location": "general_common_steps/common_something_steps.py:2443"
},
"name": "I compare \"SOME IMAGE\" image with a \"98\" similarity with \"spring\"",
"result": {
"duration": 0.0009975433349609375,
"status": "passed"
},
"step_type": "then"
}
],
"tags": [
"SOME-TAG"
],
"type": "scenario"
},
{
"keyword": "Scenario Outline",
"location": "something_features/features/something_web.feature:75",
"name": "Verify the UI content of SOME page -- @2.2 ",
"status": "passed",
"steps": [
{
"keyword": "Given",
"location": "something_features/features/something_web.feature:60",
"match": {
"arguments": [
{
"name": "page",
"value": "SOME"
},
{
"name": "affiliation",
"value": "DEFAULT_USER"
}
],
"location": "general_common_steps/common_something_steps.py:70"
},
"name": "I access to SOME in \"PAGE\" as \"DEFAULT USER\"",
"result": {
"duration": 15.28377103805542,
"status": "passed"
},
"step_type": "given"
},
{
"keyword": "When",
"location": "something_features/features/something_web.feature:61",
"match": {
"arguments": [
{
"name": "toggle_type",
"value": "default"
},
{
"name": "refresh_condition",
"value": "with"
}
],
"location": "general_common_steps/common_something_steps.py:362"
},
"name": "I set \"default\" toggle \"with\" page refresh",
"result": {
"duration": 2.4945812225341797,
"status": "passed"
},
"step_type": "when"
},
{
"keyword": "Then",
"location": "something_features/features/something_web.feature:62",
"match": {
"arguments": [
{
"name": "page_title",
"value": "SOME | PAGE"
}
],
"location": "general_common_steps/common_something_steps.py:261"
},
"name": "I can see page displayed with the title \"SOME | PAGE\"",
"result": {
"duration": 5.220450162887573,
"status": "passed"
},
"step_type": "then"
},
{
"keyword": "And",
"location": "something_features/features/something_web.feature:63",
"match": {
"arguments": [
{
"name": "banner",
"value": "SOME IMAGE"
},
{
"name": "similarity",
"value": "98"
},
{
"name": "page",
"value": "SOME PAGE"
}
],
"location": "general_common_steps/common_something_steps.py:2443"
},
"name": "I compare \"SOME IMAGE\" image with a \"98\" similarity with \"default\"",
"result": {
"duration": 0.0029616355895996094,
"status": "passed"
},
"step_type": "then"
}
],
"tags": [
"SOME-TAG"
],
"type": "scenario"
}
],
"keyword": "Feature",
"location": "something_features/features/something_web.feature:1",
"name": "SOME NAME",
"status": "passed",
"tags": []
}
]
This is the info JSON I am passing:
{
"fields": {
"project": {
"id": "the id of the project"
},
"summary": "Brand new Test execution",
"issuetype": {
"id": "10007"
}
}
}
Behave is not generating a json report compatible with Cucumber. Check the following tutorial that uses a custom reporter to generate a report that, after importation, will show details in each step: https://docs.getxray.app/display/XRAY/Testing+using+Behave+in+Python
Another possible solution is to use converters that will convert behave reports into compatible Cucumber reports.