Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

(Ruby - Cucumber)Not all scenarios are shown in report when they are run in parallel #83

Open
penknife opened this issue Nov 21, 2018 · 14 comments
Open

(Ruby - Cucumber)Not all scenarios are shown in report when they are run in parallel #83

penknife opened this issue Nov 21, 2018 · 14 comments

Comments

@penknife
Copy link

Describe the bug
When I run scenarios for example in 5 threads (1350 scenarios) (via parallel_tests gem), I can see only 3 threads (940 scenarios for example) in Repost Portal launch. I cannot see all scenarios which should be run. I have thoughts that a not all scenarios are run with -f ReportPortal::Cucumber::ParallelFormatter.

When I have run scenarios with default html report everything good. I have seen for request in build log such things with params /item?filter.eq.parent=5bf30a0bb9d8210001c48485&filter.eq.name=Folder:%20scenarios produced an exception:

RestClient::BadRequest: 400 Bad Request"  or 
" /Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:377:in `block in make_headers' : warning: Overriding "Content-Type" header "multipart/form-data" with "multipart/form-data; boundary=----RubyFormBoundary6ioZJqz4TJecRnfc" due to payload (StructuredWarnings::StandardWarning)"

Expected behavior
I should see all 5 threads in launch and all run scenarios

Versions:
-Mac OS

  • gem for report portal support have used https://github.com/reportportal/agent-ruby#formatter_modes
  • Version of RP
    [Current version: Analysis Service: 4.3.0; API Service: 4.3.0; Index Service: 4.2.0; Jira Service: 4.3.0; Rally Service: 4.3.0; Authorization Service: 4.2.0; Service UI: 4.3.0; ]
@DzmitryHumianiuk
Copy link
Member

@aliaksandr-trush, @abotalov, any ides?

@penknife
Copy link
Author

penknife commented Dec 4, 2018

Dudes DO you have any ideas?

@aliaksandr-trush
Copy link
Collaborator

@penknife

  1. "warning" message doesn't related to the issue.

  2. The issue is possible when one of the formatter threads received error and unable to continue.
    Could you attach more logs around "RestClient::BadRequest: 400 Bad Request"?

@penknife
Copy link
Author

penknife commented Dec 4, 2018 •
edited by DzmitryHumianiuk
Loading

Also I have seen this warnings and errors in log

/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:377:in `block in make_headers' : warning: Overriding "Content-Type" header "multipart/form-data" with "multipart/form-data; boundary=----RubyFormBoundaryPkAAQ2N7fi9m1vjh" due to payload (StructuredWarnings::StandardWarning)

[Step 5/5] undefined method `key?' for nil:NilClass (NoMethodError)
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:165:in block in close_child_items'" [01:12:05]	[Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:166:in block in close_child_items'
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in loop'" [01:12:05]	[Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in loop'
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in close_child_items'" [01:12:05]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:179:in block in close_child_items'"
[01:12:05] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in close_child_items' [01:12:05]	[Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:179:in block in close_child_items'
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in each'" [01:12:05]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in close_child_items'"
[01:12:05] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in each' [01:12:05]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_report.rb:61:in done'"
[01:12:05] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in public_send'" [01:12:05]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in block (2 levels) in initialize'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in loop'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in block in initialize'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in close_child_items' [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:223:in exception_with_response'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_report.rb:61:in done' [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/abstract_response.rb:103:in return!'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in public_send' [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:809:in process_result'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in block (2 levels) in initialize' [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:725:in block in transmit'"
[01:12:06] [Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in loop' [01:12:06]	[Step 5/5] /Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in block in initialize'
[01:12:06] [Step 5/5] "/Users/runner/.rvm/rubies/ruby-2.3.7/lib/ruby/2.3.0/net/http.rb:853:in start'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:715:in transmit'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:145:in execute'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/request.rb:52:in execute'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/gems/rest-client-2.0.2/lib/restclient/resource.rb:51:in get'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:165:in block (2 levels) in close_child_items'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:198:in do_request'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:165:in block in close_child_items'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in loop'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:164:in close_child_items'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:179:in block in close_child_items'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in each'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/reportportal.rb:178:in close_child_items'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_report.rb:61:in done'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in public_send'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:37:in block (2 levels) in initialize'"
[01:12:06] [Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in loop'" [01:12:06]	[Step 5/5] "/Users/runner/.rvm/gems/ruby-2.3.7/bundler/gems/agent-ruby-4e39f69faf6c/lib/report_portal/cucumber/parallel_formatter.rb:35:in block in initialize'"

@penknife
Copy link
Author

penknife commented Dec 6, 2018

@penknife

  1. "warning" message doesn't related to the issue.
  2. The issue is possible when one of the formatter threads received error and unable to continue.
    Could you attach more logs around "RestClient::BadRequest: 400 Bad Request"?

Will be it enough details?

@penknife
Copy link
Author

Aso I have seen that for example from 4 threads in report I have 3 threads, so some threads are not shown in RP or not launch

@DzmitryHumianiuk
Copy link
Member

@aliaksandr-trush can you assist with it?
@penknife is it still actual?

@mikesalvia
Copy link

@DzmitryHumianiuk There has been some work done to work on the parallel executions, it is unfortunately in a branch. I can confirm that there are issues when running in parallel but we have worked to reduce those issues.

@mikesalvia
Copy link

#33 << work that has been done. I need to do some more fixes though.

@DzmitryHumianiuk
Copy link
Member

@mikesalvia i see it merged.
Do we only need to release it?

@abotalov
Copy link
Member

abotalov commented May 21, 2019 •
edited
Loading

It targeted a different branch. I'm not sure if that branch was properly tested.

@evjlobanova evjlobanova transferred this issue from reportportal/reportportal Mar 31, 2020
@mikesalvia
Copy link

Is this still active!?

@HardNorth
Copy link
Member

HardNorth commented Mar 24, 2021 •
edited
Loading

@mikesalvia No one knows. There is no Ruby contributor.

@mikesalvia
Copy link

Oh yikes!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@aliaksandr-trush @abotalov @HardNorth @penknife @mikesalvia @DzmitryHumianiuk