Add E2E tests for preferential optimization

[HideakiImamura]

Contributor License Agreement

This repository (optuna-dashboard) and Goptuna share common code.
This pull request may therefore be ported to Goptuna.
Make sure that you understand the consequences concerning licenses and check the box below if you accept the term before creating this pull request.

• I agree this patch may be ported to Goptuna by other Goptuna contributors.

Reference Issues/PRs

Following #484, I add e2e tests using playwright for preferential optimization.

What does this implement/fix? Explain your changes.

This PR adds the e2e test file test_preferential_optimization.py. This tests the followings.

• Accessing the trial list page and confirm that the trial list page is displayed. Check that all trials are running.
• The first trial are marks as completed with "Bad" feedback. Check that it is completed and the best trial.
• The second trial are marks as completed with "So-so" feedback. Check that it is completed and the best trial.
• The third trial are marks as completed with "Good" feedback. Check that it is completed and the best trial.
• The fourth trial are marks as failed. Check that it is failed.

The following screenshot was generated by the playwright --screenshot on option.

[HideakiImamura]

@keisuke-umezawa Could you review this PR?

[codecov]

Codecov Report

Merging #691 (e10a53c) into main (6bec7e3) will not change coverage.
Report is 42 commits behind head on main.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #691   +/-   ##
=======================================
  Coverage   62.88%   62.88%           
=======================================
  Files          35       35           
  Lines        2250     2250           
=======================================
  Hits         1415     1415           
  Misses        835      835           

📣 Codecov offers a browser extension for seamless coverage viewing on GitHub. Try it in Chrome or Firefox today!

[keisuke-umezawa]

@HideakiImamura Seems not to work with optuna==2.10.0. Could you check the code?

[HideakiImamura]

optuna==2.10.0 does not support the preferential optimization since it has been introduced in optuna==3.2.0 with database schema changes, so the error occurs. I think we should skip the tests.

[keisuke-umezawa]

@HideakiImamura
Thank you for the explanation. I understand. I will check this PR.

[keisuke-umezawa]

Great example of e2e test! Thank you.

[keisuke-umezawa]

thx!