Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run pytest with --color=yes to force GitHub Actions logs to have color #3330

Merged
merged 3 commits into from
Jul 16, 2024

Conversation

weiji14
Copy link
Member

@weiji14 weiji14 commented Jul 16, 2024

Description of proposed changes

GitHub Actions does not show colour outputs for pytest by default (pytest-dev/pytest#7443), but it can be enabled using --color=yes flag.

Before (black & white 🐼) After (color 🌈)
image image

Enables color output for pytest on:

  • ci_doctests.yaml
  • ci_tests.yaml
  • ci_tests_dev.yaml
  • ci_tests_legacy.yaml

Fixes #

Reminders

  • Run make format and make check to make sure the code follows the style guide.
  • Add tests for new features or tests that would have caught the bug that you're fixing.
  • Add new public functions/methods/classes to doc/api/index.rst.
  • Write detailed docstrings for all functions/methods.
  • If wrapping a new module, open a 'Wrap new GMT module' issue and submit reasonably-sized PRs.
  • If adding new functionality, add an example to docstrings or tutorials.
  • Use underscores (not hyphens) in names of Python files and directories.

Slash Commands

You can write slash commands (/command) in the first line of a comment to perform
specific operations. Supported slash command is:

  • /format: automatically format and lint the code

GitHub Actions does not show colour outputs for pytest by default (pytest-dev/pytest#7443), but it can be enabled using `--color=yes` flag.
@weiji14 weiji14 added the maintenance Boring but important stuff for the core devs label Jul 16, 2024
@weiji14 weiji14 self-assigned this Jul 16, 2024
@weiji14 weiji14 added the run/test-gmt-dev Trigger the GMT Dev Tests workflow in PR label Jul 16, 2024
@seisman
Copy link
Member

seisman commented Jul 16, 2024

Can we add it directly to

addopts = "--verbose --durations=0 --durations-min=0.2 --doctest-modules --mpl --mpl-results-path=results"
?

@weiji14
Copy link
Member Author

weiji14 commented Jul 16, 2024

Can we add it directly to

addopts = "--verbose --durations=0 --durations-min=0.2 --doctest-modules --mpl --mpl-results-path=results"

?

We could do that yes. I'm just not sure how everyone's terminals works with color output.

@weiji14 weiji14 removed the run/test-gmt-dev Trigger the GMT Dev Tests workflow in PR label Jul 16, 2024
@seisman
Copy link
Member

seisman commented Jul 16, 2024

I'm just not sure how everyone's terminals works with color output.

--color=color         Color terminal output (yes/no/auto)

Maybe --color=auto and assume that pytest works?

@weiji14
Copy link
Member Author

weiji14 commented Jul 16, 2024

I'm just not sure how everyone's terminals works with color output.

--color=color         Color terminal output (yes/no/auto)

Maybe --color=auto and assume that pytest works?

Should be auto by default already I think. It's just that --color=auto doesn't work on GitHub Actions (see thread at pytest-dev/pytest#7443).

@weiji14 weiji14 added the run/test-gmt-dev Trigger the GMT Dev Tests workflow in PR label Jul 16, 2024
@weiji14 weiji14 changed the title CI: Show color output on pytest logs in GitHub Actions Run pytest with --color=yes to force color output logs in GitHub Actions Jul 16, 2024
@weiji14 weiji14 changed the title Run pytest with --color=yes to force color output logs in GitHub Actions Run pytest with --color=yes to force GitHub Actions logs to have color Jul 16, 2024
@weiji14 weiji14 added run/benchmark Trigger the benchmark workflow in PRs and removed run/test-gmt-dev Trigger the GMT Dev Tests workflow in PR labels Jul 16, 2024
@weiji14
Copy link
Member Author

weiji14 commented Jul 16, 2024

Ok, switched to putting --color=yes in pytest addopts. Activating the benchmark workflow just to make sure the color output works (otherwise the CI tests won't run with a change to pyproject.toml only).

Copy link

codspeed-hq bot commented Jul 16, 2024

CodSpeed Performance Report

Merging #3330 will improve performances by 38.95%

Comparing pytest-color-output (24464f6) with main (97a6f30)

Summary

⚡ 21 improvements
✅ 80 untouched benchmarks

Benchmarks breakdown

Benchmark main pytest-color-output Change
test_binstats_no_outgrid 25.4 ms 23 ms +10.39%
test_call_module 11.1 ms 8.6 ms +29.35%
test_virtual_file 90.4 ms 65.1 ms +38.95%
test_virtualfile_from_matrix 90.8 ms 65.4 ms +38.82%
test_virtualfile_from_vectors 91.4 ms 66 ms +38.42%
test_virtualfile_from_vectors_one_string_or_object_column[object] 11.2 ms 8.6 ms +29.97%
test_virtualfile_from_vectors_one_string_or_object_column[str] 11.2 ms 8.7 ms +29.94%
test_virtualfile_in_required_z_matrix[DataFrame-vector] 12.2 ms 9.6 ms +26.85%
test_virtualfile_in_required_z_matrix[Dataset-vector] 14.6 ms 12.1 ms +21.41%
test_virtualfile_in_required_z_matrix[array-matrix] 11.8 ms 9.3 ms +27.89%
test_config_format_date_map 45.1 ms 38.6 ms +17.05%
test_load_remote_dataset_benchmark_with_region 57.3 ms 52 ms +10.21%
test_geopandas_info_geodataframe 16.1 ms 13.6 ms +18.57%
test_grdlandmask_no_outgrid 135.4 ms 118.2 ms +14.61%
test_info_pandas_dataframe_date_column[date32[day][pyarrow]] 12.5 ms 9.9 ms +26.27%
test_info_pandas_dataframe_date_column[date64[ms][pyarrow]] 12.5 ms 9.9 ms +26.22%
test_info_pandas_dataframe_date_column[datetime64[ns]] 12.3 ms 9.7 ms +26.56%
test_info_xarray_dataset_time_column 12.4 ms 9.8 ms +26.53%
test_begin_end 37.3 ms 32.5 ms +14.95%
test_velo_pandas_dataframe 21.5 ms 19.1 ms +13.03%
... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

@weiji14 weiji14 removed the run/benchmark Trigger the benchmark workflow in PRs label Jul 16, 2024
@weiji14
Copy link
Member Author

weiji14 commented Jul 16, 2024

So much performance improvement from adding colour 🤣

Pytest log output looks colored at https://github.com/GenericMappingTools/pygmt/actions/runs/9950591916/job/27488697947?pr=3330#step:7:106

image

@weiji14 weiji14 marked this pull request as ready for review July 16, 2024 04:35
@weiji14 weiji14 merged commit 1f4ce5e into main Jul 16, 2024
9 checks passed
@weiji14 weiji14 deleted the pytest-color-output branch July 16, 2024 05:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
maintenance Boring but important stuff for the core devs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants