Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[INF] Set multi-test env to compat dependency and system #1143

Open
wants to merge 48 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
b8c9803
Set dependency latest version env
Zeroto521 Aug 11, 2022
a61d3c4
Add marks
Zeroto521 Aug 11, 2022
9986e8e
Test latest env
Zeroto521 Aug 11, 2022
2c38994
require minimal pyspark is 3.2.2
Zeroto521 Aug 11, 2022
f71cefc
update env name
Zeroto521 Aug 11, 2022
5192b90
empty commit to trigger ci
Zeroto521 Aug 11, 2022
287ee5b
remove duplicate `-`
Zeroto521 Aug 12, 2022
0cd95e5
Merge remote-tracking branch 'upstream/dev' into ci/testing-env
Zeroto521 Aug 20, 2022
5b8a09a
skip `test_docs_general_functions_present`
Zeroto521 Aug 20, 2022
90ce9f0
ignore flake
Zeroto521 Aug 20, 2022
f236624
still build the documentation
Zeroto521 Aug 20, 2022
b155722
Update skipif logic
Zeroto521 Aug 20, 2022
5b75198
test built result on 'build documentation' ci
Zeroto521 Aug 20, 2022
18f34e0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 20, 2022
263aaa2
lint code
Zeroto521 Aug 20, 2022
3cc6c78
Merge branch 'ci/testing-env' of https://github.com/Zeroto521/pyjanit…
Zeroto521 Aug 20, 2022
4d26689
simplify a bit
Zeroto521 Aug 20, 2022
4c429e6
Merge branch 'pyjanitor-devs:dev' into ci/testing-env
Zeroto521 Aug 21, 2022
b92925a
Merge branch 'pyjanitor-devs:dev' into ci/testing-env
Zeroto521 Oct 15, 2022
4b1e2e9
Remove py3.6 and py3.7
Zeroto521 Oct 15, 2022
bd99a1a
Merge branch 'pyjanitor-devs:dev' into ci/testing-env
Zeroto521 Oct 19, 2022
9330f43
Merge branch 'pyjanitor-devs:dev' into ci/testing-env
Zeroto521 Oct 25, 2022
34a415a
Merge remote-tracking branch 'upstream/dev' into ci/testing-env
Zeroto521 Oct 30, 2022
f2aec0b
Update name
Zeroto521 Oct 30, 2022
1391751
lint codes
Zeroto521 Oct 30, 2022
30cc453
Add `lazy_loader` into env
Zeroto521 Oct 30, 2022
609465b
reversed changes to `test_documentation_build.py`
Zeroto521 Oct 30, 2022
8079eca
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 30, 2022
f6bf739
Update CHANGELOG.md
Zeroto521 Oct 30, 2022
240acbf
Merge branch 'ci/testing-env' of https://github.com/Zeroto521/pyjanit…
Zeroto521 Oct 30, 2022
208706a
fix `hypothesis.errors.FailedHealthCheck`
Zeroto521 Oct 30, 2022
aad0f47
fix `hypothesis.errors.FailedHealthCheck`
Zeroto521 Oct 30, 2022
3a157ed
fix `hypothesis.errors.FailedHealthCheck`
Zeroto521 Oct 30, 2022
5ee95d6
fix `hypothesis.errors.FailedHealthCheck`
Zeroto521 Oct 30, 2022
2405a07
Merge branch 'dev' into ci/testing-env
Zeroto521 Nov 3, 2022
bec7293
fix `hypothesis.errors.FailedHealthCheck`
Zeroto521 Nov 3, 2022
d13fb8d
add fix for GH#1184
samukweku Nov 1, 2022
0249ce8
Merge branch 'ci/testing-env' of https://github.com/Zeroto521/pyjanit…
Zeroto521 Oct 30, 2022
fc4ba6c
Merge remote-tracking branch 'upstream/dev' into ci/testing-env
Zeroto521 Nov 7, 2022
ad217f2
Fix `hypothesis.errors.Flaky`
Zeroto521 Nov 8, 2022
5397c1b
Merge branch 'dev' into ci/testing-env
Zeroto521 Nov 9, 2022
6da2269
Merge branch 'dev' into ci/testing-env
Zeroto521 Nov 20, 2022
334cee5
Merge branch 'ci/testing-env' of https://github.com/Zeroto521/pyjanit…
Zeroto521 Nov 26, 2022
788ad2d
Merge remote-tracking branch 'upstream/dev' into ci/testing-env
Zeroto521 Nov 26, 2022
03f4f52
Add `pytest-doctestplus`
Zeroto521 Nov 26, 2022
a48292d
docstrings tests will happen on some specific conditions
Zeroto521 Nov 26, 2022
e107901
Merge branch 'dev' into ci/testing-env
ericmjl Nov 30, 2022
1378f54
Merge branch 'dev' into ci/testing-env
Zeroto521 Dec 2, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 24 additions & 11 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: pyjanitor tests
name: tests

on:
push:
Expand All @@ -14,18 +14,28 @@ concurrency:

jobs:
run-tests:
strategy:
fail-fast: false
matrix:
test-subset: ["turtle", "not turtle"]
runs-on: ubuntu-latest
name: Run pyjanitor test suite
name: ${{ matrix.os }}, ${{ matrix.python-version }}, ${{ matrix.env }}
runs-on: ${{ matrix.os }}

# https://github.com/marketplace/actions/setup-miniconda#use-a-default-shell
defaults:
run:
shell: bash -l {0}

strategy:
fail-fast: false
matrix:
os:
- ubuntu-latest
- macos-latest
- windows-latest
dev: [false]
python-version:
- "3.8"
- "3.9"
- "3.10"
env: [ci/envs/latest.yaml]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Zeroto521 would you recommend also testing on the development environment as defined in environment-dev.yml?


steps:
- name: Checkout repository
uses: actions/checkout@v2
Expand All @@ -36,18 +46,21 @@ jobs:
with:
auto-update-conda: true
miniforge-variant: Mambaforge
channels: conda-forge
environment-file: environment-dev.yml
environment-file: ${{ matrix.env }}
use-mamba: true

- name: Install pyjanitor
run: python -m pip install -e .

- name: Run docstrings tests
if: ${{ contains(matrix.env, 'latest.yaml') && contains(matrix.os, 'ubuntu') }}
run: pytest -v -r a -n auto --color=yes --durations=0 --cov=janitor --cov-append --cov-report term-missing --cov-report xml --doctest-only janitor

- name: Run unit tests
run: pytest -v -r a -n auto --color=yes --durations=0 --cov=janitor --cov-append --cov-report term-missing --cov-report xml tests -m "${{ matrix.test-subset }}"
- name: Run not turtle unit tests
run: pytest -v -r a -n auto --color=yes --durations=0 --cov=janitor --cov-append --cov-report term-missing --cov-report xml tests -m "not turtle"

- name: Run turtle unit tests
run: pytest -v -r a -n auto --color=yes --durations=0 --cov=janitor --cov-append --cov-report term-missing --cov-report xml tests -m "turtle"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Historically we've been able to reduce testing time by running the turtle and not turtle tests in parallel. Codecov will also get updated correctly as long as --cov-append is called, if I remember correctly. Can we reinstate running turtle and not turtle tests here, please?


# https://github.com/codecov/codecov-action
- name: Upload code coverage
Expand Down
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

## [Unreleased]

- [INF] Set a series of complete testing envs. Issue #1127 @Zeroto521
- [INF] Replace `pytest.ini` file with `pyproject.toml` file. PR #1204 @Zeroto521
- [INF] Extract docstrings tests from all tests. PR #1205 @Zeroto521
- [BUG] address the `TypeError` when importing v0.24.0 (issue #1201 @xujiboy and @joranbeasley)
Expand Down
33 changes: 33 additions & 0 deletions ci/envs/latest.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: pyjanitor-test
channels:
- conda-forge
dependencies:
# required
- pandas
- pandas-flavor
- multipledispatch
- lazy_loader
- scipy # math, functions

# optional
- natsort # functions
- numba # functions
- biopython # biology
- rdkit # chemistry
- tqdm # chemistry
- unyt # engineering
- requests # finance
- pyspark>=3.2.2
- xarray
# io
- openpyxl
- xlrd

# testing
- pytest
- pytest-cov
- pytest-xdist
- pytest-doctestplus
- hypothesis
- pandas-vet
- interrogate
1 change: 1 addition & 0 deletions tests/functions/test_encode_categorical.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ def df_checks():

@pytest.mark.functions
@given(df=categoricaldf_strategy())
@settings(deadline=None)
def test_encode_categorical(df):
df = df.encode_categorical("names")
assert df["names"].dtypes == "category"
Expand Down