dataset: add MECO datasets

[SiQube]

add meco datasets, there are some precomputed_reading_measures missing but we can add those later

resolves #947

Type of change

• New feature (non-breaking change which adds functionality)
• New dataset

✨ Enhancements

• add feature to read precomputed event R files

📀 Datasets

• add MECO first wave native reader (MECOL1W1)
• add MECO first wave second language reader (MECOL2W1)
• add MECO second wave second language reader (MECOL2W2)

How Has This Been Tested?

• passing tests/unit/datasets/datasets_test.py::test_public_dataset_registered for MECOL1W1
• passing tests/unit/datasets/datasets_test.py::test_public_dataset_registered for MECOL2W1
• passing tests/unit/datasets/datasets_test.py::test_public_dataset_registered for MECOL2W2
• passing adjusted test for loading precomputed event files tests/unit/dataset/dataset_files_test.py::test_load_precomputed_file_unsupported_file_format
• add and passing test for tests/unit/dataset/dataset_files_test.py::test_load_precomputed_file_rda_raise_value_error, which fails because the 'r_dataframe_key': 'joint.fix' is not defined.
• load precomputed events for R file tests/unit/dataset/dataset_files_test.py::test_load_precomputed_file_rda

[codecov]

Codecov Report

Attention: Patch coverage is 93.15068% with 5 lines in your changes missing coverage. Please review.

Project coverage is 99.87%. Comparing base (8768ff9) to head (6eab3d2).
Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
src/pymovements/dataset/dataset_files.py	73.68%	4 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##              main    #1054      +/-   ##
===========================================
- Coverage   100.00%   99.87%   -0.13%     
===========================================
  Files           87       91       +4     
  Lines         3818     3908      +90     
  Branches       679      683       +4     
===========================================
+ Hits          3818     3903      +85     
- Misses           0        4       +4     
- Partials         0        1       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[SiQube]

@dkrako don't know what is going on here...maybe you have an idea?

[SiQube]

possibly no wheels for pyreadr on windows and python 3.9?

[dkrako]

possibly no wheels for pyreadr on windows and python 3.9?

bummer. how do we want to proceed?

[dkrako]

The descirption and the citations of the datasets are missing. Please add the python class implementation of the dataset definitions, as the datasets won't be visible in the documentation otherwise.

[dkrako]

This error is very strange. You specified the pyreadr dependency as:

pymovements/pyproject.toml

Line 43 in fc7ca57

"pyreadr>=0.5.2,<0.6",

There is a windows wheel for v0.5.2: https://pypi.org/project/pyreadr/0.5.2/#files
but there is none for v0.5.3: https://pypi.org/project/pyreadr/0.5.3/#files

Why does pip try to build v0.5.3 instead of downloading v0.5.2?

Maybe there's a command-line option for pip to resolve this in the github workflow?

[dkrako]

Found it: --prefer-binary

Maybe it has to be combined with --no-binary :pymovements: to make sure that pymovements is build from source.

Here's the line to update in the github workflow:

pymovements/.github/workflows/tests.yml

Line 61 in f8152b3

run: tox -vv --notest -e ${{ matrix.tox_env }}

[SiQube]

this seems to work now. I'm pretty unsure about the line I've added to tox.ini. @dkrako can you quadruple check it? I'll add the py files later

[dkrako]

this seems to work now. I'm pretty unsure about the line I've added to tox.ini. @dkrako can you quadruple check it? I'll add the py files later

Ah great! even better to include it in tox.ini than in the github workflow.

[SiQube]

@dkrako in this PR we have two contributions, one are the three datasets. second is the logic to read R files, e.g. .rda

[dkrako]

Please add a full documentation on the new functionality.

[SiQube]

Please review the rest, I want to merge it at some point, and not account for 200 prs that happened in the meantime

[dkrako]

Alright, then first please incorporate the changes from:

• chore: remove mirrors from datasets with no real mirrors #1059
• feat: add DatasetDefinition.long_name #1096

[SiQube]

done, forced push to retrigger readthedocs which failed

[SiQube]

added changes as well

[dkrako]

please document this functionality in Dataset.load_precomputed_events()

[SiQube]

will add it after #1099 is merged due to merge conflicts

[dkrako]

please document this functionality in Dataset.load_precomputed_reading_measures()

[SiQube]

will add it after #1099 is merged due to merge conflicts

[dkrako]

if this file is a dataset related test file this should be visible in the filename.

please also make sure to not use any data from the datasets but fill it with own data.