v0.5 changes: logging, computational improvements, and more #84

mmcdermott · 2023-12-03T14:39:33Z

No description provided.

…nly still failing.

…least.

…runs now.

Duplication bug in task caching

Removed the old increment calls to assign event IDs in favor of hashes of subject IDs and timestamps which can be run lazily.

Enable flash attention

Remove the `with_row_count` to create new event IDs to speed up dataset creation and permit lazy frames to be used longer.

Added logging to other aspects of ESGPT.

codecov · 2023-12-03T14:47:36Z

Codecov Report

Attention: 72 lines in your changes are missing coverage. Please review.

Comparison is base (e944f14) 86.08% compared to head (8e38596) 88.23%.

❗ Current head 8e38596 differs from pull request most recent head c017794. Consider uploading reports for the commit c017794 to get more accurate results

Files	Patch %	Lines
EventStream/data/pytorch_dataset.py	93.01%	16 Missing ⚠️
EventStream/data/dataset_polars.py	91.25%	14 Missing ⚠️
EventStream/data/dataset_base.py	73.80%	11 Missing ⚠️
EventStream/transformer/model_output.py	11.11%	8 Missing ⚠️
EventStream/transformer/config.py	45.45%	6 Missing ⚠️
EventStream/data/config.py	94.11%	5 Missing ⚠️
EventStream/data/visualize.py	0.00%	3 Missing ⚠️
...tream/transformer/lightning_modules/fine_tuning.py	75.00%	3 Missing ⚠️
EventStream/transformer/transformer.py	88.00%	3 Missing ⚠️
...tStream/transformer/lightning_modules/embedding.py	66.66%	1 Missing ⚠️
... and 2 more

Additional details and impacted files

@@            Coverage Diff             @@
##              dev      #84      +/-   ##
==========================================
+ Coverage   86.08%   88.23%   +2.15%     
==========================================
  Files          34       33       -1     
  Lines        6401     6569     +168     
==========================================
+ Hits         5510     5796     +286     
+ Misses        891      773     -118

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

…n tests and a reference to deleted generative evaluation code.

…sues with ESDS conversion.

Adds support and a script for exporting an ESDS dataset from an ESGPT dataset (with support for modifier columns).

…d elsewhere). tests failing due to data_stats.json errors.

Nested caching

Updated some deprecated polars functions.

mmcdermott added 30 commits September 6, 2023 21:20

Added flash attention.

a339171

Merge branch 'dev' into enable_flash_attention

f479082

Added a schema output to pytorch dataset.

c313c1c

Fixed small bug with schema and added test.

8437fe0

Small updates to dataset for testing compute costs.

8cbacb9

First attempt; likely very buggy.

0504580

More changes; not sure if it is 100% working yet or not. Tests certai…

49c4f02

…nly still failing.

Fixed some more typos; now can run up through getitem and collate at …

3d577ab

…least.

Fixed some small bugs; tests still not fully passing but sample data …

00a0305

…runs now.

Fixed a number of small bugs; tests closer to passing.

260161a

I think all tests are now passing.

6052b8a

Updated notebook.

7098db8

Starter code for nested caching and subsampling

070e711

Added doctest.

a7ab111

Merge branch 'improved_compute' into duplication_bug_in_task_caching

c2c4a2e

Added test for task caching bug.

6235bd0

Fixed test.

20ef6af

Lint fixes.

c78cb4d

Merge pull request #77 from mmcdermott/duplication_bug_in_task_caching

3a1821c

Duplication bug in task caching

Merge branch 'improved_compute' into nested_caching

e4f3e98

Added loguru

cf7b9e3

Trying logger changes.

16e0c87

Added some logging and got tests passing.

e327db2

Removed the old increment calls.

9c0eeb5

Added loguru throughout.

64df044

Merge pull request #78 from mmcdermott/remove_unnecessary_increment

63a838a

Removed the old increment calls to assign event IDs in favor of hashes of subject IDs and timestamps which can be run lazily.

Merge pull request #79 from mmcdermott/enable_flash_attention

33215e7

Enable flash attention

Merge branch 'v0.5' into proper_logs

866e5c1

Added loguru initialization code to all scripts.

8c4d60f

Fixed tests.

1b5c30b

mmcdermott added 4 commits December 2, 2023 19:13

Fixed all tests and upgraded to polars 0.19.19

dd2035f

Merge pull request #82 from mmcdermott/remove_with_row_count

273e09a

Remove the `with_row_count` to create new event IDs to speed up dataset creation and permit lazy frames to be used longer.

Added logging to other aspects of ESGPT.

b34b3fe

Merge pull request #83 from mmcdermott/more_logging

ccf9da9

Added logging to other aspects of ESGPT.

mmcdermott and others added 23 commits December 3, 2023 12:46

Removing currently unfinished and unused evaluation code.

e1a3c3e

Removed other buggy code and an unnecessary break statement in sklear…

ea9ce9f

…n tests and a reference to deleted generative evaluation code.

Fixed URI function bug.

7d1efef

Added an ESDS conversion option.

cf8fe51

Added ESDS support; need to test with modifiers.

4bd8499

Added modifiers into standard sample data example and fixed slight is…

537ec1a

…sues with ESDS conversion.

Fixed bugs with saving new data_stats fot cached subset data

38a0897

Fixed tests by adding medications to the sample NA config.

1b59238

Merge pull request #85 from mmcdermott/ESDS_export

acd2bf0

Adds support and a script for exporting an ESDS dataset from an ESGPT dataset (with support for modifier columns).

Changed cache_subset for train vs other splits

f7aa908

Fixed small bugs in data pipeline inhibiting runs on real data.

65d3c75

Fixed subset caching to account for extra padding on time_delta

3078b9e

Stylistic changes.

44704e0

Merge branch 'v0.5' into nested_caching

b1af020

Made subset size work in case of float subset size (which is supporte…

acf5b69

…d elsewhere). tests failing due to data_stats.json errors.

Made all uses of full data config a consistent property.

41cbcde

Fixed tests with nested caching

a303b39

Merge pull request #86 from mmcdermott/nested_caching

0c04c89

Nested caching

Updated some deprecated polars functions.

235d8b6

Merge pull request #87 from mmcdermott/polars_0.19

8e38596

Updated some deprecated polars functions.

Added two more logging lines.

99b7e11

Merge branch 'v0.5' of github.com:mmcdermott/EventStreamML into v0.5

5faa9a5

re-set to dev.

c017794

mmcdermott merged commit dd81924 into dev Dec 14, 2023
1 of 2 checks passed

mmcdermott deleted the v0.5 branch December 14, 2023 19:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v0.5 changes: logging, computational improvements, and more #84

v0.5 changes: logging, computational improvements, and more #84

mmcdermott commented Dec 3, 2023

codecov bot commented Dec 3, 2023 •

edited

Loading

v0.5 changes: logging, computational improvements, and more #84

v0.5 changes: logging, computational improvements, and more #84

Conversation

mmcdermott commented Dec 3, 2023

codecov bot commented Dec 3, 2023 • edited Loading

Codecov Report

codecov bot commented Dec 3, 2023 •

edited

Loading