Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port more functions from MLFlow and other minor improvements #501

Merged
merged 9 commits into from
Jan 17, 2025

Conversation

matt035343
Copy link
Member

Port dict and metric logging from MLFlow, return run id after logging model, and improve type hinting

@matt035343
Copy link
Member Author

create_package

Copy link

Created package with number v3.3.2a501.dev1

Copy link

github-actions bot commented Jan 16, 2025

Coverage

Coverage Report
FileStmtsMissCoverMissing
adapta/connectors/service_bus
   __init__.py110%18
   _connector.py17170%19–61
adapta/logs
   _async_logger.py81396%55, 78–79
   _base.py55689%35, 41, 44, 94–97
   _internal_logger.py109694%276–286
adapta/logs/handlers
   datadog_api_handler.py1093172%89, 106–113, 124, 136–152, 161–195, 204, 210, 238
adapta/metrics/providers
   datadog_provider.py43430%19–147
adapta/ml
   __init__.py110%19
   _model.py10100%17–42
adapta/ml/mlflow
   __init__.py220%17–18
   _client.py57570%19–213
   _functions.py48480%17–134
adapta/process_communication
   _models.py33682%90–96
adapta/security/clients
   __init__.py261254%27–28, 34–35, 41–42, 48–49, 53–54, 58–59
   _azure_client.py735032%42, 55–65, 75–78, 81, 84–86, 95–153, 156, 159–197
adapta/security/clients/aws
   _aws_client.py381755%37–40, 47, 57, 63, 75, 100–108, 114–117
   _aws_credentials.py733059%60–79, 83, 87, 91, 95, 99, 108–112, 116, 120, 124, 128, 132
adapta/security/clients/hashicorp_vault
   hashicorp_vault_client.py31487%46, 87, 91, 95
   kubernetes_client.py21576%45–48, 67–68
   oidc_client.py452056%33–62, 80–83, 92
   token_client.py17759%42–45, 52–53, 56, 59
adapta/storage/blob
   azure_storage_client.py1195157%71–78, 88, 95–96, 99–105, 127–128, 131–156, 159, 175–188, 196–200, 219, 240–242, 251, 267–271, 281–283, 286–305, 312
   local_storage_client.py54787%42, 67, 74, 87, 101, 104, 107
   s3_storage_client.py1177734%57, 61, 68–80, 91–96, 117–125, 133–134, 148–165, 181–185, 203–212, 222–243, 254–275
adapta/storage/cache
   redis_cache.py37370%19–107
adapta/storage/database/v2
   azure_sql.py34340%21–140
   odbc.py73730%21–219
   snowflake_sql.py69690%6–228
   trino_sql.py39390%21–127
adapta/storage/database/v2/models
   __init__.py110%19
   _models.py11110%20–54
adapta/storage/database/v3
   azure_sql.py322038%55, 70, 84–122, 132
   odbc.py721185%94–104, 114, 125–126, 140, 149–155, 182
   snowflake_sql.py772370%64–78, 93–95, 106–115, 142, 178, 193–196
   trino_sql.py51510%20–146
adapta/storage/delta_lake/v2
   _functions.py684534%73–106, 153–168, 210–294
adapta/storage/delta_lake/v3
   _functions.py661282%72, 156, 161, 163, 165, 221, 231, 241–251, 257, 292
adapta/storage/distributed_object_store/v2/datastax_astra
   _models.py19953%54–59, 62, 65, 68
   astra_client.py19614029%40–43, 113–137, 143–184, 190–191, 197–198, 201, 211, 223–224, 233, 273–361, 369, 393–520, 530, 541–556, 579–594, 616–632, 656–673
adapta/storage/distributed_object_store/v3/datastax_astra
   _model_mappers.py1662784%64, 131, 137, 139, 143, 149, 153, 162–168, 180–183, 199–200, 231, 288–298, 383, 424–436, 468
   _models.py453131%63–69, 72, 75, 78–122, 125
   astra_client.py1396851%35–38, 133–174, 180–181, 187–188, 191, 201, 213–214, 223, 267–353, 361, 373, 384–403, 426–443, 467–487, 513–525
adapta/storage/models
   astra.py351071%37, 40, 44, 59–61, 64, 70–73
   aws.py361169%36–39, 46–49, 57–59, 82, 91
   azure.py601772%32, 36–40, 47, 67, 70–71, 74–75, 89–93, 100, 113, 120, 123–124, 127
   filter_expression.py147597%56, 184–185, 239, 402
   hive.py572556%37, 41, 44, 92–99, 111, 114–115, 124–172, 175
   local.py21481%31, 35, 38, 50
adapta/storage/query_enabled_store
   _qes_astra.py571377%65–66, 78, 83–96, 106–109
   _qes_delta.py39490%33, 60, 72, 80
   _qes_local.py27293%37, 74
adapta/storage/secrets
   azure_secret_client.py20200%19–66
adapta/utils
   _common.py951584%36–37, 70–83, 94, 122, 142, 162, 245
   concurrent_task_runner.py27196%109
adapta/utils/data_structures
   _functions.py34197%134
adapta/utils/decorators
   _logging.py41198%32
   _rate_limit.py25196%58
adapta/utils/python_typing
   _functions.py11191%24
tests
   test_filtering_api.py32294%197–198
   test_utils.py166199%371
   test_vault_client.py801878%33–35, 40–42, 47–51, 56–57, 62–66
TOTAL4640136471% 

Tests Skipped Failures Errors Time
225 5 💤 0 ❌ 0 🔥 1m 2s ⏱️

@matt035343
Copy link
Member Author

create_package

Copy link

Created package with number v3.3.2a501.dev4

@matt035343
Copy link
Member Author

create_package

Copy link

Created package with number v3.3.2a501.dev6

@matt035343
Copy link
Member Author

create_package

Copy link

Created package with number v3.3.2a501.dev8

@matt035343
Copy link
Member Author

create_package

Copy link

Created package with number v3.3.2a501.dev10

@matt035343
Copy link
Member Author

create_package

Copy link

Created package with number v3.3.2a501.dev12

@matt035343
Copy link
Member Author

create_package

Copy link

Created package with number v3.3.2a501.dev14

@matt035343 matt035343 changed the title Port dict and metric logging from MLFlow and other minor improvements Port more functions from MLFlow and other minor improvements Jan 16, 2025
@matt035343 matt035343 marked this pull request as ready for review January 16, 2025 15:28
adapta/ml/mlflow/_functions.py Outdated Show resolved Hide resolved
adapta/ml/mlflow/_functions.py Show resolved Hide resolved
@matt035343 matt035343 merged commit 9c7eb84 into main Jan 17, 2025
2 checks passed
@matt035343 matt035343 deleted the mlflow-update branch January 17, 2025 10:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants