Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Split wekeo provider #1214

Open
wants to merge 14 commits into
base: develop
Choose a base branch
from
Open

Split wekeo provider #1214

wants to merge 14 commits into from

Conversation

anesson-cs
Copy link
Collaborator

@anesson-cs anesson-cs commented Jun 14, 2024

Split wekeo provider into wekeo_main and wekeo_ecmwf to separate various data (Sentinel and some different kind of Copernicus data) and ECMWF data which are queried differently and that have two different way to discover their product types queryables

@sbrunato sbrunato marked this pull request as draft June 14, 2024 15:14
Copy link
Contributor

github-actions bot commented Jun 14, 2024

Test Results

    4 files  ± 0      4 suites  ±0   6m 12s ⏱️ +4s
  552 tests + 4    549 ✅ + 4    3 💤 ±0  0 ❌ ±0 
2 208 runs  +16  2 106 ✅ +16  102 💤 ±0  0 ❌ ±0 

Results for commit c33f340. ± Comparison against base commit 1e0e577.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Jun 14, 2024

badge

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     10       0  100.00%
cli.py                                         305      51  83.28%   60, 654-700, 802-853, 857
config.py                                      351      27  92.31%   81-83, 92, 100, 104-106, 183, 195, 460-462, 526-529, 576-577, 586-587, 666, 735-740, 742
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    763      82  89.25%   90-99, 372, 614, 658-661, 699, 746, 829-834, 860, 956, 1039, 1176, 1266-1278, 1318, 1320, 1348, 1352-1363, 1376-1382, 1472-1475, 1504-1524, 1580, 1597-1600, 1612-1615, 1633-1640, 1968, 2001-2007, 2274, 2278-2281, 2295-2297, 2332
api/search_result.py                            51       6  88.24%   33-35, 75, 84, 91, 105
api/product/__init__.py                          5       0  100.00%
api/product/_assets.py                          44       6  86.36%   27-29, 79, 119, 129
api/product/_product.py                        188      27  85.64%   56-63, 67-69, 239-240, 322, 351, 412, 426-429, 442, 466-469, 512-518
api/product/metadata_mapping.py                678      83  87.76%   67-69, 131-133, 234, 266-267, 313-314, 324-336, 338, 349, 355-367, 412-413, 450, 471-474, 497, 505-506, 582-583, 607-608, 614-617, 632-633, 782, 828, 1002-1007, 1138, 1152-1172, 1192, 1197, 1307, 1329, 1343, 1356-1375, 1414, 1466, 1504-1508, 1527
api/product/drivers/__init__.py                  5       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 23       3  86.96%   25, 48, 55
plugins/manager.py                             133      15  88.72%   49-51, 105-110, 156, 197, 219, 223, 248, 287-288
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           99      15  84.85%   46-54, 150-152, 199-200, 226-228
plugins/apis/usgs.py                           175      36  79.43%   60-65, 132, 205, 239, 274-276, 281, 307-308, 313, 343-350, 361-366, 388-394, 396-402, 425
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                21       1  95.24%   28
plugins/authentication/keycloak.py              49       6  87.76%   33-35, 132, 156-161
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       185      19  89.73%   41-43, 119, 133-158, 166, 320-323, 347
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 90      17  81.11%   35-37, 79, 107, 109, 131-143, 198-201
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      10  80.00%   32-34, 51-52, 71, 80-83, 85, 92-95
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      19  72.06%   28-30, 33, 72-75, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        492     165  66.46%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 459-460, 466-470, 503, 538, 542, 549, 579-587, 591, 629-637, 644-646, 687-761, 779-840, 851-856, 872-885, 914, 929-931, 934, 944-952, 960-973, 983-1002, 1009-1021, 1062, 1088, 1133-1135, 1355
plugins/download/base.py                       260      57  78.08%   58-64, 145, 180, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 688, 710, 718
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       531     132  75.14%   85-91, 197-209, 211-212, 247-250, 314-317, 319-320, 327-332, 350-365, 382-384, 396, 444, 451-457, 475, 489, 503, 511-513, 529-534, 545, 563, 605-609, 631, 671, 716, 730-736, 772-836, 854, 887-896, 922-923, 950-955, 961, 964, 980, 997-998, 1025-1026, 1033, 1095-1101, 1156-1157, 1163, 1173, 1207, 1243, 1261-1277
plugins/download/s3rest.py                     118      27  77.12%   55-58, 121, 162, 197, 227-234, 237-239, 243, 256-262, 270-271, 274-278, 301, 322-325
plugins/search/__init__.py                      26       3  88.46%   27-31
plugins/search/base.py                         133      17  87.22%   48-53, 104, 108, 121, 271, 291, 350-351, 371, 374-382
plugins/search/build_search_result.py          183      24  86.89%   62, 97, 141-142, 148, 159, 295-298, 327, 384-401, 463, 466, 476, 493, 521, 523
plugins/search/cop_marine.py                   200      49  75.50%   43-44, 55, 63-65, 71-72, 88, 90, 93, 128-130, 142-143, 183-192, 196, 199, 203, 221, 251, 255, 270, 274, 278, 282, 286-290, 296-299, 302-316, 333, 356, 359, 365
plugins/search/creodias_s3.py                   55       3  94.55%   56, 74, 108
plugins/search/csw.py                          108      83  23.15%   44-46, 58-59, 63-64, 72-120, 126-139, 147-179, 197-238
plugins/search/data_request_search.py          198      65  67.17%   53, 90-93, 109, 121, 125-126, 137, 142, 147, 154, 167-170, 224-225, 229, 239-245, 250, 276-279, 287-298, 315, 317, 324-325, 327-328, 346-350, 383, 390, 401, 414, 420-432, 437
plugins/search/qssearch.py                     663      78  88.24%   102, 386, 390-396, 404-405, 511-517, 565, 581, 591, 610-625, 663-666, 740-741, 787, 806, 821, 879, 900, 903-904, 913, 924, 933, 956, 1027-1032, 1036-1037, 1070, 1144, 1193, 1267-1271, 1337-1338, 1359, 1386-1398, 1405, 1437-1439, 1449-1455, 1499, 1514, 1534, 1613-1616, 1621-1624, 1651-1652, 1667
plugins/search/static_stac_search.py            73      12  83.56%   37-38, 95-122, 138, 151
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33       7  78.79%   35-37, 53-55, 59, 68
rest/config.py                                  26       0  100.00%
rest/constants.py                                7       0  100.00%
rest/core.py                                   238      30  87.39%   83-87, 281, 365, 475, 702, 709-757
rest/server.py                                 286      55  80.77%   81-82, 108, 131-133, 246-248, 304-305, 317-333, 425-430, 458, 626-633, 662, 706-707, 730, 803-805, 822-827, 856, 858, 862-863, 867-868
rest/stac.py                                   466     123  73.61%   65-67, 320, 342, 390-393, 417-443, 448-454, 477-479, 502, 537-538, 566, 579, 624-664, 692-708, 787-800, 807, 863-864, 931, 999-1001, 1220, 1230-1242, 1255-1277, 1291-1336, 1498-1499
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     184       9  95.11%   52-55, 235-239, 292, 295, 363
rest/types/queryables.py                        58       2  96.55%   38, 172
rest/types/stac_search.py                      131      11  91.60%   51-54, 133, 179, 194-196, 204, 208
rest/utils/__init__.py                          95      13  86.32%   51, 108-109, 128-130, 182, 192-206
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       3  86.96%   49, 61, 63
types/__init__.py                              110      11  90.00%   53, 70, 129-132, 199, 214-218, 245, 258
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              501      39  92.22%   84, 89, 110-112, 193-194, 203-230, 233, 247, 329-333, 409-413, 434-436, 518, 533, 571-572, 968-971, 979-980, 1021-1022, 1204
utils/constraints.py                           119      37  68.91%   94-103, 144, 149, 153, 164, 190-192, 202, 216-232, 241-252
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/requests.py                               55      11  80.00%   69, 96, 98, 100, 102, 104, 123, 128-130, 138
utils/rest.py                                   36       1  97.22%   57
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 143, 159-162, 215-224, 234-264
TOTAL                                         9603    1721  82.08%

Diff against develop

Filename                      Stmts    Miss  Cover
--------------------------  -------  ------  -------
api/core.py                      +6      -1  +0.21%
plugins/search/base.py            0      +4  -3.01%
plugins/search/qssearch.py      +15      +9  -1.11%
TOTAL                           +21     +12  -0.08%

Results for commit: c33f340

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

Copy link
Contributor

github-actions bot commented Jun 14, 2024

badge

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     10       0  100.00%
cli.py                                         305      51  83.28%   60, 654-700, 802-853, 857
config.py                                      351      28  92.02%   81-83, 92, 100, 104-106, 183, 195, 460-462, 526-529, 576-577, 586-587, 666, 700, 735-740, 742
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    763      82  89.25%   90-99, 372, 614, 658-661, 699, 746, 829-834, 860, 956, 1039, 1176, 1266-1278, 1318, 1320, 1348, 1352-1363, 1376-1382, 1472-1475, 1504-1524, 1580, 1597-1600, 1612-1615, 1633-1640, 1968, 2001-2007, 2274, 2278-2281, 2295-2297, 2332
api/search_result.py                            51       6  88.24%   33-35, 75, 84, 91, 105
api/product/__init__.py                          5       0  100.00%
api/product/_assets.py                          44       6  86.36%   27-29, 79, 119, 129
api/product/_product.py                        188      27  85.64%   56-63, 67-69, 239-240, 322, 351, 412, 426-429, 442, 466-469, 512-518
api/product/metadata_mapping.py                678      83  87.76%   67-69, 131-133, 234, 266-267, 313-314, 324-336, 338, 349, 355-367, 412-413, 450, 471-474, 497, 505-506, 582-583, 607-608, 614-617, 632-633, 782, 828, 1002-1007, 1138, 1152-1172, 1192, 1197, 1307, 1329, 1343, 1356-1375, 1414, 1466, 1504-1508, 1527
api/product/drivers/__init__.py                  5       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 23       4  82.61%   25, 48, 55, 68
plugins/manager.py                             133      15  88.72%   49-51, 105-110, 156, 197, 219, 223, 248, 287-288
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           99      15  84.85%   46-54, 150-152, 199-200, 226-228
plugins/apis/usgs.py                           175      36  79.43%   60-65, 132, 205, 239, 274-276, 281, 307-308, 313, 343-350, 361-366, 388-394, 396-402, 425
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                21       1  95.24%   28
plugins/authentication/keycloak.py              49       6  87.76%   33-35, 132, 156-161
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       185      19  89.73%   41-43, 119, 133-158, 166, 320-323, 347
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 90      17  81.11%   35-37, 79, 107, 109, 131-143, 198-201
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      35  30.00%   32-34, 48-53, 69-114
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      19  72.06%   28-30, 33, 72-75, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        492     165  66.46%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 459-460, 466-470, 503, 538, 542, 549, 579-587, 591, 629-637, 644-646, 687-761, 779-840, 851-856, 872-885, 914, 929-931, 934, 944-952, 960-973, 983-1002, 1009-1021, 1062, 1088, 1133-1135, 1355
plugins/download/base.py                       260      59  77.31%   58-64, 145, 180, 250-252, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 688, 710, 718
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       531     132  75.14%   85-91, 197-209, 211-212, 247-250, 314-317, 319-320, 327-332, 350-365, 382-384, 396, 444, 451-457, 475, 489, 503, 511-513, 529-534, 545, 563, 605-609, 631, 671, 716, 730-736, 772-836, 854, 887-896, 922-923, 950-955, 961, 964, 980, 997-998, 1025-1026, 1033, 1095-1101, 1156-1157, 1163, 1173, 1207, 1243, 1261-1277
plugins/download/s3rest.py                     118      27  77.12%   55-58, 121, 162, 197, 227-234, 237-239, 243, 256-262, 270-271, 274-278, 301, 322-325
plugins/search/__init__.py                      26       3  88.46%   27-31
plugins/search/base.py                         133      18  86.47%   48-53, 104, 108, 121, 271, 291, 350-351, 371, 374-382, 384
plugins/search/build_search_result.py          183      31  83.06%   62, 97, 141-142, 148, 159, 295-298, 327, 384-401, 463, 466, 476, 493, 513-528
plugins/search/cop_marine.py                   200      49  75.50%   43-44, 55, 63-65, 71-72, 88, 90, 93, 128-130, 142-143, 183-192, 196, 199, 203, 221, 251, 255, 270, 274, 278, 282, 286-290, 296-299, 302-316, 333, 356, 359, 365
plugins/search/creodias_s3.py                   55       3  94.55%   56, 74, 108
plugins/search/csw.py                          108      83  23.15%   44-46, 58-59, 63-64, 72-120, 126-139, 147-179, 197-238
plugins/search/data_request_search.py          198      65  67.17%   53, 90-93, 109, 121, 125-126, 137, 142, 147, 154, 167-170, 224-225, 229, 239-245, 250, 276-279, 287-298, 315, 317, 324-325, 327-328, 346-350, 383, 390, 401, 414, 420-432, 437
plugins/search/qssearch.py                     663     107  83.86%   102, 386, 390-396, 404-405, 511-517, 565, 568, 581, 591, 610-625, 663-666, 740-741, 787, 806, 821, 879, 900, 903-904, 913, 924, 933, 956, 1027-1032, 1036-1037, 1070, 1144, 1193, 1267-1271, 1337-1338, 1359, 1386-1398, 1405, 1437-1439, 1449-1455, 1499, 1514, 1534, 1596-1696
plugins/search/static_stac_search.py            73      12  83.56%   37-38, 95-122, 138, 151
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33      22  33.33%   35-37, 44-70
rest/config.py                                  26       4  84.62%   34-36, 68
rest/constants.py                                7       0  100.00%
rest/core.py                                   238     153  35.71%   83-87, 167-248, 272-326, 339-375, 412-444, 464-480, 503-514, 523-559, 578, 624-663, 702, 709-757
rest/server.py                                 286     286  0.00%    18-879
rest/stac.py                                   466     394  15.45%   65-67, 128-134, 142-165, 196-201, 233, 253-376, 386-455, 473-512, 528-581, 591-614, 624-664, 692-708, 718, 734-737, 756-826, 844-880, 909-934, 942-958, 966-971, 981-1008, 1018-1020, 1028-1030, 1043-1045, 1059-1076, 1086-1107, 1117-1139, 1147-1164, 1187-1210, 1220, 1230-1242, 1255-1277, 1291-1336, 1344-1361, 1371-1529
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     184      19  89.67%   52-55, 125, 235-239, 272-274, 292, 295, 301, 305, 363, 375-378
rest/types/queryables.py                        58      14  75.86%   38, 51-52, 58-59, 65-66, 95-100, 109-110, 172
rest/types/stac_search.py                      131      22  83.21%   51-54, 131-133, 157-158, 163-164, 179, 194-196, 204, 208, 255-260
rest/utils/__init__.py                          95      31  67.37%   51, 79-85, 105, 108-109, 128-130, 143, 150, 175-183, 190-211
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       6  73.91%   41, 44-45, 49, 61, 63
types/__init__.py                              110      36  67.27%   53, 66-70, 81-93, 120-122, 129-132, 172, 199, 209-221, 226, 245, 250, 258, 268
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              501      41  91.82%   84, 89, 110-112, 193-194, 203-230, 233, 247, 329-333, 409-413, 434-436, 518, 533, 571-572, 968-971, 979-980, 1021-1022, 1055, 1204, 1380
utils/constraints.py                           119      37  68.91%   94-103, 144, 149, 153, 164, 190-192, 202, 216-232, 241-252
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/requests.py                               55      11  80.00%   69, 96, 98, 100, 102, 104, 123, 128-130, 138
utils/rest.py                                   36       1  97.22%   57
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 143, 159-162, 215-224, 234-264
TOTAL                                         9603    2512  73.84%

Diff against develop

Filename                      Stmts    Miss  Cover
--------------------------  -------  ------  -------
api/core.py                      +6      -1  +0.21%
plugins/search/qssearch.py      +15     +10  -1.17%
TOTAL                           +21      +9  -0.04%

Results for commit: c33f340

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@anesson-cs anesson-cs marked this pull request as ready for review June 14, 2024 18:26
@sbrunato sbrunato marked this pull request as draft June 19, 2024 05:57
@anesson-cs anesson-cs marked this pull request as ready for review June 21, 2024 17:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant