Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: implement federation of collections calls #17

Merged
merged 6 commits into from
Sep 1, 2024

Conversation

syphax-bouazzouni
Copy link

@syphax-bouazzouni syphax-bouazzouni commented May 6, 2024

Context

This Pr is related to ontoportal/ontoportal-project#38; and implements the parallel call against multiple API. Please take a look at the tests for examples of usage.

image

Changes

  • Add unit tests of the expected behavior of the federation (db35d02)
  • Add caching debugging message option to see the cached and missed calls (9d5bb86)
  • Update the initialization of the HTTP connection to have multiple by API (1daf74f)
  • Create the RequestFederation module to implement federated_get function (63d00f0)
  • Update collections calls to use federated_get() instead of get() (1161610)
  • Update the special case of analytics to handle federated calls (f651f9f)

Benchmark

Portal name Response time with no UI cache Response time with UI cache Results count
AgroPortal      
https://data.agroportal.lirmm.fr 0.16290399990975857s 0.0006169998086988926s 1
https://data.agroportal.lirmm.fr/ontologies 0.3249760000035167s 0.13214100012555718s 185
https://data.agroportal.lirmm.fr/submissions 7.929086999967694s 2.010168000124395s 184
BioPortal      
https://data.bioontology.org/ 0.6429309998638928 0.0005680001340806484s 1
https://data.bioontology.org/ontologies 24.174467999953777s 0.7763689998537302s 1110
https://data.bioontology.org/submissions 22.10651999991387s 3.0445370003581047s 1031
EcoPortal      
https://data.ecoportal.lifewatch.eu/ 0.231649000197649s 0.292365999892354s 1
https://data.ecoportal.lifewatch.eu/ontologies 0.3730909996666014s 0.21255100006237626s 31
https://data.ecoportal.lifewatch.eu/submissions 3.1629160000011325s 1.7344190003350377s 27
EarthPortal      
https://earthportal.eu:8443 0.13366000028327107s 0.6724610002711415s 1
https://data.earthportal.eu/ontologies 0.2508490001782775s 0.02595800021663308s 40
https://data.earthportal.eu/submissions 1.5953649999573827s 0.9168690000660717s 44
BiodviPortal      
https://data.biodivportal.gfbio.org/ 0.16592800011858344s 0.6727269999682903s 1
https://data.biodivportal.gfbio.org/ontologies 0.26030199974775314s 0.019911999814212322s 29
https://data.biodivportal.gfbio.org/submissions 1.074064000044018s 0.5119559997692704s 28
Federation      
Federated Ontologies 26.728501999750733s 0.9789899997413158s 1210
Federation Submissions 36.02172500034794s 6.078201999887824s 1130
Total 50s 6s 1210

@syphax-bouazzouni syphax-bouazzouni changed the title Feature: implement fédération collections calls Feature: implement federation of collections calls May 7, 2024
@syphax-bouazzouni syphax-bouazzouni force-pushed the feature/federate-multiple-apis branch 12 times, most recently from 6e2cea0 to 1fcf35a Compare May 10, 2024 16:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants