context source implementation for query entities #1282

thomasBousselin · 2024-12-12T14:10:20Z

No description provided.

thomasBousselin · 2024-12-12T14:14:05Z

search-service/config/detekt/baseline.xml

+    <ID>LongMethod:EntityHandler.kt$EntityHandler$@GetMapping("/{entityId}", produces = [APPLICATION_JSON_VALUE, JSON_LD_CONTENT_TYPE, GEO_JSON_CONTENT_TYPE]) suspend fun getByURI( @RequestHeader httpHeaders: HttpHeaders, @PathVariable entityId: URI, @AllowedParameters( implemented = [ QP.OPTIONS, QP.TYPE, QP.ATTRS, QP.GEOMETRY_PROPERTY, QP.LANG, QP.CONTAINED_BY, QP.JOIN, QP.JOIN_LEVEL, QP.DATASET_ID, ], notImplemented = [QP.FORMAT, QP.PICK, QP.OMIT, QP.ENTITY_MAP, QP.LOCAL, QP.VIA] ) @RequestParam queryParams: MultiValueMap&lt;String, String&gt; ): ResponseEntity&lt;*&gt;</ID>
+    <ID>LongMethod:EntityHandler.kt$EntityHandler$@GetMapping(produces = [APPLICATION_JSON_VALUE, JSON_LD_CONTENT_TYPE, GEO_JSON_CONTENT_TYPE]) suspend fun getEntities( @RequestHeader httpHeaders: HttpHeaders, @AllowedParameters( implemented = [ QP.OPTIONS, QP.COUNT, QP.OFFSET, QP.LIMIT, QP.ID, QP.TYPE, QP.ID_PATTERN, QP.ATTRS, QP.Q, QP.GEOMETRY, QP.GEOREL, QP.COORDINATES, QP.GEOPROPERTY, QP.GEOMETRY_PROPERTY, QP.LANG, QP.SCOPEQ, QP.CONTAINED_BY, QP.JOIN, QP.JOIN_LEVEL, QP.DATASET_ID, ], notImplemented = [QP.FORMAT, QP.PICK, QP.OMIT, QP.EXPAND_VALUES, QP.CSF, QP.ENTITY_MAP, QP.LOCAL, QP.VIA] ) @RequestParam queryParams: MultiValueMap&lt;String, String&gt; ): ResponseEntity&lt;*&gt;</ID>


Solution for this is to put csr related calculation into its own layer.
Should i do it in this PR?

it will also be cleaner, the handlers are getting hard to read with all the CSR stuff in them.

preferably in another PR (just after this one is merged)

github-actions · 2024-12-12T14:15:32Z

Test Results

69 files ± 0 69 suites ±0 1m 20s ⏱️ -11s
1 109 tests +10 1 109 ✅ +10 0 💤 ±0 0 ❌ ±0
1 148 runs +10 1 148 ✅ +10 0 💤 ±0 0 ❌ ±0

Results for commit 9ab1be9. ± Comparison against base commit 22a8d72.

This pull request removes 191 and adds 45 tests. Note that renamed tests count towards both.

                                    { "id":…, withTemporalValues=true, withAudit=false, expectation={
                      "@id": "https://uri…
                      "@type": "@json",
                      …
                    "@value": "/A/B"
                    "@value": "/C/D"
                    "@value": 20
                    "…
                    {
                  "@type": "https://uri.etsi.org/ngsi-ld/DateTime",
…

com.egm.stellio.search.csr.service.ContextSourceCallerTests ‑ queryContextSourceEntities should return a RevalidationFailedWarning when receiving bad payload()
com.egm.stellio.search.csr.service.ContextSourceCallerTests ‑ queryContextSourceEntities should return the count and the entities when the request succeed()
com.egm.stellio.search.csr.service.ContextSourceCallerTests ‑ retrieveContextSourceEntity should return a RevalidationFailedWarning when receiving bad payload()
com.egm.stellio.search.csr.service.ContextSourceCallerTests ‑ retrieveContextSourceEntity should return the entity when the request succeeds()
com.egm.stellio.search.csr.service.ContextSourceRegistrationServiceTests ‑ query on CSR entity idPattern should filter the result()
com.egm.stellio.search.csr.service.ContextSourceRegistrationServiceTests ‑ query on CSR entity types should filter the result()
com.egm.stellio.search.csr.service.ContextSourceUtilsTests ‑ merge entitiesList should add entities with the different ids()
com.egm.stellio.search.csr.service.ContextSourceUtilsTests ‑ merge entitiesList should merge entities with the same id()
com.egm.stellio.search.csr.service.ContextSourceUtilsTests ‑ merge entitiesList should merge using getMergeNewValues and return the received warnings()
com.egm.stellio.search.csr.service.ContextSourceUtilsTests ‑ merge entitiesList should not keep conflicting data from an auxiliary csr()
…

♻️ This comment has been updated with latest results.

search-service/src/main/kotlin/com/egm/stellio/search/entity/web/EntityHandler.kt

...rvice/src/main/kotlin/com/egm/stellio/search/csr/service/ContextSourceRegistrationService.kt

search-service/src/main/kotlin/com/egm/stellio/search/csr/service/ContextSourceCaller.kt

search-service/src/main/kotlin/com/egm/stellio/search/csr/service/ContextSourceUtils.kt

search-service/src/test/resources/ngsild/csr/contextSourceRegistration_minimal_entities.json

bobeal · 2024-12-15T11:01:23Z

take care of the name of the PR (will be used by default when merging)
add missing labels on the PR
link it to an issue (create one if it does not exist yet)

ranim-n · 2024-12-17T14:02:19Z

search-service/src/main/kotlin/com/egm/stellio/search/csr/service/ContextSourceCaller.kt

@@ -49,22 +105,23 @@ object ContextSourceCaller {
                    .path(uri.path)
                    .queryParams(queryParams)
                    .build()
+            }.headers { newHeaders ->
+                httpHeaders.getOrNone(HttpHeaders.LINK).onSome { link -> newHeaders[HttpHeaders.LINK] = link }
            }


I don't understand this change

The previous implementation was sending the Link header to the contextSource even if it received a null.
This implementation prevent this.

search-service/src/test/kotlin/com/egm/stellio/search/csr/service/ContextSourceCallerTests.kt

search-service/src/main/kotlin/com/egm/stellio/search/csr/service/ContextSourceCaller.kt

Co-authored-by: Benoit Orihuela <[email protected]> Co-authored-by: Ranim Naimi <[email protected]>

sonarqubecloud · 2024-12-18T09:14:36Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
85.5% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

github-actions bot added the feature New feature or request label Dec 12, 2024

thomasBousselin commented Dec 12, 2024

View reviewed changes

thomasBousselin marked this pull request as ready for review December 13, 2024 16:29

bobeal reviewed Dec 15, 2024

View reviewed changes

thomasBousselin added the ngsild-1.6.1 label Dec 16, 2024

thomasBousselin changed the title ~~Feature/context source for query entities~~ context source implementation for query entities Dec 16, 2024

This was linked to issues Dec 16, 2024

PoC for Distributed Operations #1191

Closed

{'geometry': None, ... } when calling endpoint with empty geometry_property #1277

Closed

This was unlinked from issues Dec 16, 2024

PoC for Distributed Operations #1191

Closed

{'geometry': None, ... } when calling endpoint with empty geometry_property #1277

Closed

thomasBousselin linked an issue Dec 16, 2024 that may be closed by this pull request

PoC for Distributed Operations #1191

Closed

thomasBousselin self-assigned this Dec 16, 2024

thomasBousselin requested a review from ranim-n December 16, 2024 10:35

ranim-n reviewed Dec 17, 2024

View reviewed changes

bobeal reviewed Dec 17, 2024

View reviewed changes

ranim-n approved these changes Dec 17, 2024

View reviewed changes

thomasBousselin force-pushed the feature/context-source-for-query-entities branch from c50ee39 to 7d4294a Compare December 17, 2024 17:22

bobeal added the csr Relates to context sources (registrations) API label Dec 18, 2024

bobeal removed a link to an issue Dec 18, 2024

PoC for Distributed Operations #1191

Closed

bobeal linked an issue Dec 18, 2024 that may be closed by this pull request

Add support for Query Entities distributed operation #1287

Closed

bobeal approved these changes Dec 18, 2024

View reviewed changes

thomasBousselin added 7 commits December 18, 2024 10:07

fix: minor serialization change

43148e0

feat: untested ContextSource For query entities

9af9448

feat: count with csr

08bea35

feat: test for ContextSourceUtils and ContextSourceCaller

ee695ab

feat: test for ContextSourceUtils and ContextSourceCaller

02a7358

feat: PR comments

fb2e41d

fix: calling cSource without link header

5c755bb

thomasBousselin and others added 4 commits December 18, 2024 10:07

fix: type can be a list

c831f62

fix: id name in minimal csr

5f3ed39

Apply suggestions from code review

7cac002

Co-authored-by: Benoit Orihuela <[email protected]> Co-authored-by: Ranim Naimi <[email protected]>

fix: logging in ContextSourceCaller

9ab1be9

thomasBousselin force-pushed the feature/context-source-for-query-entities branch from 7d4294a to 9ab1be9 Compare December 18, 2024 09:07

thomasBousselin merged commit 54678fd into develop Dec 18, 2024
11 checks passed

thomasBousselin deleted the feature/context-source-for-query-entities branch December 18, 2024 12:37

github-actions bot locked and limited conversation to collaborators Dec 18, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

context source implementation for query entities #1282

context source implementation for query entities #1282

thomasBousselin commented Dec 12, 2024

thomasBousselin Dec 12, 2024 •

edited

Loading

bobeal Dec 16, 2024

github-actions bot commented Dec 12, 2024 •

edited

Loading

bobeal commented Dec 15, 2024

ranim-n Dec 17, 2024

thomasBousselin Dec 17, 2024

sonarqubecloud bot commented Dec 18, 2024

		<ID>LongMethod:EntityHandler.kt$EntityHandler$@GetMapping("/{entityId}", produces = [APPLICATION_JSON_VALUE, JSON_LD_CONTENT_TYPE, GEO_JSON_CONTENT_TYPE]) suspend fun getByURI( @RequestHeader httpHeaders: HttpHeaders, @PathVariable entityId: URI, @AllowedParameters( implemented = [ QP.OPTIONS, QP.TYPE, QP.ATTRS, QP.GEOMETRY_PROPERTY, QP.LANG, QP.CONTAINED_BY, QP.JOIN, QP.JOIN_LEVEL, QP.DATASET_ID, ], notImplemented = [QP.FORMAT, QP.PICK, QP.OMIT, QP.ENTITY_MAP, QP.LOCAL, QP.VIA] ) @RequestParam queryParams: MultiValueMap<String, String> ): ResponseEntity<*></ID>
		<ID>LongMethod:EntityHandler.kt$EntityHandler$@GetMapping(produces = [APPLICATION_JSON_VALUE, JSON_LD_CONTENT_TYPE, GEO_JSON_CONTENT_TYPE]) suspend fun getEntities( @RequestHeader httpHeaders: HttpHeaders, @AllowedParameters( implemented = [ QP.OPTIONS, QP.COUNT, QP.OFFSET, QP.LIMIT, QP.ID, QP.TYPE, QP.ID_PATTERN, QP.ATTRS, QP.Q, QP.GEOMETRY, QP.GEOREL, QP.COORDINATES, QP.GEOPROPERTY, QP.GEOMETRY_PROPERTY, QP.LANG, QP.SCOPEQ, QP.CONTAINED_BY, QP.JOIN, QP.JOIN_LEVEL, QP.DATASET_ID, ], notImplemented = [QP.FORMAT, QP.PICK, QP.OMIT, QP.EXPAND_VALUES, QP.CSF, QP.ENTITY_MAP, QP.LOCAL, QP.VIA] ) @RequestParam queryParams: MultiValueMap<String, String> ): ResponseEntity<*></ID>

context source implementation for query entities #1282

context source implementation for query entities #1282

Conversation

thomasBousselin commented Dec 12, 2024

thomasBousselin Dec 12, 2024 • edited Loading

Choose a reason for hiding this comment

bobeal Dec 16, 2024

Choose a reason for hiding this comment

github-actions bot commented Dec 12, 2024 • edited Loading

Test Results

bobeal commented Dec 15, 2024

ranim-n Dec 17, 2024

Choose a reason for hiding this comment

thomasBousselin Dec 17, 2024

Choose a reason for hiding this comment

sonarqubecloud bot commented Dec 18, 2024

Quality Gate passed

thomasBousselin Dec 12, 2024 •

edited

Loading

github-actions bot commented Dec 12, 2024 •

edited

Loading