Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Connected and typed dimensions #30

Open
wants to merge 33 commits into
base: master
Choose a base branch
from
Open

Connected and typed dimensions #30

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
94ef50c
Validate a catalog entry is in at most one catalog
RickMoynihan Feb 12, 2021
772a8a5
Update PMD4 validations to reflect current muttnik expectations (#1249)
kieranlives Sep 17, 2021
ab7d400
Init basic test suite: ExactlyOneTitle, MaxOneContents
kieranlives Sep 27, 2021
11e9ac4
Change DatasetExactlyOneGraph to DatasetMinOneGraph
kieranlives Sep 27, 2021
cf3a04c
Test PMD4 suite for validations that have been removed
kieranlives Sep 27, 2021
18f1342
deftest removed-validations is a more general name
kieranlives Sep 27, 2021
79245c7
Rearrange test-validation to be more obviously a helper around run-va…
kieranlives Sep 27, 2021
faa65c5
Simpler test data and correct datasetContents range
kieranlives Sep 28, 2021
ed725f9
Move tests into pmd-validations namespace
kieranlives Sep 28, 2021
ecd18f5
Add DISTINCT for counts for test data in multiple graphs
kieranlives Oct 6, 2021
8fbdabf
DatasetContentsIsCorrectType: filter datasets with contents now datas…
kieranlives Oct 7, 2021
8c5be69
Add DISTINCT to pmd-qb validations to account for multiple graphs
kieranlives Oct 13, 2021
83925ed
Remove ConceptSchemeMustHaveCatalogEntry
kieranlives Oct 27, 2021
5bbc0db
IC 19 operates on qb:dimension rather than qb:componentProperty
kieranlives Oct 28, 2021
9956518
Merge branch 'relax-some-validations' into 1.1
Robsteranium Nov 12, 2021
f274f95
Merge remote-tracking branch 'origin/ds-at-most-1-cat' into 1.1
Robsteranium Nov 12, 2021
45c7205
Add validation to confirm dataset graph contains contents
Robsteranium Jan 22, 2021
db895b9
Add dataset contents validation to suite manifest
Robsteranium Jan 22, 2021
02d6d8f
Fix DatasetGraphContainsContents to handle multi-graph datasets
Robsteranium Nov 12, 2021
d4dffc9
Merge branch 'dataset-graph-contents' into 1.1
Robsteranium Nov 12, 2021
9f6e36a
Validate all qb attribute and dimension properties have qb:codeList
RickMoynihan Jul 13, 2020
437ba21
Ignore measure-type dimension and datatype properties when validating…
Robsteranium Nov 16, 2021
d37f758
Revise expectations for concept labels
Robsteranium Nov 16, 2021
f41ab9c
Require that both broader and narrower be present if one is
Robsteranium Nov 18, 2021
a51b6dc
Add validation to confirm dataset metadata objects are labelled
Robsteranium Nov 19, 2021
9400e98
Revise pmd-IC3 to check for measure properties not measure types
Robsteranium Nov 22, 2021
d7e9465
Dataset dcat:modified, dcat:issued are now optional
scottlowe Feb 14, 2022
0f54031
Merge pull request #28 from Swirrl/sl/1.1-timestamps
Robsteranium Mar 21, 2022
ad4728d
Reinstate PMD Dataset required issued validation
scottlowe May 3, 2022
bf6cc37
Merge pull request #29 from Swirrl/sl/reinstate-pmd-required-ds-issued
Robsteranium May 30, 2022
d51fc8c
PMD4 doesn't require issued/modified dates for catalog records
Robsteranium May 30, 2022
b57005d
Add test that dimensions are connected and typed
AlasdairGray Oct 26, 2022
b53c5d4
Remove redundant bgp
AlasdairGray Oct 26, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/.cpcache
/.nrepl-port
40 changes: 27 additions & 13 deletions deps.edn
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,29 @@
{:aliases {:pmd4 {:extra-deps {swirrl/validations.pmd4 {:local/root "pmd4"}}}
:pmd-qb {:extra-deps {swirrl/validations.pmd-qb {:local/root "pmd-qb"}}}
:misc {:extra-deps {swirrl/validations.pmd-qb {:local/root "misc"}}}

:validate {:extra-deps { swirrl/rdf-validator {:git/url "https://github.com/Swirrl/rdf-validator.git"
:sha "c85338c44be9f7f9726c30dca4aa47ef8bd9cfe6"
}
;; Import log4j2 as the logging backend
org.apache.logging.log4j/log4j-api {:mvn/version "2.11.1"}
org.apache.logging.log4j/log4j-core {:mvn/version "2.11.1"}
org.apache.logging.log4j/log4j-slf4j-impl {:mvn/version "2.11.1"}
{ :aliases {:pmd4 {:extra-deps {swirrl/validations.pmd4 {:local/root "pmd4"}}}
:pmd-qb {:extra-deps {swirrl/validations.pmd-qb {:local/root "pmd-qb"}}}
:misc {:extra-deps {swirrl/validations.pmd-qb {:local/root "misc"}}}
:dev {:extra-deps { swirrl/rdf-validator {:git/url "https://github.com/Swirrl/rdf-validator.git"
:sha "c85338c44be9f7f9726c30dca4aa47ef8bd9cfe6"}
;; Import log4j2 as the logging backend
org.apache.logging.log4j/log4j-api {:mvn/version "2.11.1"}
org.apache.logging.log4j/log4j-core {:mvn/version "2.11.1"}
org.apache.logging.log4j/log4j-slf4j-impl {:mvn/version "2.11.1"}}
:extra-paths ["." "test"]}
:test {:extra-deps {swirrl/rdf-validator {:git/url "https://github.com/Swirrl/rdf-validator.git"
:sha "c85338c44be9f7f9726c30dca4aa47ef8bd9cfe6"}
lambdaisland/kaocha {:mvn/version "1.0.887"}
;; Import log4j2 as the logging backend
org.apache.logging.log4j/log4j-api {:mvn/version "2.11.1"}
org.apache.logging.log4j/log4j-core {:mvn/version "2.11.1"}
org.apache.logging.log4j/log4j-slf4j-impl {:mvn/version "2.11.1"}}
:main-opts ["-m" "kaocha.runner"]
:extra-paths ["." "test"]}

}
:extra-paths ["."] ;; for log4j2.xml
:main-opts ["-m" "rdf-validator.core"]}}}
:validate {:extra-deps {swirrl/rdf-validator {:git/url "https://github.com/Swirrl/rdf-validator.git"
:sha "c85338c44be9f7f9726c30dca4aa47ef8bd9cfe6"}
;; Import log4j2 as the logging backend
org.apache.logging.log4j/log4j-api {:mvn/version "2.11.1"}
org.apache.logging.log4j/log4j-core {:mvn/version "2.11.1"}
org.apache.logging.log4j/log4j-slf4j-impl {:mvn/version "2.11.1"}}
:extra-paths ["."] ;; for log4j2.xml
:main-opts ["-m" "rdf-validator.core"]}}}
21 changes: 6 additions & 15 deletions pmd-qb/src/swirrl/validations/pmd-qb/03_SELECT_DSD_Has_At_Least_1_Measure.sparql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,21 @@
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
PREFIX skos: <http://www.w3.org/2004/02/skos/core#>
PREFIX qb: <http://purl.org/linked-data/cube#>
PREFIX xsd: <http://www.w3.org/2001/XMLSchema#>
PREFIX owl: <http://www.w3.org/2002/07/owl#>

SELECT ?dsd WHERE {
?dsd a qb:DataStructureDefinition .

FILTER NOT EXISTS {
# ok if dsd has component with qb:measure
?dsd qb:component [qb:measure ?m] .
}
FILTER NOT EXISTS {
# ok if component has qb:componentProperty (or a subProperty)
# of MeasureType
?dsd qb:component ?comp .
?comp ?cp ?p .
?cp rdfs:subPropertyOf* qb:componentProperty .
?p a qb:MeasureType .
# qb:componentProperty or qb:measure etc
?component_attachment rdfs:subPropertyOf* qb:componentProperty .
?dsd qb:component [?component_attachment ?measure] .
?measure a qb:MeasureProperty .
}
}

#! IC-3. DSD includes measure - Every qb:DataStructureDefinition must include at least one declared measure.

# Note - this includes a reasoning workaround by looking for subProperties of qb:componentProperty.
#######################################################################
######################################################################
#### ORIGINAL ASK QUERY FROM SPEC IC-3 ####
#
# ASK {
Expand Down
2 changes: 1 addition & 1 deletion ...validations/pmd-qb/16_SELECT_Dataset_With_MeasureDimension_Obs_Has_Exactly_1_Value.sparql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ WHERE {
UNION
{
# Observations with more than one value of the same measure type
SELECT ?obs (COUNT(?val) AS ?count ) {
SELECT ?obs (COUNT(distinct ?val) AS ?count ) {
?obs qb:dataSet/qb:structure ?dsd ;
qb:measureType ?measure .
?obs ?measure ?val .
Expand Down
4 changes: 2 additions & 2 deletions ...wirrl/validations/pmd-qb/17_SELECT_All_Measures_Present_In_Measures_Dimension_Cube.sparql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,11 @@ PREFIX owl: <http://www.w3.org/2002/07/owl#>
# with the same non-measure dimension values for each of the declared measures.

# Count number of other measures found at each point
SELECT ?obs1 ?numMeasures (COUNT(?obs2) AS ?count)
SELECT ?obs1 ?numMeasures (COUNT(DISTINCT ?obs2) AS ?count)
WHERE {
{
# Find the DSDs and check how many measures they have
SELECT ?dsd (COUNT(?m) AS ?numMeasures) WHERE {
SELECT ?dsd (COUNT(DISTINCT ?m) AS ?numMeasures) WHERE {
?dsd qb:component/qb:measure ?m.
} GROUP BY ?dsd
}
Expand Down
4 changes: 2 additions & 2 deletions ...qb/src/swirrl/validations/pmd-qb/19_SELECT_Collection_Values_Must_Be_From_CodeList.sparql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ PREFIX owl: <http://www.w3.org/2002/07/owl#>

SELECT ?obs ?dim (?v AS ?valueNotInCodeList)
WHERE {
?obs qb:dataSet/qb:structure/qb:component/qb:componentProperty ?dim .
?obs qb:dataSet/qb:structure/qb:component/qb:dimension ?dim .
?dim a qb:DimensionProperty ;
qb:codeList ?list .
?list a skos:Collection .
Expand All @@ -22,4 +22,4 @@ WHERE {
#### ORIGINAL ASK QUERY FROM SPEC IC-19 ####
#
#
#######################################################################
#######################################################################
4 changes: 2 additions & 2 deletions ...src/swirrl/validations/pmd-qb/19_SELECT_ConceptScheme_Values_Must_Be_From_CodeList.sparql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ PREFIX owl: <http://www.w3.org/2002/07/owl#>

SELECT ?obs ?dim (?v AS ?valueNotInCodeList)
WHERE {
?obs qb:dataSet/qb:structure/qb:component/qb:componentProperty ?dim .
?obs qb:dataSet/qb:structure/qb:component/qb:dimension ?dim .
?dim a qb:DimensionProperty ;
qb:codeList ?list .
?list a skos:ConceptScheme .
Expand All @@ -22,4 +22,4 @@ WHERE {
#### ORIGINAL ASK QUERY FROM SPEC IC-19 ####
#
#
#######################################################################
#######################################################################
Loading