Skip to content

Commit

Permalink
doc/api-details: update Node examples, model format and operators
Browse files Browse the repository at this point in the history 
Sync the docs to the current model definitions and mention the `__re`
operator, including an example.

Signed-off-by: Ricardo Cañuelo <[email protected]>
  • Loading branch information
@r-c-n
r-c-n committed Feb 15, 2024
1 parent 9f80a79 commit 64338ed
Showing 1 changed file with 258 additions and 28 deletions.
286 changes: 258 additions & 28 deletions doc/api-details.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -82,13 +82,18 @@ Another way of creating users is to use `kci user add` tool from kernelci-core.

## Nodes

As a proof-of-concept, an object model called `Node` is defined in this API.
It's possible to create new objects and retrieve them via the API.
`Node` objects form the basis of the API models to represent tests runs,
kernel builds, regressions and other test-related entities and their
relationships. See [the model definitions in
kernelci-core](https://github.com/kernelci/kernelci-core/blob/main/kernelci/api/models.py)
for details on the `Node` model and its subtypes. It's possible to
create new objects and retrieve them via the API.

### Create a Node

To create an object of `Node` model, a `POST` request should be made along with
the Node attributes. This requires an authentication token:
To create a `Node` or a `Node` subtype object, for instance, a
`Checkout` node, a `POST` request should be made along with the Node
attributes. This requires an authentication token:

```
$ curl -X 'POST' \
Expand All @@ -97,14 +102,48 @@ $ curl -X 'POST' \
-H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJib2IifQ.ci1smeJeuX779PptTkuaG1SEdkp5M1S1AgYvX8VdB20' \
-H 'Content-Type: application/json' \
-d '{
"name":"checkout",
"revision":{"tree":"mainline",
"url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch":"master",
"commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26",
"describe":"v5.16-rc4-31-g2a987e65025e"}
}'
{"_id":"61bda8f2eb1a63d2b7152418","kind":"node","name":"checkout","revision":{"tree":"mainline","url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git","branch":"master","commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26","describe":"v5.16-rc4-31-g2a987e65025e"},"parent":null,"status":"pending","result":null, "created":"2022-02-02T11:23:03.157648", "updated":"2022-02-02T11:23:03.157648"}
"name": "checkout",
"kind": "checkout",
"path": ["checkout"],
"data": {
"kernel_revision": {
"tree": "mainline",
"url": "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch": "master",
"commit": "2a987e65025e2b79c6d453b78cb5985ac6e5eb26",
"describe": "v5.16-rc4-31-g2a987e65025e"
}
}
}' | jq
{
"id": "61bda8f2eb1a63d2b7152418",
"kind": "checkout",
"name": "checkout",
"path": [
"checkout"
],
"group": null,
"parent": null,
"state": "running",
"result": null,
"artifacts": null,
"data": {
"kernel_revision": {
"tree": "mainline",
"url": "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch": "master",
"commit": "2a987e65025e2b79c6d453b78cb5985ac6e5eb26",
"describe": "v5.16-rc4-31-g2a987e65025e"
}
},
"created": "2024-02-01T09:58:28.479138",
"updated": "2024-02-01T09:58:28.479142",
"timeout": "2024-02-01T15:58:28.479145",
"holdoff": null,
"owner": "admin",
"user_groups": []
}
```

### Getting Nodes back
Expand All @@ -114,42 +153,229 @@ Reading Node doesn't require authentication, so plain URLs can be used.
To get node by ID, use `/node` endpoint with node ID as a path parameter:

```
$ curl http://localhost:8001/latest/node/61bda8f2eb1a63d2b7152418
{"_id":"61bda8f2eb1a63d2b7152418","kind":"node","name":"checkout","revision":{"tree":"mainline","url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git","branch":"master","commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26","describe":"v5.16-rc4-31-g2a987e65025e"},"parent":null,"status":"pending","result":null, "created":"2022-02-02T11:23:03.157648", "updated":"2022-02-02T11:23:03.157648"}
$ curl http://localhost:8001/latest/node/61bda8f2eb1a63d2b7152418 | jq
{
"id": "61bda8f2eb1a63d2b7152418",
"kind": "checkout",
"name": "checkout",
"path": [
"checkout"
],
"group": null,
"parent": null,
"state": "running",
"result": null,
"artifacts": null,
"data": {
"kernel_revision": {
"tree": "mainline",
"url": "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch": "master",
"commit": "2a987e65025e2b79c6d453b78cb5985ac6e5eb26",
"describe": "v5.16-rc4-31-g2a987e65025e"
}
},
"created": "2024-02-01T09:58:28.479000",
"updated": "2024-02-01T09:58:28.479000",
"timeout": "2024-02-01T15:58:28.479000",
"holdoff": null,
"owner": "admin",
"user_groups": []
}
```

To get all the nodes as a list, use the `/nodes` API endpoint:

```
$ curl http://localhost:8001/latest/nodes
[{"_id":"61b052199bca2a448fe49673","kind":"node","name":"checkout","revision":{"tree":"mainline","url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git","branch":"master","commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26","describe":"v5.16-rc4-31-g2a987e65025e"},"parent":null,"status":"pending","result":null, "created":"2022-02-01T11:23:03.157648", "updated":"2022-02-02T11:23:03.157648"},{"_id":"61b052199bca2a448fe49674","kind":"node","name":"check-describe","revision":{"tree":"mainline","url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git","branch":"master","commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26","describe":"v5.16-rc4-31-g2a987e65025e"},"parent":"61b052199bca2a448fe49673","status":"pending", "result":null,"created":"2022-01-02T10:23:03.157648", "updated":"2022-01-02T11:23:03.157648"}]
{
"items": [
{
"id": "65a1355ee98651d0fe81e412",
"kind": "node",
"name": "time_test_cases",
"path": [
"checkout",
"kunit-x86_64",
"exec",
"time_test_cases"
],
"group": "kunit-x86_64",
"parent": "65a1355ee98651d0fe81e40f",
"state": "done",
"result": null,
"artifacts": null,
"data": {
"kernel_revision": {
"tree": "mainline",
"url": "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch": "master",
"commit": "70d201a40823acba23899342d62bc2644051ad2e",
"describe": "v6.7-6264-g70d201a40823",
"version": {
"version": "6",
"patchlevel": "7",
"extra": "-6264-g70d201a40823"
}
}
},
"created": "2024-01-12T12:49:33.996000",
"updated": "2024-01-12T12:49:33.996000",
"timeout": "2024-01-12T18:49:33.996000",
"holdoff": null,
"owner": "admin",
"user_groups": []
},
{
"id": "65a1355ee98651d0fe81e413",
"kind": "node",
"name": "time64_to_tm_test_date_range",
"path": [
"checkout",
"kunit-x86_64",
"exec",
"time_test_cases",
"time64_to_tm_test_date_range"
],
"group": "kunit-x86_64",
"parent": "65a1355ee98651d0fe81e412",
"state": "done",
"result": "pass",
"artifacts": null,
"data": {
"kernel_revision": {
"tree": "mainline",
"url": "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch": "master",
"commit": "70d201a40823acba23899342d62bc2644051ad2e",
"describe": "v6.7-6264-g70d201a40823",
"version": {
"version": "6",
"patchlevel": "7",
"extra": "-6264-g70d201a40823"
}
}
},
"created": "2024-01-12T12:49:33.996000",
"updated": "2024-01-12T12:49:33.996000",
"timeout": "2024-01-12T18:49:33.996000",
"holdoff": null,
"owner": "admin",
"user_groups": []
},
...
```

To get nodes by providing attributes, use `/nodes` endpoint with query
parameters. All the attributes except node ID can be passed to this endpoint.
In case of ID, please use `/node` endpoint with node ID as described above.

```
$ curl 'http://localhost:8001/latest/nodes?name=checkout&revision.tree=mainline'
[{"_id":"61b052199bca2a448fe49673","kind":"node","name":"checkout","revision":{"tree":"mainline","url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git","branch":"master","commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26","describe":"v5.16-rc4-31-g2a987e65025e"},"parent":null,"status":"pending","result":null, "created":"2022-02-01T11:23:03.157648", "updated":"2022-02-02T11:23:03.157648"}]
$ curl 'http://localhost:8001/latest/nodes?kind=checkout&data.kernel_revision.tree=mainline' | jq
{
"items": [
{
"id": "65a3982ee98651d0fe82b010",
"kind": "checkout",
"name": "checkout",
"path": [
"checkout"
],
"group": null,
"parent": null,
"state": "done",
"result": null,
"artifacts": {
"tarball": "https://kciapistagingstorage1.file.core.windows.net/staging/linux-mainline-master-v6.7-9928-g052d534373b7.tar.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D"
},
"data": {
"kernel_revision": {
"tree": "mainline",
"url": "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch": "master",
"commit": "052d534373b7ed33712a63d5e17b2b6cdbce84fd",
"describe": "v6.7-9928-g052d534373b7",
"version": {
"version": "6",
"patchlevel": "7",
"extra": "-9928-g052d534373b7"
}
}
},
"created": "2024-01-14T08:15:42.454000",
"updated": "2024-01-14T09:16:47.689000",
"timeout": "2024-01-14T09:15:42.344000",
"holdoff": "2024-01-14T08:46:39.040000",
"owner": "admin",
"user_groups": []
},
{
"id": "65a3a545e98651d0fe82b4ed",
"kind": "checkout",
"name": "checkout",
"path": [
"checkout"
],
"group": null,
"parent": null,
"state": "done",
"result": null,
"artifacts": {
"tarball": "https://kciapistagingstorage1.file.core.windows.net/staging/linux-mainline-master-v6.7-9928-g052d534373b7.tar.gz?sv=2022-11-02&ss=f&srt=sco&sp=r&se=2024-10-17T19:19:12Z&st=2023-10-17T11:19:12Z&spr=https&sig=sLmFlvZHXRrZsSGubsDUIvTiv%2BtzgDq6vALfkrtWnv8%3D"
},
"data": {
"kernel_revision": {
"tree": "mainline",
"url": "https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch": "master",
"commit": "052d534373b7ed33712a63d5e17b2b6cdbce84fd",
"describe": "v6.7-9928-g052d534373b7",
"version": {
"version": "6",
"patchlevel": "7",
"extra": "-9928-g052d534373b7"
}
}
},
"created": "2024-01-14T09:11:33.029000",
"updated": "2024-01-14T10:11:48.092000",
"timeout": "2024-01-14T10:11:32.922000",
"holdoff": "2024-01-14T09:40:19.284000",
"owner": "admin",
"user_groups": []
}
...
```

Attributes along with comparison operators are also supported for the
`/nodes` endpoint. The attribute name and operator should be separated by `__` i.e. `attribute__operator`. Supported operators are `lt`(less than), `gt`(greater than), `lte`(less than or equal to), and `gte`(greater than or equal to).
`/nodes` endpoint. The attribute name and operator should be separated
by `__` i.e. `attribute__operator`. Supported operators are `lt`(less
than), `gt`(greater than), `lte`(less than or equal to), `gte`(greater
than or equal to) and `re` (regular expression matching).

```
$ curl 'http://localhost:8001/latest/nodes?name=checkout&created__gt=2022-12-06T04:59:08.102000'
{"items":[{"_id":"638ecc1c749a8d1209b758af","kind":"node","name":"checkout","path":["checkout"],"group":null,"revision":{"tree":"mainline","url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git","branch":"master","commit":"bce9332220bd677d83b19d21502776ad555a0e73","describe":"v6.1-rc8-3-gbce9332220bd","version":{"version":6,"patchlevel":1,"sublevel":null,"extra":"-rc8-3-gbce9332220bd","name":null}},"parent":null,"state":"done","result":"pass","artifacts":{"tarball":"http://172.17.0.1:8002/linux-mainline-master-v6.1-rc8-3-gbce9332220bd.tar.gz"},"created":"2022-12-06T04:59:08.959000","updated":"2022-12-06T05:11:33.098000","timeout":"2022-12-07T04:59:08.959000","holdoff":"2022-12-06T05:11:30.873000"}],"total":1,"limit":50,"offset":0}
$ curl 'http://localhost:8001/latest/nodes?kind=checkout&created__gt=2022-12-06T04:59:08.102000'
```

> **Note** In order to support comparison operators in URL request parameters, models can not contain `__` in the field name.
Additionally, the `re` operator offers some basic regular expression
matching capabilities for query parameters. For instance:

```
$ curl 'http://localhost:8001/latest/nodes?kind=kbuild&name__re=x86'
```

returns all Kbuild nodes with the string "x86" in the node name.

Nodes with `null` fields can also be retrieved using the endpoint.
For example, the below command will get all the nodes with `parent` field set to `null`:

```
$ curl 'http://localhost:8001/latest/nodes?parent=null'
"items":[{"_id":"63c549319fb3b62c7626e7f9","kind":"node","name":"checkout","path":["checkout"],"group":null,"revision":{"tree":"kernelci","url":"https://github.com/kernelci/linux.git","branch":"staging-mainline","commit":"1385303d0d85c68473d8901d69c7153b03a3150b","describe":"staging-mainline-20230115.1","version":{"version":6,"patchlevel":2,"sublevel":null,"extra":"-rc4-2-g1385303d0d85","name":null}},"parent":null,"state":"available","result":null,"artifacts":{"tarball":"http://172.17.0.1:8002/linux-kernelci-staging-mainline-staging-mainline-20230115.1.tar.gz"},"created":"2023-01-16T12:55:13.879000","updated":"2023-01-16T12:55:51.780000","timeout":"2023-01-16T13:55:13.877000","holdoff":"2023-01-16T13:05:51.776000"},{"_id":"63c549329fb3b62c7626e7fa","kind":"node","name":"checkout","path":["checkout"],"group":null,"revision":{"tree":"kernelci","url":"https://github.com/kernelci/linux.git","branch":"staging-next","commit":"39384a5d7e2eb2f28039a92c022aed886a675fbf","describe":"staging-next-20230116.0","version":{"version":6,"patchlevel":2,"sublevel":null,"extra":"-rc4-5011-g39384a5d7e2e","name":null}},"parent":null,"state":"available","result":null,"artifacts":{"tarball":"http://172.17.0.1:8002/linux-kernelci-staging-next-staging-next-20230116.0.tar.gz"},"created":"2023-01-16T12:55:14.706000","updated":"2023-01-16T12:56:30.886000","timeout":"2023-01-16T13:55:14.703000","holdoff":"2023-01-16T13:06:30.882000"}],"total":2,"limit":50,"offset":0}
"items":[{"_id":"63c549319fb3b62c7626e7f9","kind":"node","name":"checkout","path":["checkout"],"group":null,"data":{"kernel_revision":{"tree":"kernelci","url":"https://github.com/kernelci/linux.git","branch":"staging-mainline","commit":"1385303d0d85c68473d8901d69c7153b03a3150b","describe":"staging-mainline-20230115.1","version":{"version":6,"patchlevel":2,"sublevel":null,"extra":"-rc4-2-g1385303d0d85","name":null}}},"parent":null,"state":"available","result":null,"artifacts":{"tarball":"http://172.17.0.1:8002/linux-kernelci-staging-mainline-staging-mainline-20230115.1.tar.gz"},"created":"2023-01-16T12:55:13.879000","updated":"2023-01-16T12:55:51.780000","timeout":"2023-01-16T13:55:13.877000","holdoff":"2023-01-16T13:05:51.776000"},{"_id":"63c549329fb3b62c7626e7fa","kind":"node","name":"checkout","path":["checkout"],"group":null,"data":{"kernel_revision":{"tree":"kernelci","url":"https://github.com/kernelci/linux.git","branch":"staging-next","commit":"39384a5d7e2eb2f28039a92c022aed886a675fbf","describe":"staging-next-20230116.0","version":{"version":6,"patchlevel":2,"sublevel":null,"extra":"-rc4-5011-g39384a5d7e2e","name":null}}},"parent":null,"state":"available","result":null,"artifacts":{"tarball":"http://172.17.0.1:8002/linux-kernelci-staging-next-staging-next-20230116.0.tar.gz"},"created":"2023-01-16T12:55:14.706000","updated":"2023-01-16T12:56:30.886000","timeout":"2023-01-16T13:55:14.703000","holdoff":"2023-01-16T13:06:30.882000"}],"total":2,"limit":50,"offset":0}
```

Please make sure that the query parameter provided with the `null` value in the request exists in the `Node` schema. Otherwise, the API will behave unexpectedly and return all the nodes.
Expand All @@ -167,14 +393,18 @@ $ curl -X 'PUT' \
-H 'Content-Type: application/json' \
-d '{
"name":"checkout-test",
"revision":{"tree":"mainline",
"url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch":"master",
"commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26",
"describe":"v5.16-rc4-31-g2a987e65025e"},
"data":{
"kernel_revision":{
"tree":"mainline",
"url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git",
"branch":"master",
"commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26",
"describe":"v5.16-rc4-31-g2a987e65025e"
},
},
"created":"2022-02-02T11:23:03.157648"
}'
{"_id":"61bda8f2eb1a63d2b7152418","kind":"node","name":"checkout-test","revision":{"tree":"mainline","url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git","branch":"master","commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26","describe":"v5.16-rc4-31-g2a987e65025e"},"parent":null,"status":"pending","result":null, "created":"2022-02-02T11:23:03.157648", "updated":"2022-02-02T12:23:03.157648"}
{"id":"61bda8f2eb1a63d2b7152418","kind":"node","name":"checkout-test","data":{"kernel_revision":{"tree":"mainline","url":"https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git","branch":"master","commit":"2a987e65025e2b79c6d453b78cb5985ac6e5eb26","describe":"v5.16-rc4-31-g2a987e65025e"}},"parent":null,"status":"pending","result":null, "created":"2022-02-02T11:23:03.157648", "updated":"2022-02-02T12:23:03.157648"}
```

### Getting Nodes count
Expand All @@ -190,7 +420,7 @@ To get count of nodes matching provided attributes, use `/count` endpoint with q
```
$ curl http://localhost:8001/latest/count?name=checkout
3
$ curl http://localhost:8001/latest/count?revision.branch=staging-mainline
$ curl http://localhost:8001/latest/count?data.kernel_revision.branch=staging-mainline
1
```

Expand Down

0 comments on commit 64338ed

Please sign in to comment.