Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add origin_url field #94

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add origin_url field #94

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions kcidb/db/schema.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,10 @@

# Test environment fields
ENVIRONMENT_FIELDS = (
Field(
"origin_url", "STRING",
description="The URL of the environment in the origin CI system",
),
Field(
"description", "STRING",
description="Human-readable description of the environment",
Expand All @@ -36,6 +40,10 @@
description="The name of the CI system which submitted "
"the revision",
),
Field(
"origin_url", "STRING",
description="The URL of the revision in the origin CI system",
),
Field(
"tree_name", "STRING",
description="The widely-recognized name of the sub-tree (fork) "
Expand Down Expand Up @@ -128,6 +136,10 @@
description="The name of the CI system which submitted "
"the build",
),
Field(
"origin_url", "STRING",
description="The URL of the build in the origin CI system",
),
Field(
"description", "STRING",
description="Human-readable description of the build",
Expand Down Expand Up @@ -199,6 +211,10 @@
description="The name of the CI system which submitted "
"the test run",
),
Field(
"origin_url", "STRING",
description="The URL of the test run in the origin CI system",
),
Field(
"environment", "RECORD", fields=ENVIRONMENT_FIELDS,
description="The environment the test ran in. "
Expand Down
29 changes: 29 additions & 0 deletions kcidb/io/schema/v3.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,11 @@
"The name of the CI system which submitted the revision",
"pattern": f"^{ORIGIN_PATTERN}$",
},
"origin_url": {
"type": "string",
"format": "uri",
"description": "The URL of the revision in the origin CI system",
},
"tree_name": {
"type": "string",
"description":
Expand Down Expand Up @@ -265,6 +270,15 @@
"The name of the CI system which submitted the build",
"pattern": f"^{ORIGIN_PATTERN}$",
},
"origin_url": {
"type": "string",
"format": "uri",
"description": "The URL of the build in the origin CI system",
"examples": [
"https://kernelci.org/build/net-next/branch/master/"
"kernel/v5.8-rc4-1414-g4ff91fa0a3ac/",
],
},
"description": {
"type": "string",
"description":
Expand Down Expand Up @@ -389,6 +403,14 @@
"The name of the CI system which submitted the test run",
"pattern": f"^{ORIGIN_PATTERN}$",
},
"origin_url": {
"type": "string",
"format": "uri",
"description": "The URL of the test run in the origin CI system",
"examples": [
"https://kernelci.org/test/case/id/5f0e86fc459ceb8c2885bb39/",
],
},
"environment": {
"type": "object",
"description":
Expand All @@ -397,6 +419,13 @@
"amount of memory/storage/CPUs, for each host; "
"process environment variables, etc.",
"properties": {
"origin_url": {
"type": "string",
"format": "uri",
"description":
"The URL of the environment in the origin CI system",
"examples": ["https://kernelci.org/soc/allwinner/"],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not really an environment, it's a family of devices from a same vendor.

I don't think there is anything that matches the field description on the kernelci.org dashboard. A link to the test platform in a LAVA lab would probably be a bit more relevant, for example:
https://lava.collabora.co.uk/scheduler/device/bcm2836-rpi-2-b-cbg-0

although that's not stored in the kernelci-backend database. What is stored is the name of the test lab and the name of the platform i.e. bcm2836-rpi-2-b for this Raspberry Pi 2b, which is more generic than a specific instance in a test lab.

Does CKI have some view to show details of a runtime environment or test platform?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the "environment" as a way to identify something where test executed, with some precision. As much precision as the submitter can afford. Its only purpose for KCIDB itself is to determine which tests executed in a similar-enough environment, so e.g. we can say the results should be the same, and can group them in the report, the dashboard, or take into account when locating the breaking commit. I still don't know how exactly I would implement or organize this, though.

If KernelCI doesn't expose the reported environment on the dashboard with similar precision, then it can choose not to provide a link here, or provide this link, even though it's of lower precision, just to have something. Or it can provide the Lava link you post.

For the purpose of example, I think the link here is OK. The Lava link would be better, though. Would you mind me using it even though Kernel CI wouldn't provide it?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CKI only has hostnames, I think, we can always link to Beaker which has very detailed description of the host. That might never be public, though, so we will probably not going to be using it, instead providing as much information as we can in the environment object itself (once we have the fields described).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KernelCI uses device types, which is basically a name for an "execution environment". At least that covers the immutable part of the environment, i.e. a hardware board with some firmware or a virtual device with a particular configuration. Then each test has some extra parts of the environment such as a root file system or a Docker image with test suites, which changes sometimes but is still part of the environment. The real moving part is the kernel.

So if different labs have the same Raspberry Pi, or a lab has several of them, results for any of them will appear as for the same device type. There just isn't a view on the current dashboard to show all the information specific to a particular device type, or any particular device instance.

I see the "environment" as a way to identify something where test executed, with some precision. As much precision as the submitter can afford.

That's the lab name and device type name as far we're concerned at the moment. I believe the actual instance name is also stored in the database although not shown on the dashboard, at least I think the field for it is still there.

I see the value of this kind of meta-data. But to me, that's rather different to a URL on a web interface.

So rather than using origin_url fields, maybe something like origin_metadata could be used with more arbitrary fields depending on the submitter? For LAVA labs it will be the lab name and the device type, and maybe device instance name. For your CKI/Beaker results, it will be the hostname or whatever works from your point of view.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. Thank you for explaining how Kernel CI identifies the devices, it will help me come up with a schema to actually support it. I think we more or less in agreement on what the essence of an environment is, and its importance.

Now, origin_url has nothing to do with identifying the environment. That's a job for yet-to-be defined fields.

Regarding origin_metadata, we have misc exactly for that, in environments as well.

The origin_url is just an escape hatch, for humans to reach the origin's representation of the same object (if available), with more data and more features than the implementation-in-progress can afford. In this way, it is similar to misc, which actually worries me, because it would be easier to just plop the link to your own web UI instead of submitting the data we might need to store and correlate. That would be an argument against it, IMO, and one I'm starting to find more and more weighty. Hmm...

},
"description": {
"type": "string",
"description":
Expand Down