-
Notifications
You must be signed in to change notification settings - Fork 39
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
This is the first commit in a thread of work to support updates to timeseries schema. In our first step, we're moving the definition of timeseries from the Rust structs we use today, to TOML text files. Each file describes one target and all the metrics associated with it, from which we generate exactly the same Rust code that it replaces. This opens the door to a lot of downstream work, including well-defined updates to schema; detection of conflicting schema; improved metadata for timeseries; and generation of documentation. As an example, this moves exactly one (1) timeseries into the new format, the physical datalink statistics tracked through illumos kstats. - Move the `oximeter` crate into a private implementation crate, and re-export it at its original path - Add intermediate representation of timeseries schema, and code for deserializing TOML timeseries definitions, checking them for conflicts, and handling updates. Note that **no actual updates** are currently supported. The ingesting code includes a check that version numbers are exactly 1, except in tests. We include tests that check updates in a number of ways, but until the remainder of the work is done, we limit timeseries in the wild to their current version of 1. - Add a macro for consuming timeseries definitions in TOML, and generating the equivalent Rust code. Developers should use this new `oximeter::use_timeseries!()` proc macro to create new timeseries. - Add an integration test in Nexus that pulls in timeseries schema ingested with with `oximeter::use_timeseries!()`, and checks them all for conflicts. As developers add new timeseries in TOML format, they must be added here as well to ensure we detect problems at CI time. - Updates `kstat-rs` dep to simplify platform-specific code. This is part of this commit because we need the definitions of the physical-data-link timeseries for our integration test, but those were previously only compiled on illumos platforms. They're now included everywhere, while the tests remain illumos-only.
- Loading branch information
Showing
42 changed files
with
2,772 additions
and
527 deletions.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| // This Source Code Form is subject to the terms of the Mozilla Public | ||
| // License, v. 2.0. If a copy of the MPL was not distributed with this | ||
| // file, You can obtain one at https://mozilla.org/MPL/2.0/. | ||
|
|
||
| use std::collections::BTreeMap; | ||
|
|
||
| /// This test checks that changes to timeseries schema are all consistent. | ||
| /// | ||
| /// Timeseries schema are described in a TOML format that makes it relatively | ||
| /// easy to add new versions of the timeseries. Those definitions are ingested | ||
| /// at compile-time and checked for self-consistency, but it's still possible | ||
| /// for two unrelated definitions to conflict. This test catches those. | ||
| /// | ||
| /// As developers define new timeseries, those should be added to this test for | ||
| /// checking. To define metrics in their own crates, developers use the | ||
| /// proc-macro `oximeter::use_timeseries!()` | ||
| #[test] | ||
| fn timeseries_schema_consistency() { | ||
| let mut all_schema = BTreeMap::new(); | ||
| for list in | ||
| [oximeter_instruments::kstat::link::physical_data_link::timeseries_schema()] | ||
| { | ||
| for schema in list { | ||
| let key = (schema.timeseries_name.clone(), schema.version); | ||
| if let Some(dup) = all_schema.insert(key, schema.clone()) { | ||
| assert_eq!( | ||
| dup, schema, | ||
| "Timeseries '{}' version {} is duplicated, but the \ | ||
| schema themselves differ. This will lead to a conflict \ | ||
| at registration time, so at least one of the schema \ | ||
| should be renamed, re-versioned, or include different \ | ||
| fields to differentiate them.", | ||
| dup.timeseries_name, dup.version, | ||
| ); | ||
| }; | ||
| } | ||
| } | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.