XMas-Day-4: DataSet<T>: helpers, estimators, math functions, restructure DataSet

[RalphSteinhagen]

On the fourth day of Christmas 🎄 👼 4️⃣ 🎄 , my code base gave to me...

Helpers, estimators, and math functions, oh my!

This PR enhances the DataSet<T> by introducing:

1. Helper Methods:

   • Added convenience accessors to retrieve individual axes and signals as std::span<T> and std::span<const T> depending on constness.

2. DataSet Restructure:

   • Introduced Range<T> with explicit min and max members for clear range definitions.
   • Removed signal_error in favor of using UncertainValue<T> to represent uncertainties.

3. UncertainValue Enhancements:

   • Added missing isfinite and log10 functions for better mathematical operations.
   • Implemented the << operator to facilitate easy formatting and output.

4. Estimators and Math Functions:

   • Introduced simple estimators for data analysis like mean, median, RMS, etc.
   • Implemented basic math functions to perform operations directly on DataSet<T>.

5. Unit Tests:

   • Added comprehensive unit tests for the new estimators and math functionalities.
   • Updated existing tests to align with the DataSet restructuring.

6. Utility Updates:

   • Updated accessor and utility functions to support the new features and ensure seamless integration.

7. Documentation:

   • Updated doxygen comments and added code examples to illustrate the new methods and features.

Feel free to explore the new additions and share your feedback!
Happy coding and joyful holidays! 🎅🤶🎁

Code Example:

#include <gnuradio-4.0/algorithm/dataset/DataSetTestFunctions.hpp>
#include <gnuradio-4.0/algorithm/dataset/DataSetEstimators.hpp>
#include <gnuradio-4.0/algorithm/dataset/DataSetMath.hpp>

int main() {
    using namespace gr::dataset;

    // Generate a triangular DataSet
    auto triagDS = generate::triangular<double>("triag", 11, 0.0, 1.0);
    gr::dataset::draw(triagDS); // Visualize the DataSet

    // Generate a ramp DataSet
    auto rampDS = generate::ramp<double>("ramp", 11, 0.0, 0.2);
    gr::dataset::draw(rampDS); // Visualize the DataSet

    // Generate a Gaussian DataSet
    auto gaussDS = generate::gaussFunction<double>("gauss", 11, 5.0, 1.0, 0.0, 1.0);
    gr::dataset::draw(gaussDS); // Visualize the DataSet

    // Using Estimators
    double meanTriag = estimators::getMean(triagDS);
    double medianTriag = estimators::getMedian(triagDS);
    double rmsTriag = estimators::getRms(triagDS);
    double fwhmTriag = estimators::computeFWHM(triagDS.signalValues(0), 5); // Assuming peak at index 5

    fmt::print("Triangular DataSet - Mean: {}, Median: {}, RMS: {}, FWHM: {}\n", meanTriag, medianTriag, rmsTriag, fwhmTriag);

    // Performing Math Operations
    auto dsAdded = estimators::addFunction(triagDS, 2.0); // Adds 2 to all signal values
    gr::dataset::draw(dsAdded); // Visualize the modified DataSet

    auto dsCombined = estimators::addFunction(rampDS, gaussDS); // Adds two DataSets
    gr::dataset::draw(dsCombined); // Visualize the combined DataSet

    // Frequency Estimate
    double freqEstimate = estimators::getFrequencyEstimate(dsCombined);
    fmt::print("Frequency Estimate of Combined DataSet: {}\n", freqEstimate);

    // Edge Detection
    double edgeDetect = estimators::getEdgeDetect(dsCombined, 0.5);
    fmt::print("Edge Detection at 50% Threshold: {}\n", edgeDetect);

    // Integral Calculation
    double integral = estimators::getIntegral(dsCombined);
    fmt::print("Integral of Combined DataSet: {}\n", integral);

    return 0;
}

P.S. Did I mention that I like unit-tests?

[RalphSteinhagen]

Added applyFilter(..) implementation:

Example:

for (const auto& zeta : {0.01, 0.1, 0.99}) {
    auto                filter = Filter<double>(iir::designResonatorPhysical(fs, 10., zeta));
    std::vector<double> yValue(xValues.size());
    std::vector<double> yFiltered(xValues.size());
    std::transform(xValues.cbegin(), xValues.cend(), yValue.begin(), [](double t) { return t < 0.05 ? 0.0 : 1.0; });
    std::transform(yValue.cbegin(), yValue.cend(), yFiltered.begin(), [&filter](double y) { return filter.processOne(y); });
    expect(nothrow([&] { chart.draw(xValues, yFiltered, fmt::format("resonance@zeta={}", zeta)); })) << fmt::format("resonance@zeta={} does not throw", zeta);
}

expect(nothrow([&] { chart.draw(); }));

[RalphSteinhagen]

added estimators::analyzeStepPulseResponse(...). Example:

Includes (incl. optional tags):

• (first-edge) trigger time
• 10%-90% rise-time
• overshoot peak position
• relative overshoot
• settling time
• initial- and steady-state level

[drslebedev]

Thanks a lot for the PR!

The code in this PR looks very good and is thoroughly tested.

There are a few suggestions regarding consistency and optimizations that I believe should be reviewed before merge.

Once those points are resolved, this PR will be ready for merging.

[drslebedev]

Should be checked for std::nullopt and return std::numeric_limits<TValue>::quiet_NaN(), otherwise an exception can be thrown.

[RalphSteinhagen]

This is guaranteed to always have a value in this context of the earlier checkIndexRange(..) check above and also the same check within the get[Minimum, Maximum](...) functions.

[drslebedev]

Same as for computeFWHM. Should be clarified in the documentation that function returns a difference in indices but not actual values.

[RalphSteinhagen]

I think this should be clear from the context here since there is no x-axis provided. Thus any width is always expressed in terms of the indexing.

This is different for the FWHM estimators that work on DataSet<T> though.

[drslebedev]

Not only here but a general question, should we stick to use more std::ranges?

Suggested change

	T sum = 0;
	T count = 0;
	for (const auto& val : finiteValues) {
	sum += val;
	count += T(1);
	}
	T sum = std::accumulate(finiteValues.begin(), finiteValues.end(), T(0));
	T count = static_cast<T>(std::ranges::distance(finiteValues));

[RalphSteinhagen]

Didn't use it here because this performs two accumulates/counting. To note this is because the finiteValues and signalRange views can have different sizes depending on how many actual 'NaN' are present in the data range.

We should discuss from a design point-of-view, whether we support or intercept 'NaN' floating point values. I took this implementation/choice based upon one of our other DataSet implementing math libraries (in Chart-FX) that supports 'NaN' by default. However, this choice is opinionated and we may equally disallow or explicitly not support this for GR4.

[sonarqubecloud]

Quality Gate failed

Failed conditions
62.7% Coverage on New Code (required ≥ 80%)
3.9% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud