1.5.7

GRNOC TSDS Services 1.5.7 -- Thu Mar 13 2019

Features:

• Added "align" clause to aggregate() calls. This feature does more intelligent time alignment
  than straight epoch seconds math, such as weeks or months that may start/end on inconsistent extents.
  It also does an implicit level of aggregation to that, so aligning to week will generate a weekly
  average or max, depending on what aggregate it's attached to. This allows queries to specify a different
  level of aggregation to fetch data vs how to combine it without resorting to subqueries in all cases.

  e.g. aggregate(values.input, timeframe, average) align week

• Added ability to use "count" as an aggregator function in aggregate() calls. This allows queries
  to return the number of datapoints in a bucket independent of their values.

• Refactored some of the unit tests to include longer duration interfaces, and to run the Writer
  process more efficiently.

Bugs:

• Fixed issue where histograms returned from inner queries were not processed correctly in outer queries
  making using of the aggregate() function.

1.6.3

GRNOC TSDS Services 1.6.3 -- Thu Mar 13 2019

Features:

• Added "align" clause to aggregate() calls. This feature does more intelligent time alignment
  than straight epoch seconds math, such as weeks or months that may start/end on inconsistent extents.
  It also does an implicit level of aggregation to that, so aligning to week will generate a weekly
  average or max, depending on what aggregate it's attached to. This allows queries to specify a different
  level of aggregation to fetch data vs how to combine it without resorting to subqueries in all cases.

  e.g. aggregate(values.input, timeframe, average) align week

• Added ability to use "count" as an aggregator function in aggregate() calls. This allows queries
  to return the number of datapoints in a bucket independent of their values.

• Refactored some of the unit tests to include longer duration interfaces, and to run the Writer
  process more efficiently.

• Added tsds_setup script to make first time installation and setup of pieces such as Mongo more
  streamlined and automated.

Bugs:

• Fixed issue where histograms returned from inner queries were not processed correctly in outer queries
  making using of the aggregate() function.

1.6.2-4

Minor bug fixes to 1.6.2

• Fixes issue with meta stats being incorrectly reported on queries with complex meta fields.

1.6.2-3

Minor bug fixes to 1.6.2.

• Fixed issue with grouping by complex field names introduced in 1.6.2

1.6.2-2

Minor bugfix for 1.6.2

• Fixes issue with "total" meta field not being reported correctly in certain cases.

1.6.2

GRNOC TSDS Services 1.6.2 -- Thu Nov 15 2018

Features:

• ISSUE=8708 Substantially reduce memory usage in some cases by pre-processing documents as they
  stream from mongo instead of post accumulation.

• ISSUE=8332 Changed "dirty" flag in documents from a timestamp to a boolean to increase index
  efficiency. This requires working with tsds-aggregate >= 1.2.0, as a similar change was made there.

• ISSUE=9001 Changed calls to perl MongoDB's database_names to a more direct call that can
  utilize the nameOnly parameter, avoiding excessive MongoDB locking during a lot of operations.

• ISSUE=9002 Improved speed when combining histograms in a number of common cases by avoiding
  recalculating indexes when the histograms shared the same dimensions.

• Optimized to remove an unneeded query starting in Mongo 3.4 by utilizing $facet operator. This
  allows for count of total hints + pagination to work in a single query instead of two.

• Optimized to remove testing for $exists in fields that will always be present.

• Reduced overall amount of database connections used by lazy connecting in several places, collapsing
  certain cases to a single connection, and using a singleton approach for cases where multiple
  libraries may be making the same connection.

1.5.6

GRNOC TSDS Services 1.5.6 -- Thu Nov 15 2018

Features:

• ISSUE=8708 Substantially reduce memory usage in some cases by pre-processing documents as they
  stream from mongo instead of post accumulation.

• ISSUE=9002 Improved speed when combining histograms in a number of common cases by avoiding
  recalculating indexes when the histograms shared the same dimensions.

• Optimized to remove testing for $exists in fields that will always be present.

• Reduced overall amount of database connections used by lazy connecting in several places, collapsing
  certain cases to a single connection, and using a singleton approach for cases where multiple
  libraries may be making the same connection.

Bugs:

• Made the start/end consideration in all queries more efficient by removing unneeded cases.

1.5.5

GRNOC TSDS Services 1.5.5 -- Tue Jul 24 2018

Features:

• ISSUE=8029 Refactor the way the data and metadata documents are merged during queries to
  be a lot more efficient, particularly in high metadata cardinality cases.

• ISSUE=6517 Added support for another argument to the extrapolate function to request a series
  of datapoints back. This uses the same logic but instead of returning a single number returns
  an array of points using the calculated slope/intercept so that it can be easily plotted.

• ISSUE=5336 Changed the way expiring measurement entries is internally calculated to be much
  more efficient. This now makes use of index only queries to find the needed data instead of
  performing a lot of fetches.

• Allow for queries on the "identifier" field. This is useful as a shorthand because "identifier"
  is used to unique identify a particular measurement series.

• Improved speed at which we process metadata documents that do not have any associated data
  documents. This also improves the speed of any query where no data fields are selected.

Bugs:

• ISSUE=5973 Fixed issue where queries with subqueries and math operations that also generated
  very large numbers would get handled incorrectly. This was a subtle function of how perl
  represents very large numbers in a string context.

• ISSUE=7470 Fixed issue where updating metadata with all of the same values but a different
  end time could result in duplicate in service docs.

• Fixed issue where trying to perform math operations using numeric metadata fields as one
  of the operands would crash, such as values.input / max_bandwidth.

1.6.1

GRNOC TSDS Services 1.6.1 -- Tue Jul 24 2018

Features:

• ISSUE=8029 Refactor the way the data and metadata documents are merged during queries to
  be a lot more efficient, particularly in high metadata cardinality cases.

• ISSUE=6517 Added support for another argument to the extrapolate function to request a series
  of datapoints back. This uses the same logic but instead of returning a single number returns
  an array of points using the calculated slope/intercept so that it can be easily plotted.

• ISSUE=5336 Changed the way expiring measurement entries is internally calculated to be much
  more efficient. This now makes use of index only queries to find the needed data instead of
  performing a lot of fetches.

• Allow for queries on the "identifier" field. This is useful as a shorthand because "identifier"
  is used to unique identify a particular measurement series.

• Improved speed at which we process metadata documents that do not have any associated data
  documents. This also improves the speed of any query where no data fields are selected.

Bugs:

• ISSUE=5973 Fixed issue where queries with subqueries and math operations that also generated
  very large numbers would get handled incorrectly. This was a subtle function of how perl
  represents very large numbers in a string context.

• ISSUE=7470 Fixed issue where updating metadata with all of the same values but a different
  end time could result in duplicate in service docs.

• Fixed issue where trying to perform math operations using numeric metadata fields as one
  of the operands would crash, such as values.input / max_bandwidth.

1.6.0

GRNOC TSDS Services 1.6.0 -- Fri Nov 17 2017

Features:

• ISSUE=3222 Updates to using perl driver 1.0.0+ and Mongo 3.4. This release is backwards
  incompatible with hosts running earlier versions of the driver, since the driver is itself
  backwards incompatible.

• Added systemd unit files for EL7.

• Allow updates of metadata where only the end time is set, ie decomission a measurement.

• Update Image service to use https to better interoperate with browsers implementing hsts.