0.16.0

Release notes

Enhancements

• Updated StatusMsg and made it public in preparation for releasing a status schema
• Added StatusAction, StatusReason, TradingEvent, and TriState enums for use in
  the status schema
• Added -t and --tsv flags to DBN CLI to encode tab-separated values (TSV)
• Added delimiter method to builders for DynEncoder and CsvEncoder to customize the
  field delimiter character, allowing DBN to be encoded as tab-separated values (TSV)
• Document cancellation safety for AsyncRecordDecoder::decode_ref (credit: @yongqli)
• Added new publisher values for consolidated DBEQ.MAX
• Added C FFI conversion functions from ErrorMsgV1 to ErrorMsg and SystemMsgV1
  to SystemMsg
• Improved documentation for side field and Side enum
• Upgraded async-compression to 0.4.6
• Upgraded strum to 0.26

Breaking changes

• Changed default for VersionUpgradePolicy to Upgrade
• Changed default upgrade_policy for DbnDecoder, AsyncDbnDecoder, and Python
  DBNDecoder to Upgrade so by default the primary record types can always be used
• Changed fields of previously-hidden StatusMsg record type
• Updated text serialization order of status schema to match other schemas
• Changed text serialization unit_of_measure_qty to be affected by pretty_px. While
  it's not a price, it uses the same fixed-price decimal format as other prices
• Made StatType and VersionUpgradePolicy non-exhaustive to allow future additions
  without breaking changes
• Renamed _dummy field in ImbalanceMsg and StatMsg to _reserved
• Added ts_out parameter to RecordDecoder and AsyncRecordDecoder
with_upgrade_policy methods

Bug fixes

• Fixed handling of ts_out when upgrading DBNv1 records to version 2
• Added missing StatType::Vwap variant used in the ICE datasets
• Fixed an issue with Python stub file distribution
• Fixed missing handling of ErrorMsgV1 and SystemMsgV1 in rtype dispatch macros

0.15.1

Release notes

Bug fixes

• Fixed an import error in the Python type stub file

0.15.0

Release notes

Enhancements

• Improved Debug implementation for all record types
  • Prices are formatted as decimals
  • Fixed-length strings are formatted as strings
  • Bit flag fields are formatted as binary
  • Several fields are formatted as enums instead of their raw representations
• Improved Debug implementation for RecordRef to show RecordHeader
• Added --schema option to dbn CLI tool to filter a DBN to a particular schema. This
  allows outputting saved live data to CSV
• Allowed passing --limit option to dbn CLI tool with --metadata flag
• Improved performance of decoding uncompressed DBN fragments with the dbn CLI tool
• Added builders to CsvEncoder, DynEncoder, and JsonEncoder to assist with the
  growing number of customizations
  • Added option to write CSV header as part of creating CsvEncoder to make it harder
    to forget
• Added -s/--map-symbols flag to CLI to create a symbol field in the output with
  the text symbol mapped from the instrument ID
• Added version param to Python Metadata contructor choose between DBNv1 and DBNv2
• Implemented EncodeRecordTextExt for DynEncoder
• Implemented Deserialize and Serialize for all records and enums (with serde
  feature enabled). This allows serializing records with additional encodings not
  supported by the DBN crate
• Implemented Hash for all record types
• Added new publisher value for OPRA MIAX Sapphire
• Added Python type definition for Metadata.__init__
• Added metadata_mut method to decoders to get a mutable reference to the decoded
  metadata
• Improved panic message on RecordRef::get when length doesn't match expected to be
  actionable
• Added encode::ZSTD_COMPRESSION_LEVEL constant

Breaking changes

• Increased size of SystemMsg and ErrorMsg to provide better messages from Live
  gateway
  • Increased length of err and msg fields for more detailed messages
  • Added is_last field to ErrorMsg to indicate the last error in a chain
  • Added code field to SystemMsg and ErrorMsg, although currently unused
  • Added new is_last parameter to ErrorMsg::new
  • Decoding these is backwards-compatible and records with longer messages won't be
    sent during the DBN version 2 migration period
  • Renamed previous records to compat::ErrorMsgV1 and compat::SystemMsgV1
• Split DecodeDbn trait into DecodeRecord and DbnMetadata traits for more
  flexibility. DecodeDbn continues to exist as a trait alias
• Moved decode_stream out of DecodeDbn to its own separate trait DecodeStream
• Changed trait bounds of EncodeDbn::encode_decoded and encode_decoded_with_limit to
  DecodeRecordRef + DbnMetadata

Bug fixes

• Fixed panic in TsSymbolMap when start_date == end_date
• Added missing Python __eq__ and __ne__ implementations for BidAskPair
• Fixed Python size_hint return value for InstrumentDefMsgV1 and
  SymbolMappingMsgV1
• Fixed cases where dbn CLI tool would write a broken pipe error to standard error
  such as when piping to head
• Fixed bug in sync and async MetadataEncoders where version was used to determine
  the encoded length of fixed-length symbols instead of the symbol_cstr_len field

0.14.2

Release notes

Enhancements

• Added set_upgrade_policy setters to DbnDecoder, DbnRecordDecoder,
  AsyncDbnDecoder, and AsyncDbnRecordDecoder
• Added from_schema classmethod for Python RType enum

Breaking changes

• Renamed parameter for Python Enum classmethod constructors to value from data.

0.14.1

Release notes

Enhancements

• Added new trait compat::SymbolMappingRec for code reuse when working with
  both versions of SymbolMappingMsg
• Changed PitSymbolMap::on_symbol_mapping to accept either version of
  SymbolMappingMsg

Bug fixes

• Fixed missing DBNv1 compatibility in PitSymbolMap::on_record
• Fixed missing Python export for VersionUpgradePolicy
• Fixed missing Python export and methods for InstrumentDefMsgV1 and
  SymbolMappingMsgV1
• Fixed bug where Python DbnDecoder and Transcoder would throw exceptions
  when attempting to decode partial metadata

0.14.0

Release notes

Enhancements

• This version begins the transition to DBN version 2 (DBNv2). In this version, the
  decoders support decoding both versions of DBN and the DBN encoders default to
  keeping version of the input. However, in a future version, decoders will by default
  convert DBNv1 to DBNv2 and support will be dropped for encoding DBNv1.
  • Affects SymbolMappingMsg, InstrumentDefMsg, and Metadata. All other record
    types and market data schemas are unchanged
  • Version 1 structs can be converted to version 2 structs with the From trait
• Added symbol_cstr_len field to Metadata to indicate the length of fixed symbol
  strings
• Added stype_in and stype_out fields to SymbolMappingMsg to provide more context
  with live symbology updates
• Added smart wrapping to dbn CLI help output
• Updated rtype_dispatch family of macros to check record length to handle both
  versions of records. This is temporary during the transition period
• Added VersionUpgradePolicy enum and associated methods to the decoders to
  allow specifying how to handle decoding records from prior DBN versions
• Added Metadata::upgrade() method to update Metadata from a prior DBN version to
  the latest version
• Added -u/--upgrade flags to dbn CLI that when passed upgrades DBN data from
  previous versions. By default data is decoded as-is
• Made AsyncDbnDecoder::decode_record, AsyncDbnDecoder::decode_record_ref,
  dbn::AsyncRecordDecoder::decode, and dbn::AsyncRecordDecoder::decode_ref
  cancellation safe. This makes them safe to use within a
  tokio::select!(https://docs.rs/tokio/latest/tokio/macro.select.html) statement
• Added documention around cancellation safety for async APIs
• Improved error messages for conversion errors
• Added TOB flag to denote top-of-book messages
• Added new publisher values in preparation for IFEU.IMPACT and NDEX.IMPACT datasets
• Added new publisher values for consolidated DBEQ.BASIC and DBEQ.PLUS
• Added MAX_RECORD_LEN constant for the length of the largest record type
• Exposed record flag constants in databento_dbn with F_ prefix
• Added export to Python for RType

Breaking changes

• The old InstrumentDefMsg is now compat::InstrumentDefMsgV1
• compat::InstrumentDefMsgV2 is now an alias for InstrumentDefMsg
• The old SymbolMappingMsg is now compat::SymbolMappingMsgV1
• compat::SymbolMappingMsgV2 is now an alias for SymbolMappingMsg
• Changed SYMBOL_CSTR_LEN constant to 71. Previous value is now in
  compat::SYMBOL_CSTR_V1
• Changed DBN_VERSION constant to 2
• security_update_action was converted to a raw c_char to safely support adding
  variants in the future
• Renamed _dummy in InstrumentDefMsg to _reserved
• Removed _reserved2, _reserved3, and _reserved5 from InstrumentDefMsg
• Removed _dummy from SymbolMappingMsg
• Moved position of strike_price within InstrumentDefMsg but left text serialization
  order unchanged
• Made Error non-exhaustive, meaning it can no longer be exhaustively matched against.
  This allows adding additional error variants in the future without a breaking change
• Added upgrade_policy parameter to RecordDecoder::with_version constructor to
  control whether records of previous versions will be upgraded
• Added upgrade_policy parameter to DynDecoder constructors to control whether
  records of previous versions will be upgraded
• Renamed symbol_map parameter for Python Transcoder to symbol_interval_map to
  better reflect the date intervals it contains

Deprecations

• Deprecated unused write_dbn_file function from Python interface. Please use
  Transcoder instead

Bug fixes

• Fixed typo in Python type definition for InstrumentDefMsg.pretty_high_limit_price
• Fixed type signature for Metadata.stype_in and Metadata.stype_out Python methods
• Fixed incorrect version in pyproject.toml

0.13.0

Release notes

Enhancements

• Added SymbolMappingMsgV2::new method
• Added Record trait for all types beginning with a RecordHeader
  • Added new index_ts and raw_index_ts methods to Record trait, which returns the
    primary timestamp for a record
• Added RecordMut trait for accessing a mutable reference to a RecordHeader
• Implemented PartialOrd for all record types, based on raw_index_ts
• Loosened DbnEncodable from requiring HasRType to only requiring Record. This means
  RecordRefs and concrete records can be encoded with the same methods

Breaking changes

• Split part of HasRType into new Record and RecordMut traits, which are object-
  safe: they can be used in Box<dyn>. RecordRef also implements Record, so it's
  easier to write code that works for both concrete records as well as RecordRef
• Removed RecordRef methods made redundant by it implementing Record
• Removed input_compression parameter from Python Transcoder

Deprecations

• Deprecated SymbolIndex::get_for_rec_ref, which was made redundant by loosening the
  trait bound on SymbolIndex::get_for_rec to accept RecordRefs

Bug fixes

• Fixed TsSymbolMap not always using the correct timestamp for getting the mapped
  symbol

0.12.0

Release notes

Enhancements

• Added map_symbols support to Python Transcoder
• Added new publisher variants in preparation for DBEQ.PLUS dataset
• Added from_dataset_venue function to Publisher to facilitate destructuring
• Implemented Default for most records to make testing easier
• Added from_zstd function to AsyncDbnEncoder to match synchronous encoder
• Added re-exports for enums::flags, enums::rtype, record::BidAskPair,
  record::RecordHeader, and record::WithTsOut to simplify imports
• Added --fragment CLI flag for writing DBN without the metadata header
• Added --input-dbn-version CLI option for specifying the DBN version of a DBN
  fragment
• Added serde::Deserialize implementations for Dataset, Venue, and Publisher
• Added support for Python 3.12 to databento_dbn
• Added RecordDecoder::with_version for future use when dealing with compatibility
  between different DBN versions
• Added new dispatch macros: rtype_ts_out_method_dispatch,
  rtype_ts_out_async_method_dispatch, rtype_method_dispatch, and
  schema_ts_out_method_dispatch
• Added InstrumentDefMsgV2 and SymbolMappingMsgV2 for forward compatibility with a
  version of DBN
• Added TsSymbolMap and PitSymbolMap to aid with both historical and live symbology
  • Added support for inverse symbology, i.e. with stype_in=InstrumentId

Breaking changes

• Changed Metadata::symbol_map to return TsSymbolMap
• Changed Metadata::symbol_map_for_date to return PitSymbolMap
• Changed Default implementation for BidAskPair by setting prices to UNDEF_PRICE
• Added new publisher values in preparation for DBEQ.PLUS
• Added ts_out parameter to encode_header_for_schema in CsvEncoder and
  DynEncoder to allow controlling whether "ts_out" is in the header
• Added from_dataset_venue function to Publisher to facilitate
  destructuring.

0.11.1

Release notes

Enhancements

• Upgraded async-compression to 0.4.3
• Upgraded csv to 1.3
• Upgraded num_enum to 0.7

Bug fixes

• Changed DBN stream detection to ignore the DBN version

0.11.0

Release notes

Enhancements

• Added new EncodeRecordTextExt trait which is implemented for the CSV and JSON
  encoders. It adds two methods for encoding a symbol field along side the rest of the
  record fields, matching the behavior of map_symbols in the historical API
• Added encode_header and encode_header_for_schema methods to CsvEncoder and
  DynEncoder to give more flexibility for encoding CSV headers
• Added from_file and from_zstd_file functions to AsyncDbnDecoder to match
  synchronous decoder
• Implemented Copy for RecordRef to make it behave more like a reference
• Added AsyncDbnEncoder for simpler DBN encoding and to match sync API
• Added RecordEnum and RecordRefEnum to more easily be able to pattern match on
  records of different types
• Added ARCX.PILLAR.ARCX publisher
• Added From DBN records for RecordRef
• Added re-exports to the top level of the crate for all enums and records for simpler
  imports
• Added ClosePrice and NetChange StatTypes used in the OPRA.PILLAR dataset

Breaking changes

• Split encode_record_ref into a safe method with no arguments and an unsafe method
  with a ts_out parameter to reduce unsafe usage when not working with live data
  that may contain ts_out

Bug fixes

• Fixed dbn CLI not writing CSV header when using --fragment and --zstd-fragment
  flags
• Fixed lifetime on return value from RecordRef::get_unchecked
• Fixed missing check for stype_out before building Metadata symbology maps