Skip to content

v4.4.0
Compare
Choose a tag to compare
@github-actions github-actions released this 05 Apr 13:24
· 345 commits to master since this release
v4.4.0
efeb3c2

🚀 Features

  • Prevent writing empty types by default (gives compatibility with v1.6.2 readers) in #1440
  • Improved resilience to external (out of order) replication in #1355
  • Support modifying library options, and introduce enterprise library options in #1457

You can now modify library options on an existing Arctic library:

from arcticdb.options import LibraryOptions

ac: Arctic = ...
lib = ac.create_library("lib")
ac.modify_library_option(lib, ModifiableLibraryOption.ROWS_PER_SEGMENT, int(1e6))

See arcticdb/options.py for a description of the modifiable options.

🐛 Fixes

  • Bugfix 902: Cannot filter on nans and nones in string and float columns #1276
  • Bugfix: Empty type in #1227
  • Bugfix 1334: optimise version ref key access #1345
  • Fix version map cache invalidation policies in #1351
  • Fix empty column default type in #1378
  • Bugfix #1388: Correctly check whether we are in ec2 in #1415
  • Bugfix 1423: Raise a meaningful error message when trying to use QueryBuilder with sparse data in #1435
  • Bug Fix Windows: Remove lmdb files when delete_library is called in #1437
  • Bugfix 1256: reject parallel appends to unsorted data in #1442
  • Bugfix 1209: Consistently return metadata from write, append, update, write_metadata, and batch versions thereof in #1444
  • Bugfix/935/match pandas behviour when aggregating columns with nans in #1450
  • LZ4 decoding empty type: move the segment buffer forward by the compressed data size for empty types in #1463
  • Bugfix 1268: swap out xxhash for grouping in #1416
  • Bugfix 1260: allow broader range of numeric type promotion with dynamic schema #1426

💾 Storage Exception Normalization

We have made storage-related exceptions uniform across different backend storage platforms, despite the fact that the underlying behaviour varies.

  • #447 LMDB Exceptions Normalization in #1285
  • #447 Memory Storage Exception Normalization in #1297
  • Adds a MockS3Client which can simulate s3 failures in #1281
  • #447 S3 Storage Exceptions Normalization in #1304
  • Add a MockAzureClient which can simulate azure failures in #1331
  • #447 Azure Storage Exceptions Normalization in #1344
  • #447 Exception normalization for RocksDB in #1360
  • Refactor: Move mongo client errors handling into mongo_storage.cpp before normalization in #1383
  • #447 Add a MockMongoClient which can simulate mongo failures in #1395
  • #447 Mongo Exceptions Normalization in #1411
  • LMDB Exception Normalization with mock client in #1414
Uncategorized
  • Prevent writing empty types by default (gives compatibility with v1.6.2 readers) in #1440
  • Add a way to enable/disable silencing of errors when deleting a library in #1271
  • Feature flag to use WinInet client not WinHttp in #1284
  • Abstract an S3ClientWrapper out of details-inl.hpp in #1274
  • GitHub Workflows: Make can_merge run for all files to allow any docs changes to be mergeable. in #1292
  • Fix benchmarks in #1286
  • Reduce the hashes that we use to benchmark against in #1303
  • Refactor 1278: Column data dense forward iterator in #1301
  • Add contains to Arctic class, to support lib in arctic in #1309
  • Update BSL table for v4.3.0 in #1282
  • Utility to analyze the size of various key types in a library in #1291
  • Do not compile wheel build on EC2 in #1318
  • Docs: For top level imports use arcticdb.object instead of using full path to object in #1323
  • conda-build: Adaptations for folly in #1320
  • Align docstring with the behaviour in #1243
  • Added 142 new tests for empty/missing operations in #1319
  • Abstract AzureClientWrapper out of azure_storage.cpp in #1315
  • Stop using ec2 runners in the conda+linux workflow in #1302
  • Adds python tests which simulate s3 storage exceptions in #1330
  • S3 local delete failure raising meaningful error in #1329
  • Fix dynamic strings append to fixed strings issue in #1346
  • Minor improvement on analysis flow in #1290
  • Build time improvements in #1263
  • build: Disable compilers' extensions in #1335
  • maint: Fully specify fmt::format_to in #1333
  • Fixed pd_delete_replace + added single tests in #1342
  • Use type-deduced functor for all column iterating functions in #1347
  • Disable ec2 runners on PR in #1357
  • Use 14.39 toolset in #1359
  • Removed pytest dependency from arcticdb in #1350
  • build: Use C++20 in #1332
  • mark test_symbol_list_parallel_stress_with_delete as flaky in #1368
  • Docs: Increase CSS max-wdith and build docs from a branch in #1363
  • Print error msg in ExponentialBackoff exception in #1365
  • Disable missing key warnings when expected in #1379
  • Remove pin on civetweb in #1380
  • Multiple segments within the same block: storage and library refactoring in #1307
  • Not allowing snapshotting tombstoned versions in #1280
  • Clarify Intel/AMD build support in #1389
  • Fix debug formatting in #1397
  • maint: Replace robin_hood with unordered_dense in #1390
  • Test benchmarking improvements in #1326
  • maint: Remove dependency on some elements of folly in #1370
  • Allow different testing dependency version in pipeline in #1410
  • Add metadata extraction functions in library_tool in #1375
  • Add a global timeout for pytests in #1381
  • Set upper bound for supported protobuf version in #1421
  • 1 year and 1k stars readme banner in #1425
  • read_batch set include_deleted to false by default when reading a version in #1419
  • build: Update to fmt 10 in #1427
  • Make changes for prometheus metrics in #1418
  • maint: Replace use of folly getCurrentThreadId with STL in #1417
  • maint: Ignore the diff of #1263 in #1340
  • Roll back vcpkg version to fix failing abseil build in #1436
  • maint: Remove use of folly/portability/PThread.h in #1447
  • maint: Remove use of folly/system/ThreadName.h in #1446
  • Better error messaging around pickling in #1451
  • Update analysis_workflow.yml in #1455
  • Adding update, append and delete asv benchmarks in #1434
  • Support generators for metadata vectors again in #1456
  • Remove the brotli dep in #1458

The wheels are on PyPI. Below are for debugging:

Assets 3
Loading