Releases: Altinity/ClickHouse
23.8.16.43 Altinity Stable update
New in 23.8.16.43.altinitystable
compared to 23.8.16.42.altinitystable
Bug fixes
- Fixes Crash in Engine Merge if Row Policy does not have expression (ClickHouse#61971 by @ilejn via #534)
Build/Testing/Packaging Improvement
- Pin version of cxx rust package to prevent picking up latest version (#550)
Changes compared to upstream's v23.8.16.40-lts (e143a90)
Bug fixes
- Fixes partition pruning for extra columns in a set. (ClickHouse#55172 by @amosbird via #342)
- Fixed rare bug when we produced invalid read ranges for queries with
FINAL
. Resulted inCannot read out of marks range
exception. (ClickHouse#54934 by @nickitat via #343) - Fix incorrect free space accounting for least_used JBOD policy (ClickHouse#56030 by @azat via #344)
- Fix ALTER COLUMN with ALIAS that previously threw the
NO_SUCH_COLUMN_IN_TABLE
exception. (ClickHouse#57395 by @evillique via #345) - Disable
system.kafka_consumers
by default (due to possible live memory leak) (ClickHouse#57822 by @azat via #346) - Fix expiration in RoleCache (ClickHouse#67748 by @vitibar via #451)
- Fix crash with invalid access to AccessControl instance (ClickHouse#65051 by @davenger via #465)
- fix SIGSEGV due to profiler (ClickHouse#63865 by @azat via #466)
- Projection analysis now comes after partition pruning, resulting in better plan efficiency and better statistic log (ClickHouse#56502 by @amosbird via #391)
- Fix broken partition key analysis when doing projection optimization with
force_index_by_date = 1
(ClickHouse#58638 by @amosbird via #391)*new
Performance improvements
- Fixed filtering by
IN(...)
condition forMerge
table engine. (ClickHouse#54905 by @nickitat via #341)
Improvement
- Support S3 access through AWS Private Link Interface endpoints. (ClickHouse#62208 by @arthurpassos via #389)
- Fixed accounting of memory allocated before attaching thread to a query or a user (ClickHouse#56089 by @nickitat via #388)
- Engine
Merge
filters the records according to the row policies of the underlying tables (ClickHouse#50209 by @ilejn via #446)
Build/Testing/Packaging Improvement
- Make builds possible on Altinity's infrastructure
clickhouse-regression
test suite (#338 by @MyroTk)- Changed "version is not official" log message on failure (#369 by @Enmk)
- Fix kerberized_hadoop docker image build issue (#394 by @ilejn)
- Reuse unchanged images #455 by @MyroTk
- Pinned version of pip packet requests==2.31.0 in attempt to fix: 'Error while fetching server API version: Not supported URL scheme http+docker' (#409 by @Enmk)
- Aarch64 builds with better introspection (
-no-pie
) (#396 by @Enmk) - Fixed the docker tag for docker server test image (#463 by @MyroTk)
- Add
SignAarch64
job to match SignRelease (#462 by @MyroTk) - Fix
test_distributed_directory_monitor_split_batch_on_failure_OFF
(#464 by @Enmk) - Fix
kererberized_hadoop
tests (#468, #459 by @ilejn) - Updated reference to clickhouse-regression suite (#467 by @MyroTk)
- Scheduled run + CI/CD fixups (#402 by @MyroTk)
- Fixed some dependencies versions (#507)
- Snapshot runners and other infrastructure improvements (#506)
- Updated package metadata (#505)
Build report
Packages
Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/ as either .deb
, .rpm
, or .tgz
Docker images
Available for both AMD64 and Aarch64: altinity/clickhouse-server/23.8.16.43.altinitystable
24.3.12.76 Altinity Stable
Changes compared to upstream's v24.3.12.75-lts
Build/Testing/Packaging Improvement
- Enable building and testing on Altinity infrastructure (#413)
Bug fix
- 🆕 Support executing function during assignment of parameterized view value (ClickHouse#63502 by @SmitaRKulkarni via #444)
- 🆕 Fix serialization of parameterized view parameters (ClickHouse#67654 by @shiyer7474 via #502)
- 🆕 fix intersect parts when restart after drop range (ClickHouse#63202 by @hanfei1991 via #495)
- Fix the crash loop when restoring from backup is blocked by creating an MV with a definer that hasn't been restored yet. (ClickHouse#64595 by @pufit via #412)
- Fix moving partition to itself ClickHouse#62459 (ClickHouse#62524 by @helifu via #411)
- Fix detection of number of CPUs in containers. In the case when the 'root' cgroup was used (i.e. name of cgroup was empty, which is common for containers ) ClickHouse was ignoring the CPU limits set for the container. (ClickHouse#66237 by @filimonov via #420)
Improvement
- 🆕 Support aliases in parametrized view function (ClickHouse#65190 via @kssenii #500)
- 🆕 Ignore
allow_suspicious_primary_key
onATTACH
and verify onALTER
(ClickHouse#64202 by @azat via #494) - 🆕 Thread pool metrics (ClickHouse#68674 by @filimonov via #503)
- 🆕 more advanced SSL options for Keeper (ClickHouse#69582 by @antonio2368 via #496)
- 🆕 Add fault injection for "Cannot allocate thread" (ClickHouse#62266 by @tavplubix via #508)
- 🆕 Support S3 access through AWS Private Link Interface endpoints. (ClickHouse#62208 by @arthurpassos via #498)
- 🆕 Set
input_format_parquet_filter_push_down
default value tofalse
#511
Build/Testing/Packaging Improvement
- 🆕
ubuntu:22.04
as a base image forclickhouse-server
docker image #497 - 🆕
alpine:3.20.3
as clickhouse-keeper base image to (lowest number of CVEs) #517
🆕 — new in 24.3.12.76.altinitystable
compared to 24.3.5.47.altinitystable
Build report
Packages
Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/ as either .deb
, .rpm
, or .tgz
Docker images
Available for both AMD64 and Aarch64: altinity/clickhouse-server/24.3.12.76.altinitystable
23.8.16.42 Altinity Stable update
New in 23.8.16.42.altinitystable
compared to 23.8.16.41.altinitystable
Bug fixes
- Projection analysis now comes after partition pruning, resulting in better plan efficiency and better statistic log (ClickHouse#56502 by @amosbird via #391)
- Fix broken partition key analysis when doing projection optimization with
force_index_by_date = 1
(ClickHouse#58638 by @amosbird via #391)*new
Other changes compared to upstream's v23.8.16.40-lts (e143a90)
Bug fixes
- Fixes partition pruning for extra columns in a set. (ClickHouse#55172 by @amosbird via #342)
- Fixed rare bug when we produced invalid read ranges for queries with
FINAL
. Resulted inCannot read out of marks range
exception. (ClickHouse#54934 by @nickitat via #343) - Fix incorrect free space accounting for least_used JBOD policy (ClickHouse#56030 by @azat via #344)
- Fix ALTER COLUMN with ALIAS that previously threw the
NO_SUCH_COLUMN_IN_TABLE
exception. (ClickHouse#57395 by @evillique via #345) - Disable
system.kafka_consumers
by default (due to possible live memory leak) (ClickHouse#57822 by @azat via #346) - Fix expiration in RoleCache (ClickHouse#67748 by @vitibar via #451)
- Fix crash with invalid access to AccessControl instance (ClickHouse#65051 by @davenger via #465)
- fix SIGSEGV due to profiler (ClickHouse#63865 by @azat via #466)
Performance improvements
- Fixed filtering by
IN(...)
condition forMerge
table engine. (ClickHouse#54905 by @nickitat via #341)
Improvement
- Support S3 access through AWS Private Link Interface endpoints. (ClickHouse#62208 by @arthurpassos via #389)
- Fixed accounting of memory allocated before attaching thread to a query or a user (ClickHouse#56089 by @nickitat via #388)
- Engine
Merge
filters the records according to the row policies of the underlying tables (ClickHouse#50209 by @ilejn via #446)
Build/Testing/Packaging Improvement
- Make builds possible on Altinity's infrastructure
clickhouse-regression
test suite (#338 by @MyroTk)- Changed "version is not official" log message on failure (#369 by @Enmk)
- Fix kerberized_hadoop docker image build issue (#394 by @ilejn)
- Reuse unchanged images #455 by @MyroTk
- Pinned version of pip packet requests==2.31.0 in attempt to fix: 'Error while fetching server API version: Not supported URL scheme http+docker' (#409 by @Enmk)
- Aarch64 builds with better introspection (
-no-pie
) (#396 by @Enmk) - Fixed the docker tag for docker server test image (#463 by @MyroTk)
- Add
SignAarch64
job to match SignRelease (#462 by @MyroTk) - Fix
test_distributed_directory_monitor_split_batch_on_failure_OFF
(#464 by @Enmk) - Fix
kererberized_hadoop
tests (#468, #459 by @ilejn) - Updated reference to clickhouse-regression suite (#467 by @MyroTk)
- Scheduled run + CI/CD fixups (#402 by @MyroTk)
- Fixed some dependencies versions (#507)
- Snapshot runners and other infrastructure improvements (#506)
- Updated package metadata (#505)
Build report
Packages
Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/ as either .deb
, .rpm
, or .tgz
Docker images
Available for both AMD64 and Aarch64: altinity/clickhouse-server/23.8.16.42.altinitystable
23.8.16 Altinity Stable
Changes compared to upstream's v23.8.16.40-lts (e143a90)
Bug fixes
- Fixes partition pruning for extra columns in a set. (ClickHouse#55172 by @amosbird via #342)
- Fixed rare bug when we produced invalid read ranges for queries with
FINAL
. Resulted inCannot read out of marks range
exception. (ClickHouse#54934 by @nickitat via #343) - Fix incorrect free space accounting for least_used JBOD policy (ClickHouse#56030 by @azat via #344)
- Fix ALTER COLUMN with ALIAS that previously threw the
NO_SUCH_COLUMN_IN_TABLE
exception. (ClickHouse#57395 by @evillique via #345) - Disable
system.kafka_consumers
by default (due to possible live memory leak) (ClickHouse#57822 by @azat via #346) - Fix expiration in RoleCache (ClickHouse#67748 by @vitibar via #451)
- Fix crash with invalid access to AccessControl instance (ClickHouse#65051 by @davenger via #465)
- fix SIGSEGV due to profiler (ClickHouse#63865 by @azat via #466)
Performance improvements
- Fixed filtering by
IN(...)
condition forMerge
table engine. (ClickHouse#54905 by @nickitat via #341)
Improvement
- Support S3 access through AWS Private Link Interface endpoints. (ClickHouse#62208 by @arthurpassos via #389)
- Fixed accounting of memory allocated before attaching thread to a query or a user (ClickHouse#56089 by @nickitat via #388)
- Engine
Merge
filters the records according to the row policies of the underlying tables (ClickHouse#50209 by @ilejn via #446)
Build/Testing/Packaging Improvement
- Make builds possible on Altinity's infrastructure
clickhouse-regression
test suite (#338 by @MyroTk)- Changed "version is not official" log message on failure (#369 by @Enmk)
- Fix kerberized_hadoop docker image build issue (#394 by @ilejn)
- Reuse unchanged images #455 by @MyroTk
- Pinned version of pip packet requests==2.31.0 in attempt to fix: 'Error while fetching server API version: Not supported URL scheme http+docker' (#409 by @Enmk)
- Aarch64 builds with better introspection (
-no-pie
) (#396 by @Enmk) - Fixed the docker tag for docker server test image (#463 by @MyroTk)
- Add
SignAarch64
job to match SignRelease (#462 by @MyroTk) - Fix
test_distributed_directory_monitor_split_batch_on_failure_OFF
(#464 by @Enmk) - Fix
kererberized_hadoop
tests (#468, #459 by @ilejn) - Updated reference to clickhouse-regression suite (#467 by @MyroTk)
- Scheduled run + CI/CD fixups (#402 by @MyroTk)
upstream ClickHouse changes between v23.8.16.40-lts (e143a90) and v23.8.11.28-lts (31879d2)
Critical Bug Fix (crash, LOGICAL_ERROR, data loss, RBAC))
- Fix crash with
UniqInjectiveFunctionsEliminationPass
anduniqCombined
. (#65188 by Raúl Marín via #65281) - Fix a bug in ClickHouse Keeper that causes digest mismatch during closing session. (#65198 by Aleksei Filatov via #65368)
- Fix crash in maxIntersections. (#65689 by Raúl Marín via #65743)
Bug Fix (user-visible misbehavior in an official stable release))
- The query cache now denies access to entries when the user is re-created or assumes another role. This improves prevents attacks where 1. an user with the same name as a dropped user may access the old user's cache entries or 2. a user with a different role may access cache entries of a role with a different row policy. (#58611 by Robert Schulze via #61439)
- Fix string search with constant start position which previously could lead to memory corruption. (#61547 by Antonio Andelic via #61572)
- Fix crash in
multiSearchAllPositionsCaseInsensitiveUTF8
when specifying incorrect UTF-8 sequence. Example: #61714. (#61749 by pufit via #61854) - Fixed a bug in zero-copy replication (an experimental feature) that could cause
The specified key does not exist
errors and data loss after REPLACE/MOVE PARTITION. A similar issue might happen with TTL-moves between disks. (#54193 by Alexander Tokmakov via #62898) - Fix the ATTACH query with the ON CLUSTER clause when the database does not exist on the initiator node. Closes #55009. (#61365 by Nikolay Degterinsky via #61964)
- Fix data race between
MOVE PARTITION
query and merges resulting in intersecting parts. (#61610 by János Benjamin Antal via #62527) - Fix skipping escape sequence parsing errors during JSON data parsing while using
input_format_allow_errors_num/ratio
settings. (#61883 by Kruglov Pavel via #62238) - Fix segmentation fault when using Hive table engine. Reference #62154, #62560. (#62578 by Nikolay Degterinsky via #62673)
- Setting server_name might help with recently reported SSL handshake error when connecting to MongoDB Atlas:
Poco::Exception. Code: 1000, e.code() = 0, SSL Exception: error:10000438:SSL routines:OPENSSL_internal:TLSV1_ALERT_INTERNAL_ERROR
. (#63122 by Alexander Gololobov via #63172) - The wire protocol version check for MongoDB used to try accessing "config" database, but this can fail if the user doesn't have permissions for it. The fix is to use the database name provided by user. (#63126 by Alexander Gololobov via #63164)
- Fixes #59989: runs init scripts when force-enabled or when no database exists, rather than the inverse. (#59991 by jktng via #64422)
- Fix "Invalid storage definition in metadata file" for parameterized views. (#60708 by Azat Khuzhin via #64016)
- Fix the issue where the function
addDays
(and similar functions) reports an error when the first parameter isDateTime64
. (#61561 by Shuai li via #63456) - Fix crash with untuple and unresolved lambda. (#63131 by Raúl Marín via #63289)
- Fix
X-ClickHouse-Timezone
header returning wrong timezone when usingsession_timezone
as query level setting. (#63377 by Andrey Zvonov via #63512) query_plan_remove_redundant_distinct
can break queries with WINDOW FUNCTIONS (withallow_experimental_analyzer
is on). Fixes #62820. (#63776 by Igor Nikonov via #63902)- Deserialize untrusted binary inputs in a safer way. (#64024 by Robert Schulze via #64104)
- Prevent LOGICAL_ERROR on CREATE TABLE as MaterializedView. (#64174 by Raúl Marín via #64265)
- Fixed memory possible incorrect memory tracking in several kinds of queries: queries that read any data from S3, queries via http protocol, asynchronous inserts. (#64844 by Anton Popov via #64867)
- Fix possible abort on uncaught exception in ~WriteBufferFromFileDescriptor in StatusFile. (#64206 by Kruglov Pavel via [ClickHouse#65351](https://github.com/ClickHouse/ClickHouse/issue...
24.3.5 Altinity FIPS
Changes compared to 24.3.5.47.altinitystable
Improvement
- Building BoringSSL ver
fips-20210429
(853ca1ea1168dff08011e5d42d94609cc0ca2e27
) according to FIPS-140-2 Security Policy 4407, based on build scripts from Golang versiongo1.22.5
- Added
FIPS_CLICKHOUSE
tosystem.build_options
- Modified ClickHosue keeper to use full range of openSSL options (same as ClickHouse does) for Raft connections
- using
ubuntu:22.04
as a base forclickhouse-server
docker images instead ofubuntu:20.04
Build/Testing/Packaging Improvement
- CI/CD for FIPS mode
- other fixes for CI/CD
Full Changelog: v24.3.5.47.altinitystable...v24.3.5.48.altinityfips
Build report
Packages
Available for AMD64 from https://builds.altinity.cloud/ as either .deb
, .rpm
, or .tgz
Docker images
Available for AMD64: altinity/clickhouse-server/24.3.5.48.altinityfips
24.3.5 Altinity Stable
Changes compared to upstream's v24.3.5.46
Build/Testing/Packaging Improvement
- Enable building and testing on Altinity infrastructure (#413)
Bug fix
- Fix the crash loop when restoring from backup is blocked by creating an MV with a definer that hasn't been restored yet. (ClickHouse#64595 by @pufit via #412)
- Fix moving partition to itself ClickHouse#62459 (ClickHouse#62524 by @helifu via #411)
- Fix detection of number of CPUs in containers. In the case when the 'root' cgroup was used (i.e. name of cgroup was empty, which is common for containers ) ClickHouse was ignoring the CPU limits set for the container. (ClickHouse#66237 by @filimonov via #420)
Build report
Packages
Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/ as either .deb
, .rpm
, or .tgz
Docker images
Available for both AMD64 and Aarch64: altinity/clickhouse-server/24.3.5.47.altinitystable
23.8.11 Altinity Stable
Changes compared to upstream's v23.8.11.28-lts
Bug fixes
- Fixes partition pruning for extra columns in a set. (ClickHouse#55172 by @amosbird via #342)
- Fixed rare bug when we produced invalid read ranges for queries with
FINAL
. Resulted inCannot read out of marks range
exception. (ClickHouse#54934 by @nickitat via #343) - Fix incorrect free space accounting for least_used JBOD policy (ClickHouse#56030 by @azat via #344)
- Fix ALTER COLUMN with ALIAS that previously threw the
NO_SUCH_COLUMN_IN_TABLE
exception. (ClickHouse#57395 by @evillique via #345) - Disable
system.kafka_consumers
by default (due to possible live memory leak) (ClickHouse#57822 by @azat via #346)
Performance improvements
- Fixed filtering by
IN(...)
condition forMerge
table engine. (ClickHouse#54905 by @nickitat via #341)
Build/Testing/Packaging Improvement
- Make builds possible on Altinity's infrastructure
clickhouse-regression
test suite (#338 by @MyroTk)- Changed "version is not official" log message on failure (#369 by @Enmk)
- Fix kerberized_hadoop docker image build issue (#394 by @ilejn)
Improvement
- Support S3 access through AWS Private Link Interface endpoints. (ClickHouse#62208 by @arthurpassos via #389)
- Fixed accounting of memory allocated before attaching thread to a query or a user (ClickHouse#56089 by @nickitat via #388)
upstream ClickHouse changes between v23.8.11.28-lts (31879d2) and v23.8.8.20-lts (5e012a0)
Improvement
- Output valid JSON/XML on excetpion during HTTP query execution. Add setting
http_write_exception_in_output_format
to enable/disable this behaviour (enabled by default). #52853 - Fix transfer query to MySQL compatible query. Fixes #57253. Fixes #52654. Fixes #56729. #56456
- Fetching a part waits when that part is fully committed on remote replica. It is better not send part in PreActive state. In case of zero copy this is mandatory restriction. #56808
- Handle sigabrt case when getting PostgreSQl table structure with empty array. #57618
- Add
SYSTEM JEMALLOC PURGE
for purging unused jemalloc pages,SYSTEM JEMALLOC [ ENABLE | DISABLE | FLUSH ] PROFILE
for controlling jemalloc profile if the profiler is enabled. Add jemalloc-related 4LW command in Keeper:jmst
for dumping jemalloc stats,jmfp
,jmep
,jmdp
for controlling jemalloc profile if the profiler is enabled. #58665 - Copy S3 file GCP fallback to buffer copy in case GCP returned
Internal Error
withGATEWAY_TIMEOUT
HTTP error code. #60164 - Update tzdata to 2024a. #60768
Build/Testing/Packaging Improvement
- Fix issue caught in docker-library/official-images#15846. #57571
- If you want to run initdb scripts every time when ClickHouse container is starting you shoud initialize environment varible CLICKHOUSE_ALWAYS_RUN_INITDB_SCRIPTS. #59808
Bug Fix (user-visible misbehavior in an official stable release)
- Flatten only true Nested type if flatten_nested=1, not all Array(Tuple) #56132
- Fix ALTER COLUMN with ALIAS #56493
- Prevent incompatible ALTER of projection columns #56948
- Fix segfault after ALTER UPDATE with Nullable MATERIALIZED column #57147
- Fix incorrect JOIN plan optimization with partially materialized normal projection #57196
- Fix
ReadonlyReplica
metric for all cases #57267 - Fix working with read buffers in StreamingFormatExecutor #57438
- bugfix: correctly parse SYSTEM STOP LISTEN TCP SECURE #57483
- Ignore ON CLUSTER clause in grant/revoke queries for management of replicated access entities. #57538
- Disable system.kafka_consumers by default (due to possible live memory leak) #57822
- Fix invalid memory access in BLAKE3 (Rust) #57876
- Normalize function names in CREATE INDEX #57906
- Fix invalid preprocessing on Keeper #58069
- Fix Integer overflow in Poco::UTF32Encoding #58073
- Remove parallel parsing for JSONCompactEachRow #58181
- Fix parallel parsing for JSONCompactEachRow #58250
- Background merges correctly use temporary data storage in the cache #57275
- MergeTree mutations reuse source part index granularity #57352
- Fix double destroy call on exception throw in addBatchLookupTable8 #58745
- Fix JSONExtract function for LowCardinality(Nullable) columns #58808
- Fix: LIMIT BY and LIMIT in distributed query #59153
- Fix translate() with FixedString input #59356
- Fix error "Read beyond last offset" for AsynchronousBoundedReadBuffer #59630
- Fix query start time on non initial queries #59662
- Fix leftPad / rightPad function with FixedString input #59739
- rabbitmq: fix having neither acked nor nacked messages #59775
- Fix cosineDistance crash with Nullable #60150
- Fix buffer overflow in CompressionCodecMultiple #60731
- Remove nonsense from SQL/JSON #60738
- Fix crash in arrayEnumerateRanked #60764
- Fix crash when using input() in INSERT SELECT JOIN #60765
- Remove recursion when reading from S3 #60849
NO CL ENTRY
NOT FOR CHANGELOG / INSIGNIFICANT
- Pin alpine version of integration tests helper container #57669
- Remove heavy rust stable toolchain #57905
- Fix docker image for integration tests (fixes CI) #57952
- Fix rare race in external sort/aggregation with temporary data in cache #58013
- Fix possible race in ManyAggregatedData dtor. #58624
- Fix 02720_row_policy_column_with_dots #59453
- Pin python dependencies in stateless tests #59663
- Make ZooKeeper actually sequentialy consistent #59735
- Remove broken test while we fix it #60547
- Cancel PipelineExecutor properly in case of exception in spawnThreads #57104
- Detect io_uring in tests #60373
- Cancel PipelineExecutor properly in case of exception in spawnThreads #60499
Packages
Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/ as either .deb
, .rpm
, or .tgz
Docker images
Available for both AMD64 and Aarch64: altinity/clickhouse-server/23.8.11.29.altinitystable
23.3.19 Altinity FIPS
Changes compared to 23.3.19.33.altinitystable
Improvement
- Building BoringSSL (
ae223d6138807a13006342edfeef32e813246b39
) in FIPS mode, based on procedures of Go 1.19 (#235 by @Enmk) - FIPS mode for Keeper (#255, #274 by @Enmk)
Build/Testing/Packaging Improvement
Full Changelog: v23.3.19.33.altinitystable...v23.3.19.34.altinityfips
23.3.19 Altinity Stable
Changes compared to upstream's v23.3.19.32-lts
Bug fixes
- Fix key condition on duplicate primary keys (ClickHouse#48838 by @amosbird via #275 )
- Fix excessive memory usage for FINAL (ClickHouse#50429 by @ azat via #307)
- Fixed DNS-related crash (ClickHouse#52634 by @arthurpassos via #295)
- Fix parquet array inconsistent offsets (ClickHouse#54370 by @arthurpassos via #303)
- Fixed rare bug when we produced invalid read ranges for queries with FINAL. Resulted in Cannot read out of marks range exception (ClickHouse#54934 by @nickitat via #313)
- Fixed reading from sparse columns after changing setting
ratio_of_defaults_for_sparse_serialization
back to1.0
and restarting the server. (ClickHouse#49660 by @CurtizJ via #314) - Fix incorrect free space accounting for least_used JBOD policy (ClickHouse#56030 by @azat via #365)
- Fixes race condition, prevents application crash (ClickHouse#52623 by @hanfei1991 via #366)
- Fixed leaking loggers in some cases, like creating and dropping tables (ClickHouse#58831, ClickHouse#59273 by @kitaisreal via #356)
- Fix deadlock in LDAP assigned role update for non-existing ClickHouse roles (ClickHouse#55119 by @jmaicher, ClickHouse#56544 by @jmaicher via #364)
Performance improvement
- Do not build sets for skip indexes if they are not used (ClickHouse#48299 by @CurtizJ via #276)
Improvement
- Disabled unsafe "compile_expressions" setting by default (ClickHouse#51368 by @alexey-milovidov via #306)
- Row policy for database (ClickHouse#47640 by @ilejn via #300)
Build/Testing/Packaging Improvement
- Bring back DNS tests (ClickHouse#53286 - @arthurpassos via #296)
- Made builds and tests possible in Altinity's infrastructure
- Support of Aarch64 builds (#311)
- building Aarch64 binaries
- multi-arch docker containers (Amd64, Aarch64)
- running stateles, stateful tests against Aarch64
- running clickhouse-regression tests against Aarch64
upstream ClickHouse changes between v23.3.19.32-lts
(c4d4ca8ec02) and v23.3.13.6-lts
(25635e27551)
Improvement
- Fix transfer query to MySQL compatible query. (ClickHouse#56456 by flynn via ClickHouse#58489).
- There was a potential vulnerability in previous ClickHouse versions: if a user has connected and unsuccessfully tried to authenticate with the "interserver secret" method, the server didn't terminate the connection immediately but continued to receive and ignore the leftover packets from the client. While these packets are ignored, they are still parsed, and if they use a compression method with another known vulnerability, it will lead to exploitation of it without authentication. This issue was found with [ClickHouse Bug Bounty Program] by https://twitter.com/malacupa. (ClickHouse#56794 by Alexey Milovidov via ClickHouse#56928).
- Handle sigabrt case when getting PostgreSQl table structure with empty array. (ClickHouse#57618 by Mikhail Kot via ClickHouse#57653).
Build/Testing/Packaging Improvement
- Fix strange additional QEMU logging after ClickHouse#47151, see https://s3.amazonaws.com/clickhouse-test-reports/50078/a4743996ee4f3583884d07bcd6501df0cfdaa346/stateless_tests__release__databasereplicated__[3_4].html. (ClickHouse#50442 by Mikhail f. Shiryaev via ClickHouse#54312).
- If the database is already initialized, it doesn't need to be initialized again upon subsequent launches. This can potentially fix the issue of infinite container restarts when the database fails to load within 1000 attempts (relevant for very large databases and multi-node setups). (ClickHouse#50724 by Alexander Nikolaev via ClickHouse#55671).
- Resource with source code including submodules is built in Darwin special build task. It may be used to build ClickHouse without checkouting submodules. (ClickHouse#51435 by Ilya Yatsishin via ClickHouse#55391).
- Enrich
changed_images.json
with the latest tag from master for images that are not changed in the pull request. (ClickHouse#54369 by János Benjamin Antal via ClickHouse#54703). - We build and upload them for every push, which isn't worth it. (ClickHouse#54675 by Mikhail f. Shiryaev via ClickHouse#54679).
- Fix integration check python script to use gh api url - Add Readme for CI tests. (ClickHouse#55716 by Max K. via ClickHouse#55734).
- Check sha512 for tgz; use a proper repository for keeper; write only filenames to TGZ.sha512 files for tarball packages. Prerequisite for ClickHouse#31473. (ClickHouse#55717 by Mikhail f. Shiryaev via ClickHouse#55829).
- Do not fetch changed submodules in the builder container. (ClickHouse#56689 by Mikhail f. Shiryaev via ClickHouse#56731).
- There was an attempt to have the proper listing in ClickHouse#44311, but the fix itself was in the wrong place, so it's still broken. See an example. (ClickHouse#56989 by Mikhail f. Shiryaev via ClickHouse#57019).
- Fix issue caught in docker-library/official-images#15846. (ClickHouse#57571 by Mikhail f. Shiryaev via ClickHouse#57580).
- Pin rust version to fix GLIBC compatibility check. (ClickHouse#55788 by Mikhail f. Shiryaev).
Bug Fix (user-visible misbehavior in an official stable release)
- Fix parameterized view with cte and multiple usage (ClickHouse#52328 by SmitaRKulkarni).
- Check recursion depth in OptimizedRegularExpression (ClickHouse#52451 by Alexey Milovidov).
- Fix lightweight delete after drop of projection (ClickHouse#52517 by Anton Popov).
- Fix sorting of sparse columns with large limit (ClickHouse#52827 by Anton Popov).
- Fix adding sub-second intervals to DateTime (ClickHouse#53309 by Michael Kolupaev).
- Fix: allow IPv6 for bloom filter (ClickHouse#54200 by Yakov Olkhovskiy).
- Check for overflow before addition in
analysisOfVariance
function (ClickHouse#54385 by Antonio Andelic). - Fix aggregate projections with normalized states (ClickHouse#54480 by Amos Bird).
- Fix possible parsing error in WithNames formats with disabled input_format_with_names_use_header (ClickHouse#54513 by Kruglov Pavel).
- Fix: avoid using regex match, possibly containing alternation, as a key condition. (ClickHouse#54696 by Yakov Olkhovskiy).
- Fix "Invalid number of rows in Chunk" in MaterializedPostgreSQL (ClickHouse#54844 by Kseniia Sumarokova).
- Prevent attaching parts from tables with different projections or indices (ClickHouse#55062 by János Benjamin Antal).
- Fix bug with inability to drop detached partition in replicated merge tree on top of S3 without zero copy (ClickHouse#55309 by alesapin).
- Fix trash optimization (up to a certain extent) (ClickHouse#55353 by Alexey Milovidov).
- Fix parsing of arrays in cast operator (ClickHouse#55417 by Anton Popov).
- Fix crash in QueryNormalizer with cyclic aliases (ClickHouse#55602 by vdimir).
- Fix window functions in case of sparse columns. (ClickHouse#55895 by János Benjamin Antal).
- Fix segfault during Kerberos initialization (ClickHouse#56401 by Nikolay Degterinsky).
- Fix buffer overflow in T64 (ClickHouse#56434 by Alexey Milovidov).
- Fix crash in FPC codec (ClickHouse#56795 by Alexey Milovidov).
- Prevent incompatible ALTER of projection columns (ClickHouse#56948 by Amos Bird).
- Fix buffer overflow in Gorilla codec (ClickHouse#57107 by Nikolay Degterinsky).
- Close interserver connection on any exception before authentication (ClickHouse#57142 by Antonio Andelic).
- Fix segfault after ALTER UPDATE with Nullable MATERIALIZED column (ClickHouse#57147 by Nikolay Degterinsky).
- Fix incorrect JOIN plan optimization with partially materialized normal projection (ClickHouse#57196 by Amos Bird).
- MergeTree mutations reuse source part index granularity (ClickHouse#57352 by Maksim Kita).
- Fix invalid memory access in BLAKE3 (Rust) (ClickHouse#57876 by Raúl Marín).
- Normalize function names in CREATE INDEX (ClickHouse#57906 by Alexander Tokmakov).
- Fix invalid preprocessing on Keeper (ClickHouse#58069 by Antonio Andelic).
- Fix Integer overflow in Poco::UTF32Encoding (ClickHouse#58073 by Andrey Fedotov).
- Remove parallel parsing for JSONCompactEachRow (ClickHouse#58181 by Alexey Milovidov).
- Fix incorrect createColumn call on join clause (ClickHouse#48998 by Yi Sheng).
NOT FOR CHANGELOG / INSIGNIFICANT
- Test libunwind changes. (ClickHouse#51436 by Nikolai Kochetov).
- Refactor CI_CONFIG (ClickHouse#52948 by Mikhail f. Shiryaev).
- Use pathlib.Path in S3Helper, rewrite build reports, improve small things (ClickHouse#54010 by Mikhail f. Shiryaev).
- Fixed parameterized_view test after backporting a fix 23.3 (ClickHouse#54401 by SmitaRKulkarni).
- Update automated commit status comment (ClickHouse#54441 by vdimir).
- S3 artifacts (ClickHouse#54504 by Mikhail f. Shiryaev).
- Fix C...
23.8.8 Altinity Stable
Changes compared to upstream's v23.8.8.20-lts
Bug fixes
- Fixes partition pruning for extra columns in a set. (ClickHouse#55172 by @amosbird via #342)
- Fixed rare bug when we produced invalid read ranges for queries with
FINAL
. Resulted inCannot read out of marks range
exception. (ClickHouse#54934 by @nickitat via #343) - Fix incorrect free space accounting for least_used JBOD policy (ClickHouse#56030 by @azat via #344)
- Fix ALTER COLUMN with ALIAS that previously threw the
NO_SUCH_COLUMN_IN_TABLE
exception. (ClickHouse#57395 by @evillique via #345) - Disable
system.kafka_consumers
by default (due to possible live memory leak) (ClickHouse#57822 by @azat via #346)
Performance improvements
- Fixed filtering by
IN(...)
condition forMerge
table engine. (ClickHouse#54905 by @nickitat via #341)
Build/Testing/Packaging Improvement
- Make builds possible on Altinity's infrastructure
clickhouse-regression
test suite (#338 by @MyroTk)
Packages
Available for both AMD64 and Aarch64 from https://builds.altinity.cloud/ as either .deb
, .rpm
, or .tgz
Docker images
Available for both AMD64 and Aarch64: altinity/clickhouse-server/23.8.8.21.altinitystable