forked from redis/redis
-
Notifications
You must be signed in to change notification settings - Fork 0
/
00-RELEASENOTES
982 lines (759 loc) · 42.2 KB
/
00-RELEASENOTES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
Redis 7.0 release notes
=======================
--------------------------------------------------------------------------------
Upgrade urgency levels:
LOW: No need to upgrade unless there are new features you want to use.
MODERATE: Program an upgrade of the server, but it's not urgent.
HIGH: There is a critical bug that may affect a subset of users. Upgrade!
CRITICAL: There is a critical bug affecting MOST USERS. Upgrade ASAP.
SECURITY: There are security fixes in the release.
--------------------------------------------------------------------------------
================================================================================
Redis 7.0.10 Released Mon Mar 20 16:00:00 IST 2023
================================================================================
Upgrade urgency: SECURITY, contains fixes to security issues.
Security Fixes:
* (CVE-2023-28425) Specially crafted MSETNX command can lead to assertion and denial-of-service
Bug Fixes
=========
* Large blocks of replica client output buffer may lead to psync loops and unnecessary memory usage (#11666)
* Fix CLIENT REPLY OFF|SKIP to not silence push notifications (#11875)
* Trim excessive memory usage in stream nodes when exceeding `stream-node-max-bytes` (#11885)
* Fix module RM_Call commands failing with OOM when maxmemory is changed to zero (#11319)
================================================================================
Redis 7.0.9 Released Tue Feb 28 12:00:00 IST 2023
================================================================================
Upgrade urgency: SECURITY, contains fixes to security issues.
Security Fixes:
* (CVE-2023-25155) Specially crafted SRANDMEMBER, ZRANDMEMBER, and HRANDFIELD
commands can trigger an integer overflow, resulting in a runtime assertion
and termination of the Redis server process.
* (CVE-2022-36021) String matching commands (like SCAN or KEYS) with a specially
crafted pattern to trigger a denial-of-service attack on Redis, causing it to
hang and consume 100% CPU time.
Bug Fixes
=========
* Fix a crash when reaching the maximum invalidations limit of client-side tracking (#11814)
* Fix a crash when SPUBLISH is used after passing the cluster-link-sendbuf-limit (#11752)
* Fix possible memory corruption in FLUSHALL when a client watches more than one key (#11854)
* Fix cluster inbound link keepalive time (#11785)
* Flush propagation list in active-expire of writable replicas to fix an assertion (#11615)
* Avoid propagating DEL of lazy expire from SCAN and RANDOMKEY as MULTI-EXEC (#11788)
Performance and resource utilization improvements
=================================================
* Avoid realloc to reduce size of strings when it is unneeded (#11766)
* Improve CLUSTER SLOTS reply efficiency for non-continuous slots (#11745)
================================================================================
Redis 7.0.8 Released Mon Jan 16 12:00:00 IDT 2023
================================================================================
Upgrade urgency: SECURITY, contains fixes to security issues.
Security Fixes:
* (CVE-2022-35977) Integer overflow in the Redis SETRANGE and SORT/SORT_RO
commands can drive Redis to OOM panic
* (CVE-2023-22458) Integer overflow in the Redis HRANDFIELD and ZRANDMEMBER
commands can lead to denial-of-service
Bug Fixes
=========
* Avoid possible hang when client issues long KEYS, SRANDMEMBER, HRANDFIELD,
and ZRANDMEMBER commands and gets disconnected by client output buffer limit (#11676)
* Make sure that fork child doesn't do incremental rehashing (#11692)
* Fix a bug where blocking commands with a sub-second timeout would block forever (#11688)
* Fix sentinel issue if replica changes IP (#11590)
================================================================================
Redis 7.0.7 Released Fri Dec 16 12:00:00 IST 2022
================================================================================
Upgrade urgency: MODERATE, Contains fix for a regression in Geo commands.
Bug Fixes
=========
* Fix regression from Redis 7.0.6 in distance replies of Geo commands (#11631)
================================================================================
Redis 7.0.6 Released Mon Dec 12 12:00:00 IST 2022
================================================================================
Upgrade urgency: MODERATE, Contains fixes for a few non-critical or unlikely bugs,
and some dramatic optimizations to Geo, EVAL, and Sorted sets commands.
Potentially Breaking Bug Fixes for new Redis 7.0 features
=======================================================
* RM_ResetDataset module API should not clear the functions (#11268)
* RM_Call module API used with the "C" flag to run scripts, would now cause
the commands in the script to check ACL with the designated user (#10966)
Performance and resource utilization improvements
=================================================
* Geo commands speedups (#11535, #11522, #11552, #11579)
* Fix EVAL command performance regression from Redis 7.0 (#11521, #11541)
* Reduce EXPIRE commands performance regression from Redis 7.0 (#11602)
* Optimize commands returning double values, mainly affecting zset commands (#11093)
* Optimize Lua parsing of some command responses (#11556)
* Optimize client memory usage tracking operation while client eviction is disabled (#11348)
Platform / toolchain support related improvements
=================================================
* Fix compilation on Solaris (#11327)
Module API changes
==================
* RM_SetContextUser, RM_SetModuleUserACLString, RM_GetModuleUserACLString (#10966)
* Fix crash in CLIENT_CHANGE event, when the selected database is not 0 (#11500)
Changes in CLI tools
====================
* redis-benchmark avoid aborting on NOPERM from CONFIG GET (#11096)
Bug Fixes
=========
* Avoid hang of diskless replication fork child when parent crashes (#11463)
* Fix crash with module API of list iterator and RM_ListDelete (#11383)
* Fix TLS error handling to avoid connection drops on timeouts (#11563)
* Fix runtime changes to cluster-announce-*-port to take effect on the local node too (#10745)
* Fix sentinel function that compares hostnames if failed resolve (#11419)
* Fix MIGRATE with AUTH set to "keys" is getting wrong key names leading to MOVED or ACL errors (#11253)
Fixes for issues in previous releases of Redis 7.0
--------------------------------------------------
* Fix command line startup --sentinel problem (#11591)
* Fis missing FCALL commands in monitor (#11510)
* Fix CLUSTER SHARDS showing empty hostname (#11297)
* Replica that asks for rdb-only could have missed the EOF and hang (#11296)
================================================================================
Redis 7.0.5 Released Wed Sep 21 20:00:00 IST 2022
================================================================================
Upgrade urgency: SECURITY, contains fixes to security issues.
Security Fixes:
* (CVE-2022-35951) Executing a XAUTOCLAIM command on a stream key in a specific
state, with a specially crafted COUNT argument, may cause an integer overflow,
a subsequent heap overflow, and potentially lead to remote code execution.
The problem affects Redis versions 7.0.0 or newer
[reported by Xion (SeungHyun Lee) of KAIST GoN].
Module API changes
==================
* Fix RM_Call execution of scripts when used with M/W/S flags to properly
handle script flags (#11159)
* Fix RM_SetAbsExpire and RM_GetAbsExpire API registration (#11025, #8564)
Bug Fixes
=========
* Fix a hang when eviction is combined with lazy-free and maxmemory-eviction-tenacity is set to 100 (#11237)
* Fix a crash when a replica may attempt to set itself as its master as a result of a manual failover (#11263)
* Fix a bug where a cluster-enabled replica node may permanently set its master's hostname to '?' (#10696)
* Fix a crash when a Lua script returns a meta-table (#11032)
Fixes for issues in previous releases of Redis 7.0
--------------------------------------------------
* Fix redis-cli to do DNS lookup before sending CLUSTER MEET (#11151)
* Fix crash when a key is lazy expired during cluster key migration (#11176)
* Fix AOF rewrite to fsync the old AOF file when a new one is created (#11004)
* Fix some crashes involving a list containing entries larger than 1GB (#11242)
* Correctly handle scripts with a non-read-only shebang on a cluster replica (#11223)
* Fix memory leak when unloading a module (#11147)
* Fix bug with scripts ignoring client tracking NOLOOP (#11052)
* Fix client-side tracking breaking protocol when FLUSHDB / FLUSHALL / SWAPDB is used inside MULTI-EXEC (#11038)
* Fix ACL: BITFIELD with GET and also SET / INCRBY can be executed with read-only key permission (#11086)
* Fix missing sections for INFO ALL when also requesting a module info section (#11291)
================================================================================
Redis 7.0.4 Released Monday Jul 18 12:00:00 IST 2022
================================================================================
Upgrade urgency: SECURITY, contains fixes to security issues.
Security Fixes:
* (CVE-2022-31144) A specially crafted XAUTOCLAIM command on a stream
key in a specific state may result with heap overflow, and potentially
remote code execution. The problem affects Redis versions 7.0.0 or newer.
================================================================================
Redis 7.0.3 Released Monday Jul 11 12:00:00 IST 2022
================================================================================
Upgrade urgency: MODERATE, specifically if you're using a previous release of
Redis 7.0, contains fixes for bugs in previous 7.0 releases.
Performance and resource utilization improvements
=================================================
* Optimize zset conversion on large ZRANGESTORE (#10789)
* Optimize the performance of sending PING on large clusters (#10624)
* Allow for faster restart of Redis in cluster mode (#10912)
INFO fields and introspection changes
=====================================
* Add missing sharded pubsub keychannel count to CLIENT LIST (#10895)
* Add missing pubsubshard_channels field in INFO STATS (#10929)
Module API changes
==================
* Add RM_StringToULongLong and RM_CreateStringFromULongLong (#10889)
* Add RM_SetClientNameById and RM_GetClientNameById (#10839)
Changes in CLI tools
====================
* Add missing cluster-port support to redis-cli --cluster (#10344)
Other General Improvements
==========================
* Account sharded pubsub channels memory consumption (#10925)
* Allow ECHO in loading and stale modes (#10853)
* Cluster: Throw -TRYAGAIN instead of -ASK on migrating nodes for multi-key
commands when the node only has some of the keys (#9526)
Bug Fixes
=========
* TLS: Notify clients on connection shutdown (#10931)
* Fsync directory while persisting AOF manifest, RDB file, and config file (#10737)
* Script that made modification will not break with unexpected NOREPLICAS error (#10855)
* Cluster: Fix a bug where nodes may not acknowledge a CLUSTER FAILOVER TAKEOVER
after a replica reboots (#10798)
* Cluster: Fix crash during handshake and cluster shards call (#10942)
Fixes for issues in previous releases of Redis 7.0
--------------------------------------------------
* TLS: Fix issues with large replies (#10909)
* Correctly report the startup warning for vm.overcommit_memory (#10841)
* redis-server command line allow passing config name and value in the same argument (#10866)
* Support --save command line argument with no value for backwards compatibility (#10866)
* Fix CLUSTER RESET command regression requiring an argument (#10898)
================================================================================
Redis 7.0.2 Released Sunday Jun 12 12:00:00 IST 2022
================================================================================
Upgrade urgency: MODERATE, specifically if you're using a previous release of
Redis 7.0, contains fixes for bugs in previous 7.0 releases.
Bug Fixes
=========
* Fixed SET and BITFIELD commands being wrongly marked movablekeys (#10837)
Regression in 7.0 possibly resulting in excessive roundtrip from cluster clients.
* Fix crash when /proc/sys/vm/overcommit_memory is inaccessible (#10848)
Regression in 7.0.1 resulting in crash on startup on some configurations.
================================================================================
Redis 7.0.1 Released Wed Jun 8 12:00:00 IST 2022
================================================================================
Upgrade urgency: MODERATE, specifically if you're using a previous release of
Redis 7.0, contains some behavior changes for new 7.0 features and important
fixes for bugs in previous 7.0 releases.
Improvements
============
* Add warning for suspected slow system clocksource setting
Add --check-system command line option. (#10636)
* Allow read-only scripts (*_RO commands, and ones with `no-writes` flag)
during CLIENT PAUSE WRITE (#10744)
* Add `readonly` flag in COMMAND command for EVAL_RO, EVALSHA_RO and FCALL_RO (#10728)
* redis-server command line arguments now accept one string with spaces
for multi-arg configs (#10660)
Potentially Breaking Changes
============================
* Omitting a config option value in command line argument no longer works (#10660)
* Hide the `may_replicate` flag from the COMMAND command response (#10744)
Potentially Breaking Changes for new Redis 7.0 features
-------------------------------------------------------
* Protocol: Sharded pubsub publish emits `smessage` instead of `message` (#10792)
* CLUSTER SHARDS returns slots as RESP integers, not strings (#10683)
* Block PFCOUNT and PUBLISH in read-only scripts (*_RO commands, and no-writes) (#10744)
* Scripts that declare the `no-writes` flag are implicitly `allow-oom` too (#10699)
Changes in CLI tools
====================
* redis-cli --bigkeys, --memkeys, --hotkeys, --scan. Finish nicely after Ctrl+C (#10736)
Platform / toolchain support related improvements
=================================================
* Support tcp-keepalive config interval on MacOs (#10667)
* Support RSS metrics on Haiku OS (#10687)
INFO fields and introspection changes
=====================================
* Add isolated network metrics for replication. (#10062, #10810)
Module API changes
==================
* Add two more new checks to RM_Call script mode (#10786)
* Add new RM_Call flag to let Redis automatically refuse `deny-oom` commands (#10786)
* Add module API RM_MallocUsableSize (#10795)
* Add missing REDISMODULE_NOTIFY_NEW (#10688)
* Fix cursor type in RedisModuleScanCursor to handle more than 2^31 elements (#10698)
* Fix RM_Yield bugs and RM_Call("EVAL") OOM check bug (#10786)
* Fix bugs in enum configs with overlapping bit flags (#10661)
Bug Fixes
=========
* FLUSHALL correctly resets rdb_changes_since_last_save INFO field (#10691)
* FLUSHDB is now propagated to replicas / AOF, even if the db is empty (#10691)
* Replica fail and retry the PSYNC if the master is unresponsive (#10726)
* Fix ZRANGESTORE crash when zset_max_listpack_entries is 0 (#10767)
Fixes for issues in previous releases of Redis 7.0
--------------------------------------------------
* CONFIG REWRITE could cause a config change to be dropped for aliased configs (#10811)
* CONFIG REWRITE would omit rename-command and include lines (#10761)
NOTE: Affected users who used Redis 7.0.0 to rewrite their configuration file
should review and fix the file.
* Fix broken protocol after MISCONF (persistence) error (#10786)
* Fix --save command line regression (#10690)
* Fix possible regression around TLS config changes. re-load files even if the
file name didn't change. (#10713)
* Re-add SENTINEL SLAVES command, missing in redis 7.0 (#10723)
* BZMPOP gets unblocked by non-key args and returns them (#10764)
* Fix possible memory leak in XADD and XTRIM (#10753)
================================================================================
Redis 7.0.0 GA Released Wed Apr 27 12:00:00 IST 2022
================================================================================
Upgrade urgency: SECURITY, contains fixes to security issues.
Security Fixes:
* (CVE-2022-24736) An attacker attempting to load a specially crafted Lua script
can cause NULL pointer dereference which will result with a crash of the
redis-server process. This issue affects all versions of Redis.
[reported by Aviv Yahav].
* (CVE-2022-24735) By exploiting weaknesses in the Lua script execution
environment, an attacker with access to Redis can inject Lua code that will
execute with the (potentially higher) privileges of another Redis user.
[reported by Aviv Yahav].
New Features
============
* Keyspace event for new keys (#10512)
Command replies that have been extended
---------------------------------------
* COMMAND DOCS shows deprecated_since field in command args (#10545)
* COMMAND DOCS shows module name where applicable (#10544)
Potentially Breaking Changes
============================
* Replicas panic when they fail writing persistence (#10504)
* Prevent cross slot operations in functions and scripts with shebang (#10615)
* Rephrased some error responses about invalid commands or args (#10612)
* Lua scripts do not have access to the print() function (#10651)
Performance and resource utilization improvements
=================================================
* Speed optimization in streams (#10574)
* Speed optimization in command execution pipeline (#10502)
* Speed optimization in listpack encoded sorted (#10486)
* Speed optimization in latency tracking at INFO (relevant for 7.0 RCs) (#10606)
* Speed optimization when there are many replicas (relevant for 7.0 RCs) (#10588)
New configuration options
=========================
* Allow ignoring disk persistence errors on replicas (#10504)
* Allow abort with panic when replica fails to execute a command sent by the master (#10504)
* Allow configuring shutdown flags of SIGTERM and SIGINT (#10594)
* Allow attaching an operating system-specific identifier to Redis sockets (#10349)
Module API changes
==================
* Add argument specifying ACL reason for module log entry (#10559)
Breaking API compatibility with 7.0 RCs
* Add the deprecated_since field in command args of COMMAND DOCS (#10545)
Breaking API/ABI compatibility with 7.0 RCs
* Add module API flag for using enum configs as bit flags (#10643)
* Add RM_PublishMessageShard (#10543)
* Add RM_MallocSizeString, RM_MallocSizeDict (#10542)
* Add RM_TryAlloc (#10541)
Bug Fixes
=========
* Replica report disk persistence errors in PING (#10603)
* Fixes around rejecting commands on replicas and AOF when they must be respected (#10603)
* Durability fixes for appendfsync=always policy (#9678)
Fixes for issues in previous release candidates of Redis 7.0
------------------------------------------------------------
* Fix possible crash on CONFIG REWRITE (#10598)
* Fix regression not aborting transaction on errors (#10612)
* Fix auto-aof-rewrite-percentage based AOFRW trigger after restart (#10550)
* Fix bugs when AOF enabled after startup, in case of failure before the first rewrite completes (#10616)
* Fix RM_Yield module API bug processing future commands of the current client (#10573)
================================================================================
Redis 7.0 RC3 Released Tue Apr 5 12:00:00 IST 2022
================================================================================
Upgrade urgency LOW: This is another Release Candidate of Redis 7.0.
New Features
============
New administrative and introspection commands and command arguments
-------------------------------------------------------------------
* CLUSTER SHARDS command deprecates CLUSTER SLOTS (#10293)
Potentially Breaking Changes
============================
* CONFIG GET response returned in a non-deterministic order.
It's possible that a client was relying on configs order (#10323)
* SORT / SORT_RO commands reject keys access patterns in GET and BY if ACL
doesn't grant the command full keyspace access (#10340)
* FUNCTION LOAD command introduced in 7.0-RC1 was stripped of the ENGINE, and
NAME arguments which are now part of the script itself. The DESCRIPTION
argument was completely removed (#10500)
* Set disable-thp config to be immutable (#10409)
Performance and resource utilization improvements
=================================================
* Optimize performance and memory usage on replicas (#10413)
* A faster and more robust code of zslRandomLevel using RAND_MAX (#5539)
Changes in CLI tools
====================
* redis-cli: Use exit code 1 on error (#10468)
* redis-cli: Do DNS lookup before sending CLUSTER MEET (#10436)
* redis-benchmark: Fix --cluster with IPv6. (#10393)
* redis-cli: Better --json Unicode support and --quoted-json (#10286)
INFO fields and introspection changes
=====================================
* MEMORY STATS: Show cluster.links memory usage (#10302)
Module API changes
==================
* APIs for exposing module configs to config file and CONFIG command (#10285)
* Add an event notifying about configuration changes (#10311)
* Add API for redacting command arguments from SLOWLOG and MONITOR (#10425)
* RM_Call: new flags for script mode compatibility, no writes, and error replies (#10372)
Bug Fixes
=========
* Sentinel: Fix no reconnect after auth-pass is changed (#10400)
* Cluster: Fix race condition: Turn into replica on SETSLOT (#10489, #10381)
* XREADGROUP: Unblock client when the stream key is deleted (#10306)
Fixes for issue in previous release candidates of Redis 7.0
-----------------------------------------------------------
* ACL DRYRUN does not validate the verified command args. (#10405)
* ACL DRYRUN returns the tested common permission error (#10359)
* Incorrect parsing of hostname information from nodes.conf (#10435)
* BITSET and BITFIELD SET should propagate even if just length changed (#10459)
* SHUTDOWN, Fix a possible crash when the shutdown was aborted (#10440)
* Script should not allow may-replicate commands when client pause write (#10364)
* Optimization tracking memory usage from i/o threads. (#10401)
* Initialize help when using redis-cli help or redis-cli ? (#10382)
* Dismiss COW of client output buffer now that it's dynamic (#10371)
* Fix memory corruption when EVAL fails before being processed (#10519)
================================================================================
Redis 7.0 RC2 Released Mon Feb 28 12:00:00 IST 2022
================================================================================
Upgrade urgency LOW: This is another Release Candidate of Redis 7.0.
New Features
============
* Add stream consumer group lag tracking and reporting (#9127)
* Add API for functions and eval Lua scripts to check ACL explicitly (#10220)
New user commands or command arguments
--------------------------------------
* COMMAND GETKEYSANDFLAGS sub-command (#10237)
* INFO command can take multiple section arguments (#6891)
* XGROUP CREATE and SETID: new ENTRIESREAD optional argument (#9127)
* XSETID new ENTRIESADDED and MAXDELETEDID optional arguments (#9127)
Command replies that have been extended
---------------------------------------
* XINFO reports consumer group lag and a few other fields (#9127)
* XAUTOCLAIM returns a new element with a list of deletes IDs (#10227)
Potentially Breaking Changes
============================
* X[AUTO]CLAIM skips deleted entries instead of replying with Nil, and deletes
them from the pending entry list (#10227)
* Fix messed up error codes returned from EVAL scripts (#10218, #10329)
* COMMAND INFO, Renames key-spec "CHANNEL" flag to be "NOT_KEY" (#10299)
Performance and resource utilization improvements
=================================================
* Reduce system calls and small packets for client replies (#9934)
* Reduce memory usage of stale clients (#9822)
* Fix regression in Z[REV]RANGE commands (by-rank) introduced in Redis 6.2 (#10337)
Changes in CLI tools
===================
* Adapt redis-check-aof tool for Multi Part AOF (#10061)
* Enable redis-benchmark to use RESP3 protocol mode (#10335)
Platform / toolchain support related improvements
=================================================
* Fix OpenSSL 3.0.x related issues (#10291)
INFO fields and introspection changes
=====================================
* COMMAND INFO key-specs has new variable_flags flag (#10237, #10148)
* INFO stats: add aof_rewrites and rdb_snapshots counters (#10178)
* INFO stats: add reply_buffer_shrinks and reply_buffer_expends (#9822)
* INFO modules: add no-implicit-signal-modified module option (#10284)
Module API changes
==================
* Add RM_SetCommandInfo API to set command metadata for the new COMMAND
introspection features and ACL key permissions (#10108)
* Add RM_KeyAtPosWithFlags and RM_GetCommandKeysWithFlags APIs (#10237)
* Add getchannels-api command flag and RM_IsChannelsPositionRequest,
RM_ChannelAtPosWithFlags APIs (#10299)
* Change RM_ACLCheckChannelPermissions and RM_ACLCheckKeyPermissions APIs
(released in RC1) to take different flags (#10299)
* Fix RM_SetModuleOptions flag collision. Bug in 7.0 RC1 header file, modules
that used OPTIONS_HANDLE_REPL_ASYNC_LOAD will mess up key invalidations (#10284)
Bug Fixes
=========
* Modules: Fix thread safety violation when a module thread adds an error reply,
broken in 6.2 (#10278)
* Lua: Fix Eval scripts active defrag, broken 7.0 in RC1 (#10271)
* Fix geo search bounding box check causing missing results (#10018)
* Lua: Add checks for min-slave-* configs when evaluating Lua scripts and
Functions (#10160)
* Modules: Prevent crashes and memory leaks when MODULE UNLOAD is used on module
with a pending timer (#10187)
* Fix error stats and failed command stats for blocked clients (#10309)
* Lua/Modules: Fix missing and duplicate error stats for scripts and modules (#10329, #10278)
* Check target node is a primary during cluster setslot (#10277)
* Fix key deletion not to invalidate WATCH when used on a logically expired key (#10256)
* Sentinel: return an error if configuration save fails (#10151)
* Sentinel: fix a free-after-use issue re-registering Sentinels (#10333)
================================================================================
Redis 7.0 RC1 Released Mon Jan 31 12:00:00 IST 2022
================================================================================
Upgrade urgency LOW: This is the first Release Candidate of Redis 7.0.
Redis Release Candidate (RC) versions are early versions that are made available
for early adopters in the community to test them. We do not consider
them suitable for production environments.
Introduction to the Redis 7.0 release
=====================================
Redis 7.0 includes several new user-facing features, significant performance
optimizations, and many other improvements. It also includes changes that
potentially break backwards compatibility with older versions. We urge users to
review the release notes carefully before upgrading.
In particular, users should be aware of the following changes:
1. Redis 7 stores AOF as multiple files in a folder; see Multi-Part AOF below.
2. Redis 7 uses a new version 10 format for RDB files, which is incompatible
with older versions.
3. Redis 7 converts ziplist encoded keys to listpacks on the fly when loading
an older RDB format. Conversion applies to loading a file from disk or
replicating from a Redis master and will slightly increase loading time.
4. See sections about breaking changes mentioned below.
Here is a comprehensive list of changes in this release compared to 6.2.6.
Each one includes the PR number that added it so that you can get more details
at https://github.com/redis/redis/pull/<number>
New Features
============
* Redis Functions: A new way to extend Redis with server-side scripts (#8693)
see https://redis.io/topics/functions-intro
* ACL: Fine-grained key-based permissions and allow users to support multiple
sets of command rules with selectors (#9974)
see https://redis.io/topics/acl#key-permissions and https://redis.io/topics/acl#selectors.
* Cluster: Sharded (node-specific) Pub/Sub support (#8621)
see https://redis.io/topics/pubsub#sharded-pubsub
* First-class handling of sub-commands in most contexts (affecting ACL
categories, INFO commandstats, etc.) (#9504, #10147)
* Command metadata and documentation (#10104)
see https://redis.io/commands/command-docs, https://redis.io/topics/command-tips
* Command key-specs. A better way for clients to locate key arguments and their
read/write purpose (#8324, #10122, #10167)
see https://redis.io/topics/key-specs
* Multi-Part AOF mechanism to avoid AOF rewrite overheads (#9788)
* Cluster: Support for hostnames, instead of IP addresses only (#9530)
* Improved management of memory consumed by network buffers, and an option to
drop clients when total memory exceeds a limit (#8687)
* Cluster: A mechanism for disconnecting cluster bus connections to prevent
uncontrolled buffer growth (#9774)
* AOF: Timestamp annotations and support for point-in-time recovery (#9326)
* Lua: support Function flags in EVAL scripts (#10126)
see https://redis.io/topics/eval-intro#eval-flags
* Lua: Support RESP3 reply for Verbatim and Big-Number types (#9202)
* Lua: Get Redis version via redis.REDIS_VERSION, redis.REDIS_VERSION_NUM (#10066)
New user commands or command arguments
--------------------------------------
* ZMPOP, BZMPOP commands (#9484)
* LMPOP, BLMPOP commands (#9373)
* SINTERCARD, ZINTERCARD commands (#8946, #9425)
* SPUBLISH, SSUBSCRIBE, SUNSUBSCRIBE, PUBSUB SHARDCHANNELS/SHARDNUMSUB (#8621)
* EXPIRETIME and PEXPIRETIME commands (#8474)
* EXPIRE command group supports NX/XX/GT/LT options (#2795)
* SET command supports combining NX and GET flags (#8906)
* BITPOS, BITCOUNT accepts BIT index (#9324)
* EVAL_RO, EVALSHA_RO command variants, to run on read-only replicas (#8820)
* SORT_RO command, to run on read-only replicas (#9299)
* SHUTDOWN arguments: NOW, FORCE, ABORT (#9872)
* FUNCTION *, FCALL, FCALL_RO - https://redis.io/commands/function-load
* CONFIG SET/GET can handle multiple configs atomically, in one call (#9748, #9914)
* QUIT promoted to be a proper command, HOST: and POST demoted (#9798)
* XADD supports auto sequence number via <ms>-* (#9217)
New administrative and introspection commands and command arguments
-------------------------------------------------------------------
* COMMAND DOCS (#9656, #10056, #10104)
* COMMAND LIST (#9504)
* COMMAND INFO accepts sub-commands as args, and no args too (#9504, #10056)
* LATENCY HISTOGRAM (#9462)
* CLUSTER LINKS (#9774)
* CLUSTER DELSLOTSRANGE and CLUSTER ADDSLOTSRANGE (#9445)
* CLIENT NO-EVICT (#8687)
* ACL DRYRUN (#9974)
* SLOWLOG GET supports passing in -1 to get all entries (#9018)
Command replies that have been extended
---------------------------------------
* COMMAND and COMMAND INFO extended with tips, key-specs and sub-commands
see https://redis.io/commands/command
* ACL CAT, COMMAND LIST list sub-commands (#10127)
* MODULE LIST reply includes path and args (#4848)
* OBJECT ENCODING returns listpack instead of ziplist (#8887, #9366)
* CLUSTER SLOTS hostname support (#9530)
* COMMAND command: Added the `blocking` and `module` flags (#10104, #9656)
Potentially Breaking Changes
============================
* Modifying the bind parameter to a non-default value will no longer implicitly
disable protected-mode (#9034)
* Remove EVAL script verbatim replication, propagation, and deterministic
execution logic (#9812)
This has been deprecated and off by default since Redis 6 and is no longer
supported.
* ACL: pub/sub channels are blocked by default (acl-pubsub-default=resetchannels) (#10181)
* SCRIPT LOAD and SCRIPT FLUSH are no longer propagated to replicas / AOF (#9812)
* ACL: Declarations of duplicate ACL users in startup files and command line
arguments will result in an error, whereas previously the last declaration
would overwrite the others. (#9330)
* Replication: TTLs are always replicated as absolute (not relative) millisecond
timestamps (#8474)
* Fixes in handling multi-key commands with expired keys on writable replicas (#9572)
* CONFIG SET maxmemory returns before starting eviction (#10019)
* AOF: The new Multi-Part mechanism stores data as a set of multiple files in a
designated folder (#9788)
* Remove STRALGO command, preserve LCS a standalone command which only works on
keys (#9799)
* Remove gopher protocol support (#9057)
* MODULE and DEBUG commands disabled (protected) by default, for better security (#9920)
* Snapshot-creating and other admin commands in MULTI/EXEC transactions are now
rejected (#10015)
* PING is now rejected with -MASTERDOWN when replica-serve-stale-data=no (#9757)
* ACL GETUSER reply now uses ACL syntax for `keys` and `channels` (#9974)
* COMMAND reply drops `random` and `sort-for-scripts` flags, which are now part
of command tips (#10104)
* LPOP/RPOP with count against non-existing list return null array (#10095)
* INFO commandstats now shows the stats per sub-command (#9504)
* ZPOPMIN/ZPOPMAX used to produce wrong replies when count is 0 with non-zset (#9711)
* LPOP/RPOP used to produce wrong replies when count is 0 (#9692)
* CONFIG GET bind now returns the current value in effect, even if the implicit
default is in use (#9034)
* CONFIG REWRITE now rewrites the list of modules to load (#4848)
* Config: repl-diskless-sync is now set to yes by default (#10092)
* When shutting down, Redis can optionally wait for replicas to catch up on the
replication link (#9872)
* Most CONFIG SET, REWRITE, RESETSTAT commands are now allowed during loading (#9878)
* READONLY and READWRITE commands are now allowed when loading and on stale
replicas (#7425)
* Fix ACL category for SELECT, WAIT, ROLE, LASTSAVE, READONLY, READWRITE, ASKING (#9208)
* RESET is now allowed even when on unauthenticated connections (#9798)
* SCRIPT LOAD is now allowed on stale replicas (#10126)
Security improvements
=====================
* Sensitive configs and commands blocked (protected) by default (#9920)
* Improve bind and protected-mode config handling (#9034)
* Sentinel: avoid logging auth-pass value (#9652)
* redis-cli: sensitive commands bypass the history file (#8895)
Performance and resource utilization improvements
=================================================
* Significant memory saving and latency improvements in cluster mode (#9356)
* Significant memory savings in case of many hash or zset keys (#9228)
* Replication backlog and replicas use one global shared replication buffer (#9166)
* Significant reduction of copy-on-write memory overheads (#8974)
* Free unused capacity in the cluster send buffer (#9255)
* Memory efficiency, make full use of client struct memory for reply buffers (#8968)
* Replace ziplist with listpack in Hash, List, Zset (#8887, #9366, #9740)
* Add support for list type to store elements larger than 4GB (#9357)
* Reuse temporary client objects for blocked clients by module (#9940)
* Remove command argument count limit, dynamically grow argv buffer (#9528)
* Optimize list type operations to seek from the nearest end (#9454)
* Improvements in fsync to avoid large writes to disk (#9409)
* BITSET and BITFIELD SET only propagated when the value actually changed (#9403)
* Improve latency when a client is unblocked by module timer (#9593)
Other General Improvements
==========================
* Make partial sync possible after master reboot (#8015)
* Always create a base AOF file when redis starts from empty (#10102)
* Replica keep serving data during repl-diskless-load=swapdb for better
availability (#9323)
Changes in CLI tools
====================
* redis-cli --json, and -2 options (#9954)
* redis-cli --scan, add sleep interval option (#3751)
* redis-cli --replica optimization, skip RDB generation (#10044)
* redis-cli --functions-rdb, generate RDB with Functions only (#9968)
* redis-cli -X, take an arbitrary arg from stdin, extend --cluster call take -x (#9980)
* redis-benchmark -x takes an argument from stdin (#9130)
* redis-benchmark, Added URI support (#9314)
* redis-cli monitor and pubsub can be aborted with Ctrl+C, keeping the cli alive (#9347)
Platform / toolchain support related improvements
=================================================
* Upgrade jemalloc 5.2.1 (#9623)
* Fix RSS metrics on NetBSD and OpenBSD (#10116, #10149)
* Check somaxconn system settings on macOS, FreeBSD and OpenBSD (#9972)
* Better fsync on MacOS, improve power failure safety (#9545)
New configuration options
=========================
* CONFIG SET/GET can handle multiple configs in one call (#9748, #9914)
* Support glob pattern matching for config include files (#8980)
* appenddirname, folder where multi-part AOF files are stored (#9788)
* shutdown-timeout, default 10 seconds (#9872)
* maxmemory-clients, allows limiting the total memory usage by all clients (#8687)
* cluster-port, can control the bind port of cluster bus (#9389)
* bind-source-addr, configuration argument control IP of outgoing connections (#9142)
* busy-reply-threshold, alias for the old lua-time-limit (#9963)
* repl-diskless-sync-max-replicas, allows faster replication in some cases (#10092)
* latency-tracking, enabled by default, and latency-tracking-info-percentiles (#9462)
* cluster-announce-hostnameand cluster-preferred-endpoint-type (#9530)
* cluster-allow-pubsubshard-when-down (#8621)
* cluster-link-sendbuf-limit (#9774)
* list-max-listpack-*, hash-max-listpack-*, zset-max-listpack-* as aliases for
the old ziplist configs (#8887, #9366, #9740)
INFO fields and introspection changes
=====================================
* INFO: latencystats section (#9462)
* INFO: total_active_defrag_time and current_active_defrag_time (#9377)
* INFO: total_eviction_exceeded_time and current_eviction_exceeded_time (#9031)
* INFO: evicted_clients (#8687)
* INFO: mem_cluster_links, total_cluster_links_buffer_limit_exceeded (#9774)
* INFO: current_cow_peak (#8974)
* INFO: Remove aof_rewrite_buffer_length (#9788)
* MEMORY STATS: Report slot to keys map size in in cluster mode (#10017)
* INFO MEMORY: changes to separate memory usage of Functions and EVAL (#9780)
* INFO MEMORY: Add mem_total_replication_buffers, change meaning of
mem_clients_slaves (#9166)
* CLIENT LIST: tot-mem, multi-mem (#8687)
* CLIENT LIST, INFO: Show RESP version (#9508)
* SENTINEL INFO: tilt_mode_since (#9000)
* LATENCY: Track module-acquire-GIL latency (#9608)
Module API changes
==================
* Add API for replying with RESP3 types (#8521, #9639, #9632)
* Add API for parsing RESP3 replies from RM_Call (#9202)
* Add RM_Call '0' and '3' flags to control RESP version to be used (#9202)
* Add Support for validating ACL explicitly (#9309, #9974)
* Add missing list type functionality APIs (#8439)
* Add API for yielding to Redis events during long busy jobs (#9963)
* Add API for registering other file descriptors to the Redis event loop (#10001)
* Enhance mem_usage/free_effort/unlink/copy and IO callbacks to have key name
and DB index (#8999)
* Enhance mem_usage callback to get the requested sample size (#9612)
* RM_GetContextFlags: CTX_FLAGS_ASYNC_LOADING, CTX_FLAGS_RESP3 (#9323, #9202)
* Mark APIs as non-experimental (#9983)
* RM_CreateSubcommand (#9504)
* RM_KeyExists (#9600)
* RM_TrimStringAllocation (#9540)
* RM_LoadDataTypeFromStringEncver (#9537)
* RM_MonotonicMicroseconds (#10101)
* Add ReplAsyncLoad event and deprecate the ReplBackup event (#9323)
* Add RM_SetModuleOptions OPTIONS_HANDLE_REPL_ASYNC_LOAD flag (#9323)
Bug Fixes
=========
* Fix COMMAND GETKEYS on EVAL without keys (#9733)
* Improve MEMORY USAGE with allocator overheads (#9095)
* Unpause clients after manual failover ends instead of waiting for timed (#9676)
* Lua: fix crash on a script call with many arguments, a regression in v6.2.6 (#9809)
* Lua: Use all characters to calculate string hash to prevent hash collisions (#9449)
* Prevent LCS from allocating temp memory over proto-max-bulk-len (#9817)
* Tracking: Make invalidation messages always after command's reply (#9422)
* Cluster: Hide empty replicas from CLUSTER SLOTS responses (#9287)
* CLIENT KILL killed all clients when used with ID of 0 (#9853)
* Fix bugs around lists with list-compress-depth (#9849, #9779)
* Fix one in a blue moon LRU bug in RESTORE, RDB loading, and module API (#9279)
* Reset lazyfreed_objects info field with RESETSTAT, test for stream lazyfree (#8934)
* Fix RDB and list node compression for handling values larger than 4GB (#9776)
* Fix a crash when adding elements larger than 2GB to a Set or Hash (#9916)
* Diskless replication could not count as a change and skip next database SAVE (#9323)
* Fix excessive stream trimming due to an overflow (#10068)
* Safe and organized exit when receiving SIGTERM while loading (#10003)
* Improve EXPIRE TTL overflow detection (#9839)
* Add missed error counting for INFO errorstats (#9646)
* DECRBY LLONG_MIN caused negation overflow (#9577)
* Delay discarding cached master when full synchronization (#9398)
* Fix Stream keyspace notification and persistence triggers in consumer
creation and deletion (#9263)
* Fix rank overflow in zset with more than 2B entries (#9249)
* Avoid starting in check-aof / check-rdb / sentinel modes if only the folder
name contains that name (#9215, #9176)
* create the log file only after done parsing the entire config file (#6741)
* redis-cli: Fix SCAN sleep interval for --bigkeys, --memkeys, --hotkeys (#9624)
* redis-cli: Fix prompt to show the right DB num and transaction state after
RESET (#9096)
* Module API: fix possible propagation bugs in case a module calls CONFIG SET
maxmemory outside a command (#10019, #9890)
* Module API: carry through client RESP version to module blocked clients (#9634)
* Module API: release clients blocked on module commands in cluster resharding
and down state (#9483)
* Sentinel: Fix availability after master reboot (#9438)
* Sentinel: Fix memory leak with TLS (#9753)
* Sentinel: Fix possible failover due to duplicate zero-port (#9240)
* Sentinel: Fix issues with hostname support (#10146)
* Sentinel: Fix election failures on certain container environments (#10197)
Thanks to all the users and developers who made this release possible.
We'll follow up with more RC releases, until the code looks production ready
and we don't get reports of serious issues for a while.
A special thank you for the amount of work put into this release by:
- Guy Benoish
- Meir Shpilraien
- Oran Agra
- Chen Yang
- Zhu Binbin
- Yoav Steinberg
- sundb
- Madelyn Olson
- Yossi Gottlieb
- Viktor Söderqvist
- Wang Yuan
- Harkrishn Patro
- Nick Chun
- Ozan Tezcan
- Wen Hui
- Huang Zhw
- Nan Yan
- Filipe Oliveira
- Eduardo Semprebon
- Yaacov Hazan
- Itamar Haber
- Zhao Zhao
- Itay Perry
- Moti Cohen
- Ning Sun
- zhugezy
- Ran Shidlansik
- menwen
- Andy Pan