zfs-2.4.0-rc2 patchset #17716
Merged
zfs-2.4.0-rc2 patchset #17716
+5,077
−827
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Update the zfsunlock initramfs hook to provide instructions on how to unlock the root filesystem when appropriate. The intent is to make the dropbear ssh MOTD more user friendly. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Ahelenia Ziemiańska <[email protected]> Signed-off-by: Cong Zhang <[email protected]> Closes openzfs#17661 Closes openzfs#17662
The concurrent execution of feature_sync() can lead to a panic due to an unprotected update of the feature refcount. Resolve this by using the spa->spa_feat_stats_lock to synchronize the update of the refcount. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Youzhong Yang <[email protected]> Closes openzfs#17184 Closes openzfs#17632
Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Mark Johnston <[email protected]> Closes openzfs#17665
Add a test case to reproduce issue openzfs#17277: 1. Make a pool 2. Write a file to the pool 3. Mount the file as a loopback device 4. Make an XFS filesystem on the loopback device 5. Mount the XFS filesystem... <hangs> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Rob Norris <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Issue openzfs#17277 Closes openzfs#17329
If $KERNEL_CC was not defined, configure status output would print an empty string where the kernel compiler should have been. Fix this and simplify the code generally. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ivan Shapovalov <[email protected]> Closes openzfs#16997
Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Ivan Shapovalov <[email protected]> Closes openzfs#16997
The sorting logic is all in cmd/zfs/zfs_iter.c. I borrowed where I could from the comments in the source code, but please note that the comment to zfs_sort() is a little imprecise, or at least incomplete, because it doesn't give any indication of the chronological sort that will be used by default for snapshots in zfs_compare(). While adding this description, I took the liberty to copy-edit the rest of the file lightly. In those edits, I've removed "If specified, you can list property information by the absolute pathname or the relative pathname" because, in context, it seems more confusing than helpful. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Rob Norris <[email protected]> Signed-off-by: Shawn Bayern <[email protected]> Closes openzfs#15713 Closes openzfs#15869
Add an openzfs-2.4 compatibility file for the next release. While there are no compatibility difference between Linux and FreeBSD for 2.4 symlinks for the -linux and -freebsd names are created for any scripts expecting that convention. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: ofthesun9 <[email protected]> Closes openzfs#17672 Closes openzfs#17673
As described in freebsd/freebsd-src#1305, FreeBSD's installer defaults to zroot/home for user home directories. For FreeBSD only, set the default prefix for pam_zfs_key to match. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Eric A. Borisch <[email protected]> Closes openzfs#17600
glibc includes linux/stat.h for statx, but musl defines its own statx struct and associated constants, which does not include STATX_MNT_ID yet. Thus, including linux/stat.h directly should be avoided for maximum libc compatibility. Tested on: - glibc: x86_64, i686, aarch64, armv7l, armv6l - musl: x86_64, aarch64, armv7l, armv6l Reviewed-by: Brian Behlendorf <[email protected]> Tested-By: Achill Gilgenast <[email protected]> Signed-off-by: classabbyamp <[email protected]> Closes openzfs#17675
Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Alexander Ziaee <[email protected]> Closes openzfs#17676
LLVM-21 enables -Wuninitialized-const-pointer which results in the
following compiler warning and the bdev_file_open_by_path() interface
not being detected for 6.9 and newer kernels. The blk_holder_ops
are not used by the ZFS code so we can safely use a NULL argument
for this check.
bdev_file_open_by_path/bdev_file_open_by_path.c:110:54: error:
variable 'h' is uninitialized when passed as a const pointer
argument here [-Werror,-Wuninitialized-const-pointer]
Reviewed-by: Rob Norris <[email protected]>
Signed-off-by: Brian Behlendorf <[email protected]>
Closes openzfs#17682
Closes openzfs#17684
Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17690
If ARCH environment variable is set it can cause the failure of the kernel modules check during the configure step. The resulting error will be confusing, and may looks like this: > checking for kernel config option compatibility... done > checking whether CONFIG_MODULES is defined... no > configure: error: > *** This kernel does not include the required loadable module > *** support! Detect when ARCH is print a warning. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Maksym Shkolnyi <[email protected]> Closes openzfs#17680
Because GitHub creates a merge commit on top of real head, so the check on HEAD will fail regardlessly. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#17695
Otherwise it might become `if [ == "" ]` which is ill-formed. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#17695
They will become zarcsummary and zarcstat in 2.4.0. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#16357 Closes openzfs#17695
Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#16357 Closes openzfs#17695
Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#16357 Closes openzfs#17695
If we call ddt_log_load() for legacy ddt, we will end up going into ddt_log_update_stats() and filling uninitialized value into ddo_dspace. This value will then get added to dedup_table_size during ddt_get_dedup_object_stats(). Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Closes openzfs#17019 Closes openzfs#17699 Signed-off-by: Chunwei Chen <[email protected]> Co-authored-by: Chunwei Chen <[email protected]>
If the target already exists, lt will fail. Force it to recreate the symlinks. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17702
FreeBSD now has a pathconf name called _PC_CLONE_BLKSIZE which is the block size supported for block cloning for the file system. Since ZFS's block size varies per file, return the largest size likely to be used, or zero if block cloning is not supported. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Rick Macklem <[email protected]> Closes openzfs#17645
GCC complains about casting a 64-bit integer to a 32-bit pointer. Originally committed downstream as freebsd/freebsd-src@2d76470b701 Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Alan Somers <[email protected]> Sponsored by: ConnectWise Closes openzfs#17706
This is one problem currently preventing OpenZFS from building on FreeBSD/i386. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Alan Somers <[email protected]> Sponsored by: ConnectWise Closes openzfs#17704
fault_limits would often hit the 10min timeout and be killed on Fedora 41-42. Investigation showed that the 'fill_fs' portion of the test, which would fill the pool with junk data before vdev replacement, was writing highly compressible data (~126x), which would have taxed the CPUs, potentially causing the timeout. The fix is to write random data and reduce the number of writes. This has an added benefit that more real data being is written to the pool (~1GB) vs the old way (~300-400MB). It also speeds up the test. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Paul Dagnelie <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes openzfs#17709
Accidentally removed calls in ed048fd. Sponsored-by: https://despairlabs.com/sponsor/ Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17621
We only have extremely narrow uses, so move it all into a single function that does only what we need, with and without d_set_d_op(). Sponsored-by: https://despairlabs.com/sponsor/ Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17621
While rw_destroy() may do nothing on Linux, we still want to make sure that we don't have any holders outstanding like we do for mutexes. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17718
Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Colm Buckley <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#16357 Closes openzfs#17712
As a quality assurance measure, `typeset` is added to local variable declarations to actually enforce their intended scope. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: buzzingwires <[email protected]> Closes openzfs#17732
Traditionally, unused inodes would be held on the superblock inode cache until the associated on-disk file is removed or the kernel requests reclaim. On filesystems with millions of rarely-used files, this can be a lot of unusable memory. Here we implement the superblock drop_inode method, and add a zfs_delete_inode tunable to control its behaviour. By default it continues the traditional behaviour, but when the tunable is enabled, we signal that the inode should be deleted immediately when the last reference is dropped, rather than cached. This releases the associated data to the dbuf cache and ARC, allowing them to be reclaimed normally. Sponsored-by: Klara, Inc. Sponsored-by: Fastmail Pty Ltd Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17746
Traditionally, unused dentries would be cached in the dentry cache until the associated entry is no longer on disk. The cached dentry continues to hold an inode reference, causing the inode to be pinned (see previous commit). Here we implement the dentry op d_delete, which is roughly analogous to the drop_inode superblock op, and add a zfs_delete_dentry tunable to control its behaviour. By default it continues the traditional behaviour, but when the tunable is enabled, we signal that an unused dentry should be freed immediately, releasing its inode reference, and so allowing that inode to be deleted if no longer in use. Sponsored-by: Klara, Inc. Sponsored-by: Fastmail Pty Ltd Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17746
On i386, Clang complains about misaligned atomic operations. Silence these warnings to fix the build on FreeBSD/i386. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Alan Somers <[email protected]> Sponsored by: ConnectWise Closes openzfs#17708
This is breaking the build on FreeBSD/i386. Originally committed downstream as freebsd/freebsd-src@2d76470b701 Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Alan Somers <[email protected]> Sponsored by: ConnectWise Closes openzfs#17705
This commit fixes the issue and includes the zfs kernel module even when dracut is used in hostonly mode. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Jo Zzsi <[email protected]> Closes openzfs#17754
zfs-helpers.sh is a utility script that sets up udev symlinks so you can run ZTS from a local ZFS git workspace. However, it doesn't check that the udev symlinks point to the current workspace. They may point to an old workspace that has been deleted. This means the udev rules never get executed, which in turn causes the zvol tests to fail. This commit removes old symlinks that do not point to the current ZFS workspace. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes openzfs#17766
Update documentation to use the correct terminology. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: trick2011 <[email protected]> Closes openzfs#17734 Closes openzfs#17755
Spacemap entry might be too big to fit into a block pointer ashift. We hit an assertion trying to run `zdb -bvy` on a large pool. But it seems the code does not really need size there, since we only need to search for a range of offsets, so setting it to zero should just make btree return position just before the first entry. I suspect the previous code could actually miss the first entry due to this if its size was smaller. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Igor Kozhukhov <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Closes openzfs#17764
Eliminates the need for the following workaround > Add other drivers to dracut: ``` if grep mpt3sas /proc/modules; then echo 'force_drivers+=" mpt3sas "' >> /etc/dracut.conf.d/zfs.conf fi if grep virtio_blk /proc/modules; then echo 'filesystems+=" virtio_blk "' >> /etc/dracut.conf.d/fs.conf fi ``` Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Jo Zzsi <[email protected]> Closes openzfs#17762
Modify the test case to use the `zfs mount` command instead of directly calling the mount command, create a dedicated dataset, and use the default mount point. These changes are intended to preserve the intent of the original test case and resolve some spurious mount failures which have been observed by the CI. Reviewed-by: Igor Kozhukhov <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17785
Three cases were discovered where 'zpool add' would fail to warn when adding vdevs to a pool with a mismatched replication level. These are: 1. When a pool contains mixed file and disk vdevs. 2. When a pool contains an active dRAID distributed spare 3. When a pool contains an active hot spare The lack of warnings are caused by get_replication() assessing the current pool configuration an inconsistent and disabling the mismatched replication check for the new pool configuration after 'zpool add'. This change updates get_replication() to be slightly more tolerant in the non-fatal case. The zpool_add_010_pos.ksh test case was split in to separate tests: zpool_add_warn_create.ksh, pool_add_warn_degraded.ksh, and zpool_add_warn_removal. These test were extended to include coverage for dRAID pools and the three scenarios described above. Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17780
Provide an interface to retrieve the lowest and highest minimum allocation size for the normal allocation class. This can be used by external consumers of the DMU to estimate potential wasted capacity when setting the recordsize for an object. The new "min_alloc" and "max_alloc" keys are added to the pool configuration and used by default_volblocksize() to warn when an ineffecient block size is requested. For older kmods which don't yet include the new keys fallback to the previous logic. Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17758
This changes the basic search algorithm from a single search up and down the tree to a full depth-first traversal to handle conditions where the tree matches at a higher level but not a lower level. Normally higher level blocks always point to matching blocks, but there are cases where this does not happen: 1. Racing block pointer updates from dbuf_write_ready. Before f664f1e (openzfs#8946), both dbuf_write_ready and dnode_next_offset held dn_struct_rwlock which protected against pointer writes from concurrent syncs. This no longer applies, so sync context can f.e. clear or fill all L1->L0 BPs before the L2->L1 BP and higher BP's are updated. dnode_free_range in particular can reach this case and skip over L1 blocks that need to be dirtied. Later, sync will panic in free_children when trying to clear a non-dirty indirect block. This case was found with ztest. 2. txg > 0, non-hole case. This is openzfs#11196. Freeing blocks/dnodes breaks the assumption that a match at a higher level implies a match at a lower level when filtering txg > 0. Whenever some but not all L0 blocks are freed, the parent L1 block is rewritten. Its updated L2->L1 BP reflects a newer birth txg. Later when searching by txg, if the L1 block matches since the txg is newer, it is possible that none of the remaining L1->L0 BPs match if none have been updated. The same behavior is possible with dnode search at L0. This is reachable from dsl_destroy_head for synchronous freeing. When this happens open context fails to free objects leaving sync context stuck freeing potentially many objects. This is also reachable from traverse_pool for extreme rewind where it is theoretically possible that datasets not dirtied after txg are skipped if the MOS has high enough indirection to trigger this case. In both of these cases, without backtracking the search ends prematurely as ESRCH result implies no more matches in the entire object. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Akash B <[email protected]> Signed-off-by: Robert Evans <[email protected]> Closes openzfs#16025 Closes openzfs#11196
This change adds support for ZFS_KEYFORMAT_RAW to zdb_derive_key in zdb.c. The implementation reads the raw key from the file specified by the -K option which is consistent with how raw keys are handled in the other parts of ZFS, along with a check to ensure that the keyfile doesn't have too many bytes. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Patrick Xia <[email protected]> Closes openzfs#17783
When updating a Fedora instance to an experimental kernel make sure to include the matching versioned perf and bpftool packages. This helps ensure there are no unexpected conflicts which would prevent the new packages from being installed. Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17791
Update the META file to reflect compatibility with the 6.17 kernel. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Rob Norris <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17789
The Buildbot CI infrastructure has been fully replaced by GitHub Actions. Remove any lingering references from the repository. Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#17794
Signed-off-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Closes openzfs#17795
The zvol blk-mq codepaths would erroneously send FLUSH and TRIM commands down the read codepath, rather than write. This fixes the issue, and updates the zvol_misc_fua test to verify that sync writes are actually happening. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Ameer Hamza <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes openzfs#17761 Closes openzfs#17765
Add a -O option to zfs-test.sh to dump debug information on test timeout. The debug info includes: - 30 lines from 'top' - /proc/<PID>/stack output of process with highest CPU usage - Last lines strace-ing process with highest CPU usage - /proc/sysrq-trigger kernel stack traces All debug information gets dumped to /dev/kmsg (Linux only). In addition, print out the VM console lines from the "Setup Testing Machines" step. We have often see VMs timeout at this step and don't know why. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Tony Hutter <[email protected]> Closes openzfs#17753
When running zpool iostat in interval mode, it would not notice any new pools created or imported, and would forget any destroyed or exported, so would not notice if they came back. This leads to outputting "no pools available" every interval until killed. It looks like this was at least intended to work; the comment above zpool_do_iostat() indicates that it is expected to "deal with pool creation/destruction" and that pool_list_update() would detect new pools. That call however was removed in 3e43edd, though its unclear if that broke this behaviour and it wasn't noticed, or if it never worked, or if something later broke it. That said, the lack of pool_list_update() is only part of the reason it doesn't work properly. The fundamental problem is that the various things involved in refreshing or updating the list of pools would aggressively ignore, remove, skip or fail on pools that stop existing, or that already exist. Mostly this meant that once a pool is removed from the list, it will never be seen again. Restoring pool_list_update() to the zpool_do_iostat() loop only partially fixes this - it would find "new" pools again, but only in the "all pools" (no args) mode, and because its iterator callback add_pool() would abort the iterator if it already has a pool listed, it would only add pools if there weren't any already. So, this commit reworks the structure somewhat. pool_list_update() becomes pool_list_refresh(), and will ensure the state of all pools in the list are updated. In the "all pools" mode, it will also add new pools and remove pools that disappear, but when a fixed list of pools is used, the list doesn't change, only the state of the pools within it. The rest of the commit is adjusting things for this much simpler structure. Regardless of the mode in use, pool_list_refresh() will always do the right thing, so the driver code can just get on with the display. Now that pools can appear and disappear, I've made it so the header (if enabled) is re-printed when the list changes, so that its easier to see what's happening if the column widths change. Since this is all rather complicated, I've included tests for the "all pools" and "set of pools" modes. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Igor Kozhukhov <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#17786
Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: hoshinomori <[email protected]> Closes openzfs#17800
Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: nav1s <[email protected]> Closes openzfs#17797
Signed-off-by: Brian Behlendorf <[email protected]>
tonyhutter
approved these changes
Sep 30, 2025
|
@tonyhutter I'm right now sorting out a regression in |
|
@robn sorry, I've already pushed out the commits and tagged it. We'll be sure to include your fix in rc3. |
|
Dang. Then |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Initial patch set for zfs-2.4.0-rc2. I expect to include some additional cherry picks from master in this PR before tagging rc2.
Description
This pull in all of the recent fixes on the master branch with a one significant exception. The #17375 patch to drop the legacy FreeBSD aliases wont be included until zfs-2.5.
How Has This Been Tested?
Will be tested by the CI.
Types of changes