Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

It appears that the sfdisk output may have changes introducing space … #8

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

mkomarsr
Copy link

…padding for cosmetic purposes. This appears to have broken the subsequent 'sed' call to get the sectors. This change should work with or without the padding.

mkomarsr added 2 commits July 28, 2019 12:29
…padding for cosmetic purposes. This appears to have broken the subsequent 'sed' call to get the sectors. This change should work with or without the padding.
@mkomarsr
Copy link
Author

@SpoddyCoder - Have you had a chance to review? I've tested these changes against same size, smaller and larger SD cards and all worked as expected.

@SpoddyCoder
Copy link
Owner

@mkomarsr in the middle of testing now :) Should have an update for you later this evening.

@SpoddyCoder
Copy link
Owner

SpoddyCoder commented Aug 19, 2019

@mkomarsr - firstly, thanks very much for reporting, chasing up and contributing to the resolution. On the latest Rasbian Buster I can recreate the issue on cloning to a smaller card.

Testing the PR here didn't resolve for me, clonepi output below.

It looks like you may be running a NOOBS build atm? I'm running a standard 2 partition Raspbian Lite. ClonePi currently works with both (& in theory supports an arbitrary number of partitions) - so your approach would need more dynamic handling of the partitions inside the initDest() function. I'll add some annotations to the PR for you to check out.

Your approach may still be a winner - but I'm going to investigate the root cause in the original code over the next couple of days.

$ sudo clonepi /dev/sdb --init-destination --ignore-warnings

Warning: destination disk /dev/sdb is smaller than the source disk /dev/mmcblk0
ClonePi will attempt to resize the last partition on the destination,
but this may result in a partition too small to take all of the content.
Pre-flight checks completed with WARNINGS, proceed with caution
Info: the following actions will be taken...
Info: 1) destinaton disk /dev/sdb will be initialised to match the source disk structure
Info: 2) the last partition on /dev/sdb will be resized to fill the disk
Info: 3) destination disk partitions will be mounted inside /mnt/clone/:
Info:    /dev/sdb2
Info:    /dev/sdb1
Info: 4) filesystems will be synced:
Info:    /  -->  /mnt/clone/sdb2
Info:    /boot  -->  /mnt/clone/sdb1
Info: 5) cleanup source + destination mounts & dirs

Do you wish to continue (yes|no)? y
User confirmed, starting clone process : 2019-08-19T22:49:56
Info: initialising /dev/sdb, this may take some time...
Info: setting up destination partition structure, copying 278MB...
sfdisk: cannot open /dev//dev/sdb: No such file or directory
Checking that no-one is using this disk right now ... OK

Disk /dev/sdb: 14.9 GiB, 15931539456 bytes, 31116288 sectors
Disk model: USB3.0 CRW   -SD
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xab4c34f8

Old situation:

Device     Boot Start      End  Sectors  Size Id Type
/dev/sdb1        8192    96453    88262 43.1M  c W95 FAT32 (LBA)
/dev/sdb2       98304 31116287 31017984 14.8G 83 Linux

>>> Created a new DOS disklabel with disk identifier 0x114bdad5.
/dev/sdb1: Created a new partition 1 of type 'W95 FAT16 (LBA)' and of size 63 MiB.
/dev/sdb2: Created a new partition 2 of type 'Extended' and of size 14.8 GiB.
/dev/sdb3: Created a new partition 5 of type 'Linux' and of size 32 MiB.
/dev/sdb6: Created a new partition 6 of type 'W95 FAT32 (LBA)' and of size 256 MiB.
/dev/sdb7: Created a new partition 7 of type 'Linux' and of size 14.5 GiB.
/dev/sdb8: Done.

New situation:
Disklabel type: dos
Disk identifier: 0x114bdad5

Device     Boot  Start      End  Sectors  Size Id Type
/dev/sdb1         2048   131071   129024   63M  e W95 FAT16 (LBA)
/dev/sdb2       131072 31116287 30985216 14.8G  5 Extended
/dev/sdb5       133120   198655    65536   32M 83 Linux
/dev/sdb6       200704   724991   524288  256M  c W95 FAT32 (LBA)
/dev/sdb7       727040 31116287 30389248 14.5G 83 Linux

The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
mkfs.fat 4.1 (2017-01-24)
mke2fs 1.44.5 (15-Dec-2018)
The file /dev/sdb5 does not exist and no size was specified.
mkfs.fat 4.1 (2017-01-24)
mkfs.vfat: unable to open /dev/sdb6: No such file or directory
Info: running filesystem repair on last partition /dev/sdb2
Info: Fixing MBR on sdb...
sfdisk: cannot open /dev//dev/sdb: No such file or directory
Checking that no-one is using this disk right now ... FAILED

This disk is currently in use - repartitioning is probably a bad idea.
Umount all file systems, and swapoff all swap partitions on this disk.
Use the --no-reread flag to suppress this check.

sfdisk: Use the --force flag to overrule all checks.
mkfs.fat 4.1 (2017-01-24)
mke2fs 1.44.5 (15-Dec-2018)
The file /dev/sdb5 does not exist and no size was specified.
mkfs.fat 4.1 (2017-01-24)
mkfs.vfat: unable to open /dev/sdb6: No such file or directory
Info: ... updated MBR OK
Info: rebuilding ext4 filesystem on last partition /dev/sdb2...
mke2fs 1.44.5 (15-Dec-2018)
/dev/sdb2 contains a ext4 file system labelled 'rootfs'
	last mounted on / on Thu Jan 24 10:17:02 2019
Creating filesystem with 3877248 4k blocks and 971040 inodes
Filesystem UUID: 4d5fde03-2f35-4e8d-be86-309eccdc3ff9
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208

Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

Info: ...filesystem rebuild completed OK
Info: /dev/sdb initialised OK
Info: setting up source directories and mounts...
Info: setting up destination directories and mounts...
Info: created clone dir at /mnt/clone
Info: created clone partition dir at /mnt/clone/sdb2
Info: created clone partition dir at /mnt/clone/sdb1
Info: mounted /dev/sdb2 to /mnt/clone/sdb2
Info: mounted /dev/sdb1 to /mnt/clone/sdb1
Info: starting filesystem sync processes, this may take some time...
Info: syncing: /  -->  /mnt/clone/sdb2
Info: this is the running root directory, applying OS excludes
Info: rsync --force --delete -rltgopxWDEHAX --exclude=/mnt/source --exclude=/mnt/clone --exclude-from=/etc/clonepi/raspbian.excludes / /mnt/clone/sdb2/
Info: syncing: /boot  -->  /mnt/clone/sdb1
Info: rsync --force --delete -rltgopxWDEHAX /boot/ /mnt/clone/sdb1/
rsync: write failed on "/mnt/clone/sdb1/start_db.elf": No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(374) [receiver=3.1.3]
Info: ...all sync processes finished OK
Info: cleaning up...
Info: unmounted destination partition from /mnt/clone/sdb2
Info: deleted empty destination partition dir /mnt/clone/sdb2
Info: unmounted destination partition from /mnt/clone/sdb1
Info: deleted empty destination partition dir /mnt/clone/sdb1
Info: removed clone dir /mnt/clone
Info: total clone process time : 2m 32s
ClonePi finished succesfully but with WARNINGS : 2019-08-19T22:52:28

@@ -1091,6 +1091,17 @@ runPostSyncHook()
fi
}

initDest()
{
sfdisk --delete /dev/$DEST_DISK
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
sfdisk --delete /dev/$DEST_DISK
sfdisk --delete $DEST_DISK

$DEST_DISK contains /dev/ already

echo -e "size=+63M, type=e\ntype=5\nsize=+32M,type=83\nsize=+256M,type=c\ntype=83"| sfdisk $DEST_DISK
partprobe
mkfs.vfat ${DEST_DISK}1
mkfs -t ext4 -F ${DEST_DISK}5
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes there is a partition 5 - won't be the case for a vanilla Raspbian install (2 partitions).

initDest()
{
sfdisk --delete /dev/$DEST_DISK
echo -e "size=+63M, type=e\ntype=5\nsize=+32M,type=83\nsize=+256M,type=c\ntype=83"| sfdisk $DEST_DISK
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is very specific structure, that matches your current install & will likely match most with NOOBS, but won't match many others.

This is the real reason ClonePi currently does a partial dd to copy the MBR - there's a lot of ways to setup a system at the partition level.

@mkomarsr
Copy link
Author

Yeah, my entire thought process was in replicating NOOBS images. The problem with a more dynamic partitioning is in determining which partition(s) to resize based on the dest disk size.

Maybe the answer is to resize them all relative to the percentage of the original disk that each one is using.

@mkomarsr
Copy link
Author

I haven't taken the time to roll back and investigate ... but my gut says the root problem here has to do with the parsing of the output from fdisk/sfdisk/something else and that output changing between versions and thus the pattern matching no longer matches.

That's pretty much entirely based on the one pattern match that I fixed. Either the resizing NEVER worked (which I find unlikely) or the output changed such that it broke.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants