meta data for this page
Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
linux:fs:btrfs_as_root2 [2017/01/20 14:02] niziak [Replace bad disc] |
linux:fs:btrfs_as_root2 [2021/02/17 08:53] (current) niziak |
||
---|---|---|---|
Line 52: | Line 52: | ||
**NOTE:** Create subvolumes for things which shouldn't be included in snapshot of parent volume. Like some /var/../cache, etc | **NOTE:** Create subvolumes for things which shouldn't be included in snapshot of parent volume. Like some /var/../cache, etc | ||
+ | |||
+ | [[https://wiki.archlinux.org/index.php/Snapper#Suggested_filesystem_layout]] | ||
<code bash> | <code bash> | ||
Line 119: | Line 121: | ||
Create subvolumes: | Create subvolumes: | ||
- | <code> | + | <code bash> |
cd /mnt | cd /mnt | ||
btrfs subvolume create @pool | btrfs subvolume create @pool | ||
Line 130: | Line 132: | ||
</file> | </file> | ||
+ | [[https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Btrfs_on_top_of_dmcrypt]] | ||
===== Add 2nd disc ===== | ===== Add 2nd disc ===== | ||
- | Copy partition layouts from sdb to sda drive: | + | See [[btrfs_multidisk]] |
- | <code bash> | + | |
- | sudo sfdisk -d /dev/sdb > parts | + | |
- | sudo sfdisk /dev/sda < parts | + | |
- | </code> | + | |
- | + | ||
- | Recreate LVM layout. | + | |
- | + | ||
- | Add 2nd disc to btrfs: | + | |
- | <code bash> | + | |
- | btrfs device add /dev/disc1/btrfs1 / | + | |
- | btrfs device usage / | + | |
- | btrfs balance start -v -dconvert=raid1 -mconvert=raid1 / | + | |
- | btrfs device usage / | + | |
- | </code> | + | |
- | Make sure that data on both disc are the same. Especially system data are on boths disc in RAID1 mode. | + | |
- | + | ||
- | + | ||
- | Let system boot without one disc: | + | |
- | <file | /etc/default/grub> | + | |
- | GRUB_CMDLINE_LINUX_DEFAULT="rootflags=degraded" | + | |
- | </file> | + | |
- | <code bash>update-grub</code> | + | |
- | + | ||
- | Install grub | + | |
- | <code bash> | + | |
- | sudo grub-install /dev/sdb | + | |
- | sudo grub-install /dev/sdc | + | |
- | reboot | + | |
- | </code> | + | |
- | + | ||
- | ==== Issues ==== | + | |
- | + | ||
- | === Grub === | + | |
- | **PROBLEM** | + | |
- | **''grub-pc'' 2.02~beta2-22+deb8u1 0** incorrectly handle multiple roots. Internal variable ${GRUB_DEVICE} which is used to generate ''root='' parameter include newline character, so grub menuconfig entry is broken: | + | |
- | <code> | + | |
- | linux<->/@/@root/boot/vmlinuz-4.8.0-0.bpo.2-amd64 root=/dev/mapper/disc2-btrfs2 | + | |
- | /dev/mapper/disc1-btrfs1 ro rootflags=subvol=@/@root rootflags=degraded | + | |
- | </code> | + | |
- | More here: [[https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1238347]] and [[https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1582811]] | + | |
- | **SOLUTION** | + | |
- | Fixed in Debian Stretch (testing) version **grub-pc_2.02~beta3-3** | + | |
- | Now UUID is used multiple roots. UUID of btrfs filesystem placed on 2 disc is the same: | + | |
- | <code bash> | + | |
- | /dev/mapper/disc2-btrfs2: UUID="52944cdd-f8d0-4798-bd1c-80539c45253d" UUID_SUB="54703ddb-5789-4cac-bfd0-691acadfa33c" TYPE="btrfs" | + | |
- | /dev/mapper/disc1-btrfs1: UUID="52944cdd-f8d0-4798-bd1c-80539c45253d" UUID_SUB="9bc49b38-c014-40e3-876e-08f6873293b8" TYPE="btrfs" | + | |
- | </code> | + | |
- | So, kernel cmdline parameter **''root=UUID=...''** is now correct. | + | |
- | + | ||
- | === no symbol table === | + | |
- | **PROBLEM** | + | |
- | Grub says: | + | |
- | <code>grub error: no symbol table</code> | + | |
- | **SOLUTION** | + | |
- | Grub was not reinstalled after update | + | |
- | <code bash> | + | |
- | grub-install /dev/sda | + | |
- | grub-install /dev/sdb | + | |
- | update-grub | + | |
- | </code> | + | |
- | + | ||
- | === ALERT! /dev/disk/by-uuid/xxxxxxxxx does not exist. Dropping to a shell === | + | |
- | + | ||
- | **PROBLEM** | + | |
- | Grub loads kernel and initramfs correctly, but: | + | |
- | ''Begin: Waiting for root file system ...'' and Grub still boot into initramfs. | + | |
- | After some while, initramfs shell is available. | + | |
- | <code> | + | |
- | ALERT! | + | |
- | Dropping to shell! | + | |
- | </code> | + | |
- | + | ||
- | Unfortunatelly initramfs cannot activate LVM volumes when kernel cmdline ''root=UUID=...'' is used. | + | |
- | <code bash> | + | |
- | lvm lvscan | + | |
- | </code> | + | |
- | To boot manually: | + | |
- | <code bash> | + | |
- | lvm vgchange -ay | + | |
- | mount /dev/disc2/btrfs2 /root -o device=/dev/disc1/btrfs1 | + | |
- | exit | + | |
- | </code> | + | |
- | System should start - but once. | + | |
- | + | ||
- | Problem is located in ''/usr/share/initramfs-tools/scripts/local-top/lvm2''. | + | |
- | Script check if specified root device needs to be activated by LVM. | + | |
- | When UUID is used it is executing code: | + | |
- | <code bash> | + | |
- | /dev/*/*) | + | |
- | # Could be /dev/VG/LV; use lvs to check | + | |
- | if lvm lvs -- "$dev" >/dev/null 2>&1; then | + | |
- | lvchange_activate "$dev" | + | |
- | fi | + | |
- | ;; | + | |
- | </code> | + | |
- | In result, command | + | |
- | <code bash>lvm lvs -- disc/by-uuid/52944cdd-f8d0-4798-bd1c-80539c45253d<code> | + | |
- | is executed, resulting following output: | + | |
- | <code>"disc/by-uuid/52944cdd-f8d0-4798-bd1c-80539c45253d": Invalid path for Logical Volume.</code> | + | |
- | + | ||
- | Debian bugs about similar problem: | + | |
- | * [[https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=612402]] | + | |
- | * [[https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=741342]] | + | |
- | + | ||
- | + | ||
- | **solution** | + | |
- | + | ||
- | Install lvm2 package version 2.02.168-1 (Debian Stretch). Previously was 2.02.111-2.2+deb8u1 (Debian Jessie). | + | |
- | <code bash> | + | |
- | update-initramfs -k all -u | + | |
- | btrfs scrub / | + | |
- | </code> | + | |
- | + | ||
- | or | + | |
- | force LVM activation: [[http://askubuntu.com/a/834626|Can't find LVM root dropped back to initramfs]] | + | |
- | + | ||
- | Also ''GRUB_DISABLE_OS_PROBER=true'' can be added. | + | |
- | + | ||
- | + | ||
- | <file | /etc/mkinitcpio.conf> | + | |
- | BINARIES="/bin/btrfs" | + | |
- | </file> | + | |
- | <code bash>update-init -u -k all</code> | + | |
- | + | ||
- | ===== Disc fail (removed) ===== | + | |
- | + | ||
- | Make sure, all system, metadata and data is balanced to RAID1 mode. Without this it cannot be possible to mount degraded btrfs in rw mode. | + | |
- | + | ||
- | During RW operation in degraded mode, single system, metadata and data structures will be created on BTRFS so it will need rebalance after missing disc will be connected again. | + | |
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | === error: disk not found === | + | |
- | <code> | + | |
- | error: disk 'lvmid/8vc3Xl-....' not found. | + | |
- | Entering rescue mode... | + | |
- | grub rescue> | + | |
- | </code> | + | |
- | + | ||
- | GRUB 2 is unable to find the grub folder or its contents are missing/corrupted. The grub folder contains the GRUB 2 menu, modules and stored environmental data. | + | |
- | + | ||
- | grub.cfg sets prefix variable to missing LVM (lvimd/<VG UUID/<LV UUID>), and then call to load module which fails: | + | |
- | <code> | + | |
- | prefix='(lvmid/8vc3Xl-a40o-ILx2-AoRf-9Z37-4h51-jq0OR5/Hr8CPf-dVqs-uIm1-pkfH-fJdI-14io-rwtx3l)/@/@root/boot/grub' | + | |
- | insmod gettext | + | |
- | </code> | + | |
- | + | ||
- | Rescue shell is very limited. Try to start ''normal'' shell. | + | |
- | <code> | + | |
- | set prefix='(lvm/disc1-btrfs1)/@/@root/boot/grub' | + | |
- | insmod normal | + | |
- | normal | + | |
- | </code> | + | |
- | + | ||
- | Keeping /boot/grub in multidevice BTRFS is stupid idea. Another problem comes when BTRFS is not clean, and cannot be used by GRUB to load own modules. | + | |
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | + | ||
- | ===== Replace bad disc ===== | + | |
- | + | ||
- | <code bash> | + | |
- | btrfs device add /dev/disc3/btrfs3 / | + | |
- | btrfs device delete missing / | + | |
- | </code> | + | |
- | + | ||
- | <code bash> | + | |
- | btrfs filesystem show | + | |
- | btrfs replace start 6 /dev/disc3/btrfs3 / # change 6 to your setup | + | |
- | </code> | + | |
- | + | ||
- | Don't forget to restore correct data state: | + | |
- | <code bash> | + | |
- | btrfs scrub start / | + | |
- | btrfs balance start -v -dconvert=raid1 -mconvert=raid1 / | + | |
- | </code> | + | |
- | + | ||
- | ==== Fine tuning ==== | + | |
- | * Add your ssh public key to ''/etc/initramfs-tools/root/.ssh/authorized_keys'' | + | |
- | * Change MAC address of eth0 (for initramfs remote access) | + | |
- | <file | /etc/udev/rules.d/75-mac-spoof.rule> | + | |
- | ACTION=="add", SUBSYSTEM=="net", ATTR{address}=="08:00:27:f9:3d:3e", RUN+="/sbin/ip link set dev %k address 08:00:27:f9:12:34" | + | |
- | </file> | + | |
- | Another option is to add ''local-top'' script like described here for IPv6 [[https://www.danrl.com/2015/10/21/debian-jessi-ssh-fde-unlock.html]] | + | |
- | * | ||
- | ==== TODO ==== | ||
- | * backup GPT header | ||
- | * backup LVM header | ||
- | * backup LUKS header | ||