meta data for this page
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
linux:fs:btrfs_multidisk [2017/02/06 13:33] niziak [GRUB] |
linux:fs:btrfs_multidisk [2021/02/16 21:07] (current) niziak |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| ====== BTRFS on multiple devices ====== | ====== BTRFS on multiple devices ====== | ||
| + | |||
| + | [[https://serverfault.com/questions/814546/how-to-obtain-read-speeds-of-two-disks-using-mdadm-btrfs-raid1-or-zfs-mirror]] | ||
| + | |||
| + | ===== Encryption layout ===== | ||
| + | |||
| + | ==== BTRFS on crypted devices ==== | ||
| + | | btrfs raid1 || | ||
| + | | crypt | crypt | | ||
| + | | sda1 | sdb1 | | ||
| + | |||
| + | * (-) Encryption has to be performed twice (no problem for modern CPUs) | ||
| + | * (+) Bit-flips can be detected immediately and corrected by BTRFS | ||
| + | * (-) not possible to get benefit from hybrid RAID1 (mixed disc HDD & SSD) | ||
| + | |||
| + | |||
| + | ==== BTRFS on crypted devices with LVM ==== | ||
| + | |||
| + | | btrfs raid1 || | ||
| + | | crypt | crypt | | ||
| + | | LVM | LVM | | ||
| + | | (DM-Cache on fast SSD) || | ||
| + | | sda1 | sdb1 | | | ||
| + | |||
| + | * (-) Encryption has to be performed twice (no problem for modern CPUs) | ||
| + | * (+) Bit-flips can be detected immediately and corrected by BTRFS | ||
| + | * (-) not possible to get benefit from hybrid RAID1 (mixed disc HDD & SSD) | ||
| + | * (-) small overhead from LVM but | ||
| + | * (+) DM-cache with SSD disc can be used: [[https://www.redhat.com/en/blog/improving-read-performance-dm-cache]] | ||
| + | |||
| + | ==== BTRFS on soft RAID1 ==== | ||
| + | | btrfs || | ||
| + | | crypt || | ||
| + | | raid1 || | ||
| + | | sda1 | sdb1 | | ||
| + | |||
| + | * (+) encryption performed only once (less CPU consumption) | ||
| + | * (-) Bit-flips detected only during RAID1 scan. | ||
| + | * (+) Benefits from hybrid RAID1 (SSH & HHD) using "writemostly" feature). See [[http://tansi.info/hybrid/]] | ||
| + | |||
| + | |||
| ===== Prepare 2nd disc ===== | ===== Prepare 2nd disc ===== | ||
| Line 18: | Line 58: | ||
| ===== btrfs 2nd disk ===== | ===== btrfs 2nd disk ===== | ||
| - | Add 2nd disc to btrfs: | + | ==== Add 2nd disc to btrfs ==== |
| <code bash> | <code bash> | ||
| btrfs device add /dev/disc1/btrfs1 / | btrfs device add /dev/disc1/btrfs1 / | ||
| btrfs device usage / | btrfs device usage / | ||
| - | btrfs balance start -v -dconvert=raid1 -mconvert=raid1 / | + | nice ionice -c3 btrfs balance start -v -dconvert=raid1 -mconvert=raid1 -dsoft -msoft / |
| btrfs device usage / | btrfs device usage / | ||
| </code> | </code> | ||
| + | Note: "soft" filter will convert only chunks which are not already converted (usefull when previous balance was incomplete) | ||
| + | |||
| Make sure that data on both disc are the same. Especially system data are on both disc in RAID1 mode. | Make sure that data on both disc are the same. Especially system data are on both disc in RAID1 mode. | ||
| Line 46: | Line 88: | ||
| </code> | </code> | ||
| + | |||
| + | OR | ||
| + | |||
| + | ==== Create blank 2nd disc ==== | ||
| + | NOT POSSIBLE: | ||
| + | |||
| + | <del>Create degraded RAID1 with single disc (to add 2nd later)</del> | ||
| + | <code bash> | ||
| + | mkfs.btrfs -d raid1 -m raid1 -L btrfs_pool2 /dev/mapper/pool2 | ||
| + | </code> | ||
| + | |||
| ==== Balance command ==== | ==== Balance command ==== | ||
| Line 141: | Line 194: | ||
| WORKS! | WORKS! | ||
| - | ==== Grub ==== | + | ==== Grub wrong root= ==== |
| - | **PROBLEM** | + | === PROBLEM === |
| **''grub-pc'' 2.02~beta2-22+deb8u1 0** incorrectly handle multiple roots. Internal variable ${GRUB_DEVICE} which is used to generate ''root='' parameter include newline character, so grub menuconfig entry is broken: | **''grub-pc'' 2.02~beta2-22+deb8u1 0** incorrectly handle multiple roots. Internal variable ${GRUB_DEVICE} which is used to generate ''root='' parameter include newline character, so grub menuconfig entry is broken: | ||
| - | <code> | + | <file /boot/grub/grub.cfg> |
| - | linux<->/@/@root/boot/vmlinuz-4.8.0-0.bpo.2-amd64 root=/dev/mapper/disc2-btrfs2 | + | linux /@/@root/boot/vmlinuz-4.8.0-0.bpo.2-amd64 root=/dev/mapper/disc2-btrfs2 |
| /dev/mapper/disc1-btrfs1 ro rootflags=subvol=@/@root rootflags=degraded | /dev/mapper/disc1-btrfs1 ro rootflags=subvol=@/@root rootflags=degraded | ||
| - | </code> | + | </file> |
| - | More here: [[https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1238347]] and [[https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1582811]] | + | More here: |
| - | **SOLUTION** | + | * [[https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1238347]] |
| + | * [[https://bugs.launchpad.net/ubuntu/+source/grub2/+bug/1582811]] | ||
| + | |||
| + | === SOLUTION === | ||
| Fixed in Debian Stretch (testing) version **grub-pc_2.02~beta3-3** | Fixed in Debian Stretch (testing) version **grub-pc_2.02~beta3-3** | ||
| Now UUID is used multiple roots. UUID of btrfs filesystem placed on 2 disc is the same: | Now UUID is used multiple roots. UUID of btrfs filesystem placed on 2 disc is the same: | ||
| + | <file /boot/grub/grub.cfg> | ||
| + | linux /@/@root/boot/vmlinuz-4.8.0-0.bpo.2-amd64 root=UUID=52944cdd-f8d0-4798-bd1c-80539c45253d ro rootflags=subvol=@/@root rootflags=degraded | ||
| + | </file> | ||
| <code bash> | <code bash> | ||
| + | # blkid | ||
| /dev/mapper/disc2-btrfs2: UUID="52944cdd-f8d0-4798-bd1c-80539c45253d" UUID_SUB="54703ddb-5789-4cac-bfd0-691acadfa33c" TYPE="btrfs" | /dev/mapper/disc2-btrfs2: UUID="52944cdd-f8d0-4798-bd1c-80539c45253d" UUID_SUB="54703ddb-5789-4cac-bfd0-691acadfa33c" TYPE="btrfs" | ||
| /dev/mapper/disc1-btrfs1: UUID="52944cdd-f8d0-4798-bd1c-80539c45253d" UUID_SUB="9bc49b38-c014-40e3-876e-08f6873293b8" TYPE="btrfs" | /dev/mapper/disc1-btrfs1: UUID="52944cdd-f8d0-4798-bd1c-80539c45253d" UUID_SUB="9bc49b38-c014-40e3-876e-08f6873293b8" TYPE="btrfs" | ||