meta data for this page
Differences
This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
linux:fs:btrfs [2020/05/06 15:27] niziak |
linux:fs:btrfs [2021/05/02 11:34] (current) niziak |
||
---|---|---|---|
Line 45: | Line 45: | ||
!!! It is possible to remove any device if there is free space to which data can be moved during removal. | !!! It is possible to remove any device if there is free space to which data can be moved during removal. | ||
- | ===== Compression ===== | ||
- | |||
- | [[https://btrfs.wiki.kernel.org/index.php/Compression]] | ||
- | |||
- | Compression: | ||
- | * ZLIB -- slower, higher compression ratio (uses zlib level 3 setting) | ||
- | * LZO -- faster compression and decompression than zlib, worse compression ratio, designed to be fast | ||
- | * ZSTD -- (since v4.14) compression comparable to zlib with higher compression/decompression speeds and different ratio levels [[https://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git/commit/?h=next&id=5c1aab1dd5445ed8bdcdbb575abc1b0d7ee5b2e7|details]] | ||
- | |||
- | Default compression for kernels up to 3.6 it's ZLIB. For compatibility with old kernel ZLIB is default. | ||
- | LZO is fast and new algo, but can be buggy. | ||
- | |||
- | |||
- | [[https://btrfs.wiki.kernel.org/index.php/FAQ#Does_a_balance_operation_recompress_files.3F|Does a balance operation recompress files?]] | ||
- | >No. Balance moves entire file extents and does not change their contents. If you want to recompress files, use btrfs filesystem defrag with the -c option. | ||
- | >Balance does a defragmentation, but not on a file level rather on the block group level. It can move data from less used block groups to the remaining ones, eg. using the usage balance filter. | ||
- | |||
- | NOTE: Defragmentation with recompression destroys deduplication. | ||
- | * Size changed from 6,94 GB to 7,39 GB | ||
- | * After deduplication with 1024k block drops to 7.17 GB (net change in shared extents of: 490.0M) | ||
- | * After deduplication with 4k block drops from 7.17 GB to 6,94 GB(a net change in shared extents of: 306.6M) | ||
- | |||
- | ==== mount options ==== | ||
- | Can be enabled during mount time: | ||
- | <code bash> | ||
- | mount BTRFS -o remount,compress=zlib | ||
- | mount BTRFS -o remount,compress-force=zlib | ||
- | </code> | ||
- | > if the first portion of data being compressed is not smaller than the original, the compression of the file is disabled -- unless the filesystem is mounted with compress-force | ||
- | |||
- | **compress-force=<method>** - Enable compression even for files that don't compress well, like videos and dd images of disks. | ||
- | |||
- | Test data of size 651MB takes 371MB after compression. | ||
- | |||
- | ==== chattr +c ==== | ||
- | Also single file compression possible using ''chattr +c filename.txt'' | ||
- | |||
- | To recompress exisitng files, start defragmentation with -c param: | ||
- | <code>btrfs filesystem defrag -c file.iso</code> | ||
- | <code>btrfs filesystem defrag -c -r mydir</code> | ||
- | |||
- | It is not possible to get compression ratio of file. | ||
- | |||
- | Internals: | ||
- | * In compressed extents, individual blocks are not compressed separately; rather, the compression stream spans the entire extent. | ||
- | * BTRFS: There is a simple decision logic: if the first portion of data being compressed is not smaller than the original, the compression of the file is disabled -- unless the filesystem is mounted with compress-force | ||
- | |||
- | ==== btrfs property ==== | ||
- | <code bash> | ||
- | btrfs property set <file> compression <zlib|lzo|zstd> | ||
- | </code> | ||
- | ===== Deduplication ===== | ||
- | External tool to find duplicated extens available | ||
- | [[https://github.com/markfasheh/duperemove]] | ||
- | |||
- | Simply do 'make' and 'make install' | ||
- | Following binaries will be installed: | ||
- | * btrfs-extent-same | ||
- | * csum-test | ||
- | * duperemove | ||
- | * hashstats | ||
- | * show-shared-extents | ||
- | |||
- | Recursively find duplicates in <dir> | ||
- | <code>duperemove -r <dir></code> | ||
- | |||
- | The same as above, but make real deduplication (on BTRFS only) | ||
- | <code>duperemove -rd <dir></code> | ||
- | |||
- | Increase deduplication chance by reducing block size to 4k: | ||
- | <code>duperemove -b 4k -rd <dir></code> | ||
- | |||
- | This deduplication works nice, it can find some shared parts inside Maildir. | ||
- | |||
- | |||
- | ===== Subvolumes ===== | ||
- | |||
- | <code> | ||
- | btrfs subvolume create /BTRFS/sub1 | ||
- | btrfs subvolume create /BTRFS/sub2 | ||
- | btrfs subvolume create /BTRFS/sub3 | ||
- | </code> | ||
- | |||
- | <code> | ||
- | btrfs subvolume list /BTRFS | ||
- | ID 256 gen 234 top level 5 path sub1 | ||
- | ID 257 gen 227 top level 5 path sub2 | ||
- | ID 258 gen 228 top level 5 path sub3 | ||
- | </code> | ||
- | |||
- | Mount subvolume | ||
- | <code>mount -o subvol=sv1/sv12 /dev/sdb /mnt</code> | ||
- | |||
- | Getting data size on each subcvolume: | ||
- | * Enable quota on btrfs ''btrfs quota enable /'' | ||
- | * Install script [[https://github.com/agronick/btrfs-size]] | ||
- | |||
- | |||
- | ==== Different mount options ==== | ||
- | NOT SUPPORTED YET: | ||
- | In general, btrfs doesn't _yet_ have the runtime infrastructure to handle | ||
- | per-subvolume mount options. The on-device format and general approach | ||
- | in the kernel was designed to allow it, and it's on the roadmap, it just | ||
- | hasn't been done... yet. | ||
- | |||
- | To make workaround it is possible to use attributes on files or directories: | ||
- | * C disable COW feature (for newly created directories or zero sized files). New files in directores with C attributes will be created with C attrib. But not subdirectories. | ||
- | * c enable compression | ||
- | |||
- | ==== Move data between subvolumes ==== | ||
- | See `cp` command arg: | ||
- | When --reflink[=always] is specified, perform a lightweight copy, where the data blocks are copied only when modified. If this is not possible the copy fails, or if | ||
- | --reflink=auto is specified, fall back to a standard copy. | ||
- | |||
- | <code bash> | ||
- | cp -pr --reflink=always srcDirectory dstDirectory/ | ||
- | rm -r srcDirectory | ||
- | </code> | ||
====== Scrub ====== | ====== Scrub ====== | ||
Line 182: | Line 64: | ||
<code bash>btrfs scrub status /</code> | <code bash>btrfs scrub status /</code> | ||
- | ===== periodic scrub ===== | ||
- | From arch linux ''btrfs-progs'' | + | ====== migration to BTRFS ====== |
- | <file | /etc/systemd/system/btrfs-scrub@.service> | + | <code bash>apt-get install btrfs-tools</code> |
- | [Unit] | + | |
- | Description=Btrfs scrub on %f | + | |
- | [Service] | + | ====== Issues ====== |
- | Nice=19 | + | |
- | IOSchedulingClass=idle | + | |
- | ExecStart=/bin/btrfs scrub start -B %f | + | |
- | </file> | + | |
- | <file | /etc/systemd/system/btrfs-scrub@.timer> | + | ===== Compression type 0x3 not supported ===== |
- | [Unit] | + | |
- | Description=Monthly Btrfs scrub on %f | + | |
- | + | ||
- | [Timer] | + | |
- | OnCalendar=monthly | + | |
- | AccuracySec=1d | + | |
- | Persistent=true | + | |
- | + | ||
- | [Install] | + | |
- | WantedBy=multi-user.target | + | |
- | </file> | + | |
+ | GRUB 2.02 doesn't support ZSTD compression. | ||
+ | Solution is to exlude ''/boot'' from ZSTD compression (below) or install more recent Grub (from Debian Bullseye) | ||
<code bash> | <code bash> | ||
- | systemctl daemon-reload | + | sudo btrfs filesystem defragment -v -r -czlib /boot |
- | systemctl enable btrfs-scrub@-.timer | + | sudo btrfs property set /boot compression zlib |
- | systemctl start btrfs-scrub@-.timer # to enable for / | + | # disable compression for newly created files |
- | + | chattr -Rc /boot | |
- | systemctl enable btrfs-scrub@home.timer # to enable for /home | + | |
- | systemctl start btrfs-scrub@home.timer # to enable for /home | + | |
</code> | </code> | ||
- | |||
- | |||
- | ====== migration to BTRFS ====== | ||
- | |||
- | <code bash>apt-get install btrfs-tools</code> | ||
- | |||
- | ====== Issues ====== | ||
===== error inheriting props for ino 3336468 (root 264): -28 ===== | ===== error inheriting props for ino 3336468 (root 264): -28 ===== |