meta data for this page
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision | ||
|
linux:fs:zfs:tuning [2022/06/08 17:23] niziak |
linux:fs:zfs:tuning [2023/02/01 07:16] (current) niziak |
||
|---|---|---|---|
| Line 1: | Line 1: | ||
| - | ====== ZFS tuning tips ====== | + | ====== ZFS performance tuning tips ====== |
| + | |||
| + | ===== Tune L2ARC for backups ===== | ||
| + | |||
| + | When huge portion of data are written (new backups) or read (backup verify) L2ARC is constantly written with current data. | ||
| + | To change this behaviour to cache only Most Frequent Use: | ||
| + | |||
| + | <file conf /etc/modprobe.d/zfs.conf> | ||
| + | options zfs l2arc_mfuonly=1 l2arc_noprefetch=0 | ||
| + | </file> | ||
| + | |||
| + | Explanation: | ||
| + | * [[https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html#l2arc-mfuonly|l2arc_mfuonly]] Controls whether only MFU metadata and data are cached from ARC into L2ARC. This may be desirable to avoid wasting space on L2ARC when reading/writing large amounts of data that are not expected to be accessed more than once. By default both MRU and MFU data and metadata are cached in the L2ARC. | ||
| + | * [[https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html#l2arc-noprefetch|l2arc_noprefetch]] Disables writing prefetched, but unused, buffers to cache devices. Setting to 0 can increase L2ARC hit rates for workloads where the ARC is too small for a read workload that benefits from prefetching. Also, if the main pool devices are **very slow**, setting to 0 can improve some workloads such as **backups**. | ||
| ===== I/O scheduler ===== | ===== I/O scheduler ===== | ||
| Line 28: | Line 41: | ||
| </code> | </code> | ||
| - | ===== reduce ARC RAM usage ===== | + | ===== reduce ZFS ARC RAM usage ===== |
| By default ZFS can sue 50% of RAM for ARC cache: | By default ZFS can sue 50% of RAM for ARC cache: | ||
| <code bash> | <code bash> | ||
| + | # apt install zfsutils-linux | ||
| + | |||
| # arcstat | # arcstat | ||
| time read miss miss% dmis dm% pmis pm% mmis mm% size c avail | time read miss miss% dmis dm% pmis pm% mmis mm% size c avail | ||
| Line 52: | Line 67: | ||
| </code> | </code> | ||
| - | ARC size can be tuned by settings ''zfs'' kernel module parameters ([[https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html#zfs-arc-max|Module Parameters]]: | + | ARC size can be tuned by settings ''zfs'' kernel module parameters ([[https://openzfs.github.io/openzfs-docs/Performance%20and%20Tuning/Module%20Parameters.html#zfs-arc-max|Module Parameters]]): |
| * ''zfs_arc_max'': Maximum size of ARC in bytes. If set to 0 then the maximum size of ARC is determined by the amount of system memory installed (50% on Linux) | * ''zfs_arc_max'': Maximum size of ARC in bytes. If set to 0 then the maximum size of ARC is determined by the amount of system memory installed (50% on Linux) | ||
| * ''zfs_arc_min'': Minimum ARC size limit. When the ARC is asked to shrink, it will stop shrinking at ''c_min'' as tuned by ''zfs_arc_min''. | * ''zfs_arc_min'': Minimum ARC size limit. When the ARC is asked to shrink, it will stop shrinking at ''c_min'' as tuned by ''zfs_arc_min''. | ||
| - | * ''zfs_arc_meta_limit_percent'': Sets the limit to ARC metadata, arc_meta_limit, as a percentage of the maximum size target of the ARC, ''c_max''. | + | * ''zfs_arc_meta_limit_percent'': Sets the limit to ARC metadata, arc_meta_limit, as a percentage of the maximum size target of the ARC, ''c_max''. Default is 75. |
| - | [[https://pve.proxmox.com/wiki/ZFS_on_Linux#sysadmin_zfs_limit_memory_usage|Limit ZFS Memory Usage]] | + | Proxmox recommends following [[https://pve.proxmox.com/wiki/ZFS_on_Linux#sysadmin_zfs_limit_memory_usage|rule]]: |
| As a general rule of thumb, allocate at least 2 GiB Base + 1 GiB/TiB-Storage | As a general rule of thumb, allocate at least 2 GiB Base + 1 GiB/TiB-Storage | ||
| + | |||
| + | ==== Examples ==== | ||
| | | ||
| - | Set ''zfs_arc_max'' to 4GB: | + | Set ''zfs_arc_max'' to 4GB and ''zfs_arc_min'' to 128MB: |
| <code bash> | <code bash> | ||
| echo "$[4 * 1024*1024*1024]" >/sys/module/zfs/parameters/zfs_arc_max | echo "$[4 * 1024*1024*1024]" >/sys/module/zfs/parameters/zfs_arc_max | ||
| Line 69: | Line 86: | ||
| Make options persistent: | Make options persistent: | ||
| <file /etc/modprobe.d/zfs.conf> | <file /etc/modprobe.d/zfs.conf> | ||
| + | options zfs zfs_prefetch_disable=1 | ||
| + | options zfs zfs_arc_max=4294967296 | ||
| + | options zfs zfs_arc_min=134217728 | ||
| + | options zfs zfs_arc_meta_limit_percent=75 | ||
| </file> | </file> | ||