meta data for this page
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionLast revisionBoth sides next revision | ||
linux:fs:zfs:dedup [2021/02/17 07:07] – niziak | linux:fs:zfs:dedup [2021/03/14 22:44] – niziak | ||
---|---|---|---|
Line 2: | Line 2: | ||
For deduplication, | For deduplication, | ||
- | |||
- | |||
- | |||
* For every TB of pool data, you should expect 5 GB of dedup table data, assuming an average block size of 64K. | * For every TB of pool data, you should expect 5 GB of dedup table data, assuming an average block size of 64K. | ||
* This means you should plan for at least 20GB of system RAM per TB of pool data, if you want to keep the dedup table in RAM, plus any extra memory for other metadata, plus an extra GB for the OS. | * This means you should plan for at least 20GB of system RAM per TB of pool data, if you want to keep the dedup table in RAM, plus any extra memory for other metadata, plus an extra GB for the OS. | ||
Line 17: | Line 14: | ||
Huge CPU usage by over 96 ZFS kernel threads noticed with open-zfs v8.0.6 (ZFS On Linux), when some big parts of data deleted (auto snapshot rotation). It is connected with deduplication enabled and causes system to almost freeze because of high CPU usage! | Huge CPU usage by over 96 ZFS kernel threads noticed with open-zfs v8.0.6 (ZFS On Linux), when some big parts of data deleted (auto snapshot rotation). It is connected with deduplication enabled and causes system to almost freeze because of high CPU usage! | ||
+ | |||
+ | ==== WARNING! ==== | ||
+ | |||
+ | Issue when deleting large portion of data and deduplication enabled. | ||
+ | ZFS driver creates 96 '' | ||
+ | These threads kills CPU and IO. | ||
+ | |||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | |||
Line 31: | Line 40: | ||
Once all deduped datasets are destroyed the dedup table will be removed and the performance impact is cleared. | Once all deduped datasets are destroyed the dedup table will be removed and the performance impact is cleared. | ||
+ | |||
===== status ===== | ===== status ===== |