CEPH provides only a pools of object. To use it for VMs block devices additional layer (RBD) is needed.
It can be created manually or during CEPH pool creation (option Add as Storage
)
It is implementation of POSIX compliant FS top of CEPH POOL. It requires one pool for data (block data) and to keep filesystem information (metadata). Performance strictly depends on metadata pool, so it is recommended to use for backups files.
Used ports:
Read Proxmox CEPH requirements. It requires at least one spare hard drive on each node. Topic for later.
Datacenter
–> Ceph
–> Install Ceph-nautilus
Ceph
–> Monitor
Create
button in Monitor section, and select available nodes.Create Object Storage Daemon
On every node in cluster
Ceph
→ OSD
Create: OSD
Create
If there is no unused
disk to choose, erase content of disk with command:
ceph-volume lvm zap /dev/... --destroy
systemctl stop ceph\*.service ceph\*.target systemctl start ceph.target
Size
- number of replicas for poolMin. Size
- Crush Rule
- only possible to choose 'replicated_rule
*
pg_num (Placement Groups) use Ceph PGs per Pool Calculator to calculate
pg_num
* NOTE: It's also important to know that the PG count can be increased, but NEVER decreased without destroying / recreating the pool. However, increasing the PG Count of a pool is one of the most impactful events in a Ceph Cluster, and should be avoided for production clusters if possible.
* Placement Groups
*
Add as Storage'' - automatically create Proxmox RBD storage (Disc Image, Container)Benchmarks for pool name 'rbd' and 10 seconds duration
# Write benchmark rados -p rbd bench 10 write --no-cleanup # Read performance rados -p rbd bench 10 seq