====== Move DB to bigger storage ======
I.e: to move from 4GB partition to 30GB partition in case of spillover:
1 OSD(s) experiencing BlueFS spillover
osd.3 spilled over 985 MiB metadata from 'db' device (3.8 GiB used of 4.0 GiB) to slow device
lvdisplay
--- Logical volume ---
LV Path /dev/ceph-66ad9fa6-0f65-4fc0-baeb-41089a509347/osd-block-4a9e142e-ed4d-4890-a018-5651c36071a5
LV Name osd-block-4a9e142e-ed4d-4890-a018-5651c36071a5
--- Logical volume ---
LV Path /dev/ceph-ffd9911c-2d6c-43aa-b637-bec8a2092a30/osd-db-ad0a2108-bcca-4f15-9747-29d86624b911
LV Name osd-db-ad0a2108-bcca-4f15-9747-29d86624b911
VG Name ceph-ffd9911c-2d6c-43aa-b637-bec8a2092a30
LV Size 4,00 GiB
Current LE 1024
vgdisplay
--- Volume group ---
VG Name ceph-ffd9911c-2d6c-43aa-b637-bec8a2092a30
VG Size 4,00 GiB
PE Size 4,00 MiB
Total PE 1025
Alloc PE / Size 1024 / 4,00 GiB
Free PE / Size 1 / 4,00 MiB
pvdisplay
--- Physical volume ---
PV Name /dev/nvme0n1p5
VG Name ceph-ffd9911c-2d6c-43aa-b637-bec8a2092a30
PV Size <4,01 GiB / not usable 2,00 MiB
Allocatable yes
PE Size 4,00 MiB
Total PE 1025
Free PE 1
Allocated PE 1024
PV UUID pjYGgY-SS26-m8Sc-aWyQ-3zzY-f7Iz-XGoI90
===== resize exsiting DB partition =====
Remove separate DB and migrate data back to main storage:
ceph osd set noout
systemctl stop ceph-osd@3.service
cat /var/lib/ceph/osd/ceph-3/fsid
lvdisplay
ceph-volume lvm migrate --osd-id 3 --osd-fsid 024c05b3-6e22-4df1-b0af-5cb46725c5c8 --from db --target ceph-5582d170-f77e-495c-93c6-791d9310872c/osd-block-024c05b3-6e22-4df1-b0af-5cb46725c5c8
--> Migrate to existing, Source: ['--devs-source', '/var/lib/ceph/osd/ceph-0/block.db'] Target: /var/lib/ceph/osd/ceph-0/block
--> Migration successful.
Create new PV,VG,LV for DB and attach it:
pvcreate /dev/nvme0n1p6
vgcreate ceph-db-8gb /dev/nvme0n1p6
lvcreate -n db-8gb -l 100%FREE ceph-db-8gb
ceph-volume lvm new-db --osd-id 0 --osd-fsid 024c05b3-6e22-4df1-b0af-5cb46725c5c8 --target ceph-db-8gb/db-8gb
--> Making new volume at /dev/ceph-db-8gb/db-8gb for OSD: 0 (/var/lib/ceph/osd/ceph-0)
Running command: /usr/bin/chown -h ceph:ceph /var/lib/ceph/osd/ceph-0/block.db
Running command: /usr/bin/chown -R ceph:ceph /dev/dm-0
--> New volume attached.
ceph-volume lvm migrate --osd-id 0 --osd-fsid 024c05b3-6e22-4df1-b0af-5cb46725c5c8 --from data db --target ceph-db-8gb/db-8gb
--> Migrate to existing, Source: ['--devs-source', '/var/lib/ceph/osd/ceph-0/block'] Target: /var/lib/ceph/osd/ceph-0/block.db
--> Migration successful.
systemctl start ceph-osd@3.service
ceph osd unset noout
===== OR: create new one =====
Create new 41GB (41984MB) partition (/dev/nvme0n1p6)
pvcreate /dev/nvme0n1p6
vgcreate ceph-db-40gb /dev/nvme0n1p6
lvcreate -n db-40gb -l 100%FREE ceph-db-40gb
ceph osd set noout
systemctl stop ceph-osd@3.service
cat /var/lib/ceph/osd/ceph-3/fsid
ceph-volume lvm migrate --osd-id 3 --osd-fsid 4a9e142e-ed4d-4890-a018-5651c36071a5 --from db --target ceph-db-40gb/db-40gb
systemctl start ceph-osd@3.service
ceph osd unset noout
====== Move DB disks ======
In case of replace (full copy by ''dd'') NVM drive where partition for OSD's DB is located, OSD service cannot start.
Reason is that CEPH doesn't have permission to access device owned by ''root''. To repair everything with one command:
ceph-volume lvm activate --all
====== draft ======
$ systemctl stop ceph-osd@68
$ ceph-bluestore-tool --path /var/lib/ceph/osd/ceph-68 --devs-source
/var/lib/ceph/osd/ceph-68/block --dev-target
/var/lib/ceph/osd/ceph-68/block.db bluefs-bdev-migrate
$ systemctl start ceph-osd@68