Proxmox

• Proxmox VE Administration Guide
• Wiki
• Video Tutorials

Always upgrade with dist-upgrade !

apt update && apt dist-upgrade

• BIOS:
  • Enable VT
  • Enable IOMMU
• Choose ZFS
  • Leave some free space on HDD for swap partition
    • Click “Advanced Options” and manually set “hdsize”
    • Installer 6.1-2 BUG: On system with 2x 2TB HDDs, Proxmox installer on ZFS RAID1 propose only hdsize=953MB. This issue gone with Proxmox 6.2.
  • Do not create swap on ZFS zvol!
• reboot
• manually configure VLAN network from console VLAN 802.1Q
  • check if it is still necessary. Perhaps enabling VLAN aware birdge from GUI is enough
• https://ip_address:8006

Proxmox installed on ZFS RADI1. After reboot, Proxmox boots into initramfs. From initramfs console:

zfs list
Try running '/sbin/modprobe zfs' as root to load them.

REASON: no root cmdline, because UEFI boot was used instead of grub loader. SOLUTION: force legacy boot in BIOS OR TRY:

echo -n " rootdelay=5" >> /etc/kernel/cmdline
pve-efiboot-tool refresh

REASON: ZFS invoked too soon: https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks#Boot_fails_and_goes_into_busybox

modprobe zfs
zpool import -N -f rpool
exit

• set MAC address prefix: `Datacenter` → `Options`. Locally Administered Address Ranges are:

  x2-xx-xx-xx-xx-xx
  x6-xx-xx-xx-xx-xx
  xA-xx-xx-xx-xx-xx
  xE-xx-xx-xx-xx-xx

• Speedup migrations to set unencrypted network traffic (in secured networks!):

  migration: network=first_cluster_host_ip/22,type=insecure

• disable subscription repo:
  

  sed -i.bak 's|deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise|\#deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise|' /etc/apt/sources.list.d/pve-enterprise.list

• add no subscription repo:

  echo 'deb http://download.proxmox.com/debian/pve buster pve-no-subscription' > /etc/apt/sources.list.d/pve-no-sub.list

• apt-get update

• apt-get upgrade

• apt-get install dropbear busybox-static --no-install-recommends

• Update grub config:

  /etc/default/grub

  GRUB_CMDLINE_LINUX_DEFAULT="rootdelay=15 mitigations=off panic=30 nomodeset textonly video=vesafb:off video=efifb:off"

  update-grub

• SMART disk monitoring
  

  /etc/smartd.conf

  #                             Short   1AM  Long  Sat    2AM
  DEVICESCAN -a -o on -S on -s (S/../.././01|L/../../6/02) -m root

  systemctl restart smartmontools.service

  • More here: Disk Health Email Alerts
• Activate ZFS e-mail notification https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs

  • systemctl enable --now zfs-zed

• Use own internal CA signed cert for Web UI: Certificate Management
  • For clustering Proxmox is using own self signed certs! Do not replace them!
• Postfix relay:

  apt-get install postfix bsd-mailx
  wget https://raw.githubusercontent.com/Tontonjo/proxmox/master/ez_proxmox_mail_configurator.sh
  bash ez_proxmox_mail_configurator.sh

• Assign admin permissions: https://pve.proxmox.com/wiki/User_Management#_real_world_examples
• For HDDs storage, limit Proxmox operation speed to 50000KiB:

  pvesm set local --bwlimit default=50000

  • to remove limit:

    pvesm set local --delete bwlimit

• Enable IOMMU
  • in `GRUB_CMDLINE_LINUX_DEFAULT` and add kernel parameter `intel_iommu=on` or `amd_iommu=on`

  • vfio
    vfio_iommu_type1
    vfio_pci
    vfio_virqfd