====== Proxmox ====== * [[https://pve.proxmox.com/pve-docs/pve-admin-guide.html|Proxmox VE Administration Guide]] * [[https://pve.proxmox.com/wiki/Main_Page|Wiki]] * [[https://www.proxmox.com/en/training/video-tutorials|Video Tutorials]] * [[https://community-scripts.github.io/ProxmoxVE/|Proxmox VE Helper-SCripts]] ===== Upgrading ===== Always upgrade with ''dist-upgrade'' ! apt update && apt dist-upgrade ===== Installation ===== * BIOS: * Enable VT * Enable IOMMU * Choose ZFS * Leave some free space on HDD for swap partition * Click "Advanced Options" and manually set "hdsize" * Installer 6.1-2 BUG: On system with 2x 2TB HDDs, Proxmox installer on ZFS RAID1 propose only ''hdsize=953MB''. This issue gone with Proxmox 6.2. * Do not create swap on ZFS zvol! * reboot * manually configure VLAN network from console [[https://pve.proxmox.com/wiki/Network_Configuration#_vlan_802_1q|VLAN 802.1Q]] * check if it is still necessary. Perhaps enabling ''VLAN aware birdge'' from GUI is enough * https://ip_address:8006 ==== issues ==== === initramfs === Proxmox installed on ZFS RADI1. After reboot, Proxmox boots into initramfs. From initramfs console: zfs list Try running '/sbin/modprobe zfs' as root to load them. REASON: no root cmdline, because UEFI boot was used instead of grub loader. SOLUTION: force legacy boot in BIOS OR TRY: echo -n " rootdelay=5" >> /etc/kernel/cmdline pve-efiboot-tool refresh === initramfs === REASON: ZFS invoked too soon: [[https://pve.proxmox.com/wiki/ZFS:_Tips_and_Tricks#Boot_fails_and_goes_into_busybox]] modprobe zfs zpool import -N -f rpool exit ==== post-installation tips ==== * set MAC address prefix: `Datacenter` -> `Options`. Locally Administered Address Ranges are: x2-xx-xx-xx-xx-xx x6-xx-xx-xx-xx-xx xA-xx-xx-xx-xx-xx xE-xx-xx-xx-xx-xx * Speedup migrations to set unencrypted network traffic (in secured networks!): migration: network=first_cluster_host_ip/22,type=insecure * disable subscription repo:\\ sed -i.bak 's|deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise|\#deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise|' /etc/apt/sources.list.d/pve-enterprise.list * add no subscription repo: echo 'deb http://download.proxmox.com/debian/pve buster pve-no-subscription' > /etc/apt/sources.list.d/pve-no-sub.list * apt-get update * apt-get upgrade * apt-get install dropbear busybox-static --no-install-recommends * Update grub config: GRUB_CMDLINE_LINUX_DEFAULT="rootdelay=15 mitigations=off panic=30 nomodeset textonly video=vesafb:off video=efifb:off" update-grub * SMART disk monitoring\\ # Short 1AM Long Sat 2AM DEVICESCAN -a -o on -S on -s (S/../.././01|L/../../6/02) -m root systemctl restart smartmontools.service * More here: [[https://pve.proxmox.com/wiki/Disk_Health_Email_Alerts|Disk Health Email Alerts]] * Activate ZFS e-mail notification [[https://pve.proxmox.com/pve-docs/pve-admin-guide.html#chapter_zfs]] * systemctl enable --now zfs-zed * Use own internal CA signed cert for Web UI: [[https://pve.proxmox.com/pve-docs/pve-admin-guide.html#sysadmin_certificate_management|Certificate Management]] * For clustering Proxmox is using own self signed certs! Do not replace them! * Postfix relay:apt-get install postfix bsd-mailx wget https://raw.githubusercontent.com/Tontonjo/proxmox/master/ez_proxmox_mail_configurator.sh bash ez_proxmox_mail_configurator.sh * Assign admin permissions: [[https://pve.proxmox.com/wiki/User_Management#_real_world_examples]] * For HDDs storage, limit Proxmox operation speed to 50000KiB: pvesm set local --bwlimit default=50000 * to remove limit: pvesm set local --delete bwlimit * Enable IOMMU * in `GRUB_CMDLINE_LINUX_DEFAULT` and add kernel parameter `intel_iommu=on` or `amd_iommu=on` * vfio vfio_iommu_type1 vfio_pci vfio_virqfd