GoWiki
Index
Sections
Search
Login
wiki
Proxmox VE
# Proxmox Documentation ## Hetzner Online https://wiki.hetzner.de/index.php/Proxmox_VE Replace the package mirror: ``` cat <<EOF> /etc/apt/sources.list #deb http://mirror.hetzner.de/debian/packages stretch main contrib non-free deb http://http.debian.net/debian stretch main contrib non-free deb http://http.debian.net/debian/ stretch-updates main contrib non-free deb http://security.debian.org/ stretch/updates main contrib non-free EOF cat <<EOF> /etc/apt/sources.list.d/proxmox.list #deb http://mirror.hetzner.de/debian/pve stretch pve-no-subscription deb http://download.proxmox.com/debian stretch pve-no-subscription EOF ``` ## SSD RAID1 ``` mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/sda /dev/sdb vgcreate vgssd /dev/md2 cat /proc/mdstat mdadm --readwrite /dev/md2 ``` ## ISO Path ISO Images can be placed here: `/var/lib/vz/template/iso` ## Fail2Ban ### Install `apt install fail2ban` ### Config ``` cat <<EOF> /etc/fail2ban/jail.local [proxmox] enabled = true port = 8006 filter = proxmox logpath = /var/log/daemon.log maxretry = 10 bantime = 3600 EOF ``` ``` cat <<EOF> /etc/fail2ban/filter.d/proxmox.conf [Definition] failregex = pvedaemon\[.*authentication failure; rhost=<HOST> user=.* msg=.* ignoreregex = EOF ``` Config-Test: `fail2ban-regex /var/log/daemon.log /etc/fail2ban/filter.d/proxmox.conf` `systemctl restart fail2ban` ## TLS The Proxmox VE API proxy `pveproxy` listens on port 8006. It can be secured by creating the `/etc/default/pveproxy` file. Here is a config sample: ``` CIPHERS="ECDHE-RSA-CHACHA20-POLY1305:ECDHE+AESGCM:ECDHE+AES256:!SHA" ``` Lets Encrypt Certificates can be issued from the webconsole. ## Firewall ``` cat <<EOF> /etc/rc.local #!/bin/sh -e iptables -F ip6tables -F iptables -A INPUT -i lo -j ACCEPT iptables -A INPUT -p udp --sport 53 -m state --state ESTABLISHED -j ACCEPT iptables -A INPUT -p tcp --sport 53 -m state --state ESTABLISHED -j ACCEPT iptables -A INPUT -p udp --sport 123 -m state --state ESTABLISHED -j ACCEPT iptables -A INPUT -p tcp -m multiport --sports 20,21,22,80,443 -m state --state ESTABLISHED -j ACCEPT iptables -A INPUT -p icmp -m state --state ESTABLISHED,RELATED -j ACCEPT ip6tables -A INPUT -i lo -j ACCEPT ip6tables -A INPUT -p udp --sport 53 -m state --state ESTABLISHED -j ACCEPT ip6tables -A INPUT -p tcp --sport 53 -m state --state ESTABLISHED -j ACCEPT ip6tables -A INPUT -p udp --sport 123 -m state --state ESTABLISHED -j ACCEPT ip6tables -A INPUT -p tcp -m multiport --sports 20,21,22,80,443 -m state --state ESTABLISHED -j ACCEPT ip6tables -A INPUT -p icmpv6 -m state --state ESTABLISHED,RELATED -j ACCEPT iptables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT iptables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT iptables -A INPUT -p tcp -m tcp --dport 8006 -j ACCEPT ip6tables -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT ip6tables -A INPUT -p tcp -m tcp --dport 80 -j ACCEPT ip6tables -A INPUT -p tcp -m tcp --dport 8006 -j ACCEPT iptables -P INPUT DROP iptables -P FORWARD ACCEPT iptables -P OUTPUT ACCEPT ip6tables -P INPUT DROP ip6tables -P FORWARD ACCEPT ip6tables -P OUTPUT ACCEPT exit 0 EOF ``` ## pfsense When running a pfsense firewall on proxmox with the VirtIO network interfaces you need to disable `Hardware Checksum Offloading` under the `Advanced -> Networking` tab. https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=165059 ## Import Disk Image Store the disk image in the right location: `/var/lib/vz/<pool>/images/<vmid>` Rescan the image storage: `qm rescan -vmid <vmid>` ## Cloud Init Create template: ``` curl -s https://cloud-images.ubuntu.com/minimal/releases/bionic/release/ubuntu-18.04-minimal-cloudimg-amd64.img >/tmp/ubuntu-18.04-minimal-cloudimg-amd64.img qm create 9000 --memory 2048 --net0 virtio,bridge=vmbr0 qm importdisk 9000 /tmp/ubuntu-18.04-minimal-cloudimg-amd64.img local rm /tmp/ubuntu-18.04-minimal-cloudimg-amd64.img qm set 9000 --scsihw virtio-scsi-pci --scsi0 local:9000/vm-9000-disk-0.raw qm set 9000 --ide2 local:cloudinit qm set 9000 --boot c --bootdisk scsi0 qm set 9000 --serial0 socket --vga serial0 qm template 9000 ``` ``` qm clone 9000 400 --full --name ci-test qm resize 400 scsi0 30G qm set 400 --sshkey ~/.ssh/id_rsa.pub qm set 400 --ipconfig0 ip=192.168.0.2/24,gw=192.168.0.1 ``` ### Config Merger The config merger merges proxmox UI and default configs with custom user supplied configs and injects them into the cloud init image. Install the yq dependency: ``` curl -s -L https://github.com/mikefarah/yq/releases/download/3.3.0/yq_linux_amd64 >/usr/bin/yq [[ "$(sha256sum /usr/bin/yq|cut -d" " -f1)" == "e70e482e7ddb9cf83b52f5e83b694a19e3aaf36acf6b82512cbe66e41d569201" ]] && chmod 755 /usr/bin/yq || { rm /usr/bin/yq; echo "checksum validation failed"; } ``` Cloud Init user config merger hook script: **Note:** change the `cfg` variable to the directory where you will store your custom configs. ``` cat <<'EOF'> /var/lib/vz/snippets/merge-hook.sh #!/bin/bash vmid=$1 phase=$2 cfg="k8s" snip="/var/lib/vz/snippets" local="local:snippets" reload=0 [[ ! -d "${snip}" ]] && mkdir -p ${snip} merge() { file="${snip}/${vmid}_$1" cfgf="${snip}/${cfg}/$1" [[ ! -f "${cfgf}" ]] && touch ${cfgf} qm cloudinit dump ${vmid} $1 | yq m - ${cfgf} >${file} sum="$(md5sum ${file}|cut -d" " -f1)" [[ "${sum}" != "$(cat ${file}.sum 2>/dev/null)" ]] && { echo -n "${sum}" >${file}.sum; return 1; } || return 0 } if [[ "${phase}" == "pre-start" ]]; then merge meta || reload=1 merge network || reload=1 merge user || reload=1 if [[ $reload -eq 1 ]]; then custom="meta=${local}/${vmid}_meta,network=${local}/${vmid}_network,user=${local}/${vmid}_user" nohup bash -c "sleep 5;qm set ${vmid} --cicustom '${custom}';qm start ${vmid}" </dev/null >/dev/null 2>&1 & echo "Restarting VM ${vmid} in 5 seconds.." exit 1 fi fi EOF chmod 755 /var/lib/vz/snippets/merge-hook.sh ``` There are three possible config files: * meta * network * user Create custom cloud init config: ``` mkdir /var/lib/vz/snippets/k8s cat <<EOF> /var/lib/vz/snippets/k8s/network config: - name: eth0 mtu: 1500 - name: eth1 mtu: 9000 EOF cat <<EOF> /var/lib/vz/snippets/k8s/user runcmd: - sysctl net.bridge.bridge-nf-call-ip6tables=1 - sysctl net.bridge.bridge-nf-call-iptables=1 - sysctl net.ipv4.ip_forward=1 - sysctl vm.overcommit_memory=1 - sysctl kernel.panic=10 - sysctl kernel.panic_on_oops=1 EOF ``` Add hook script to VM: ``` qm set <vmid> --hookscript /var/lib/vz/snippets/merge-hook.sh ``` ## Remove No Subscription Message ``` t=$'\t' patch -d/ -p0 <<EOF --- /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js${t}2021-06-12 13:26:02.130527196 +0200 +++ /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js${t}2021-06-12 13:27:05.530087709 +0200 @@ -453,6 +453,7 @@ ${t}${t} Ext.Msg.alert(gettext('Error'), response.htmlStatus); ${t}${t}}, ${t}${t}success: function(response, opts) { +return; ${t}${t} let res = response.result; ${t}${t} if (res === null || res === undefined || !res || res ${t}${t}${t}.data.status.toLowerCase() !== 'active') { EOF ```