prodik
/
proxmox-upgrade
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

fix

main
Дмитрий Фёдоров 2024-05-28 18:05:13 +03:00
parent 67be0c35ea
commit 656d1e1f69
19 changed files with 786 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
inv.yml Executable file
View File

@ -0,0 +1,8 @@
all:
hosts:
proxmox:
ansible_host: 192.168.88.56
# sbk-proxmox:
# ansible_host: 172.30.30.24

8
inv2.yml Executable file
View File

@ -0,0 +1,8 @@
all:
hosts:
proxmox:
ansible_host: 192.168.88.57
# sbk-proxmox:
# ansible_host: 172.30.30.24

7
proxmox-upgrade.yml Executable file
View File

@ -0,0 +1,7 @@
---
- name: Upgrade proxmox server
become: true
hosts: proxmox
roles:
- roles/proxmox-upgrade
...

5
roles/proxmox-upgrade/.ansible-lint Executable file
View File

@ -0,0 +1,5 @@
skip_list:
- yaml
- role-name
- command-instead-of-shell
- package-latest

21
roles/proxmox-upgrade/LICENSE Executable file
View File

@ -0,0 +1,21 @@
MIT License
Copyright (c) 2023 Marvin Stark
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

64
roles/proxmox-upgrade/README.md Executable file
View File

@ -0,0 +1,64 @@
# proxmox-upgrade
This playbook automates the upgrade process for [Proxmox Virtual Environment](https://www.proxmox.com/en/proxmox-ve) (PVE) and [Proxmox Backup Server](https://www.proxmox.com/en/proxmox-backup-server) (PBS).
Currently following upgrade paths are supported:
* Proxmox Virtual Environment version `7->8`
* Proxmox Backup Server version `2->3`
This playbooks follows the instructions from the offical documentation:
* [Proxmox Virtual Environment Upgrade from 7 to 8](https://pve.proxmox.com/wiki/Upgrade_from_7_to_8)
* [Proxmox Backup Server Upgrade from 2 to 3](https://pbs.proxmox.com/wiki/index.php/Upgrade_from_2_to_3)
This is a work in progress, if you find any bugs or problems please open a new issue.
## details of PVE upgrade:
* check prerequisites
* create backup of important configuration files under `/var/backups`
* install latest packages from current version
* shutdown virtual maschines
* Debian dist-upgrade
* reboot
I highly recommend running the `pve7to8 --full` checklist tool first and fix all detected problems before upgrading.
You should also use it after the upgrade to verify that everything is ok.
## details of PBS upgrade:
* check prerequisites
* create backup of important configuration files under `/var/backups`
* install latest packages from current version
* put datastores in read-only mode
* Debian dist-upgrade
* reboot
* put datastores back in write mode
* check needed services
## playbook usage
1. install this playbook via Ansible Galaxy
```
ansible-galaxy install marvhh.proxmox_upgrade
```
2. create a playbook yaml file
example:
```yaml
- name: Upgrade proxmox server
become: true
hosts: proxmox_server
roles:
- marvhh.proxmox_upgrade
```
4. run ansible-playbook
## Options
Following [options](defaults/main.yml) are currently supported:
* `reboot`: Choose if the reboot of the maschine should be done automatically
* `shutdown_vms`: shutdown virtual maschines automatically
* `use_enterprise_repos`: should the [Proxmox enterprise repositories](https://pve.proxmox.com/wiki/Package_Repositories) be used?
## Author
[Marvin Stark](https://github.com/marvhh)

39
roles/proxmox-upgrade/defaults/main.yml Executable file
View File

@ -0,0 +1,39 @@
---
# should the server be rebooted automatically (boolean)
reboot: true
# should the server be rebooted automatically (boolean)
shutdown_vms: true
# should the enterpise repos be used? (boolean)
use_enterprise_repos: false
# download created backups? (boolean)
download_backups: true
# where to save backups?
backup_path: "/var/backups"
# proxmox pathes
proxmox_pveversion: "/usr/bin/pveversion"
proxmox_backup_manager: "/usr/sbin/proxmox-backup-manager"
proxmox_qm_manager: "/usr/sbin/qm"
proxmox_pve_min_version: "7.4-15"
proxmox_pbs_min_version: "2.4.2"
proxmox_get_vms_cmd: "qm list |grep running |awk '{print $1}'"
proxmox_shutdown_vms_opts: "--forceStop 1"
proxmox_pve_enterprise_list: "/etc/apt/sources.list.d/pve-enterprise.list"
proxmox_pbs_enterprise_list: "/etc/apt/sources.list.d/pbs-enterprise.list"
# proxmox services
proxmox_backup_services:
- proxmox-backup-proxy.service
- proxmox-backup.service
# define proxmox newest version
proxmox_newest_deb_relase: "bookworm"
# identify roles
# do not change this!
is_pve_role: false
is_pbs_role: false

12
roles/proxmox-upgrade/handlers/main.yml Executable file
View File

@ -0,0 +1,12 @@
---
# reboot server
- name: reboot server
ansible.builtin.reboot:
when: reboot
# check if needed services are running
- name: check proxmox-backup-services
ansible.builtin.service:
name: "{{ item }}"
state: started
loop: "{{ proxmox_backup_services }}"

16
roles/proxmox-upgrade/meta/main.yml Executable file
View File

@ -0,0 +1,16 @@
galaxy_info:
author: Marvin Stark
description: automate Proxmox upgrade process
license: MIT
min_ansible_version: 2.14
platforms:
- name: Debian
versions:
- bullseye
- buster
galaxy_tags:
- proxmox
- debian
- distupgrade
- role

12
roles/proxmox-upgrade/tasks/check-new-configs.yml Executable file
View File

@ -0,0 +1,12 @@
---
- name: find config files changed by package maintainer
ansible.builtin.find:
path: /etc
patterns: '*.dpkg-dist'
recurse: yes
register: dpkg_dist_files
- name: please check following new configuration files and apply your changes, files could be deleted after that.
ansible.builtin.debug:
msg: "{{ dpkg_dist_files.files | map(attribute='path') }}"
when: dpkg_dist_files.matched > 0

22
roles/proxmox-upgrade/tasks/fetch-backups.yml Executable file
View File

@ -0,0 +1,22 @@
---
- name: collect files to backup
ansible.builtin.find:
paths: "{{ backup_path }}"
patterns: 'proxmox-*-backup-*.tar.gz'
register: backup_files_info
tags:
- backup
- name: define backup files
ansible.builtin.set_fact:
backup_files: "{{ backup_files_info.files | selectattr('path', 'defined') | map(attribute='path') }}"
tags:
- backup
- name: download backups from server
ansible.builtin.fetch:
src: "{{ item }}"
dest: "{{ role_path }}/backups/{{ inventory_hostname }}"
loop: "{{ backup_files }}"
tags:
- backup

49
roles/proxmox-upgrade/tasks/main.yml Executable file
View File

@ -0,0 +1,49 @@
---
- name: stop when system is already on newest version
ansible.builtin.fail:
msg: "Server is already on newest version, stopping here!"
when: ansible_distribution_release == proxmox_newest_deb_relase
- name: include OS specific variables
ansible.builtin.include_vars: "{{ lookup('ansible.builtin.first_found', params) }}"
vars:
params:
files:
- "{{ ansible_distribution }}-{{ ansible_distribution_major_version }}.yml"
paths:
- "vars"
tags:
- always
- name: gather facts and check prerequisites
ansible.builtin.include_tasks:
file: prepare.yml
tags:
- always
- name: perform tasks for Proxmox Backup Server
ansible.builtin.include_tasks:
file: upgrade-pbs.yml
when: is_pbs_role
tags:
- backup
- name: perform tasks for Proxmox Virtual Environment Server
ansible.builtin.include_tasks:
file: upgrade-pve.yml
when: is_pve_role
tags:
- backup
- name: Download backup files
ansible.builtin.include_tasks:
file: fetch-backups.yml
when: download_backups
tags:
- backup
- name: check configuration file changes
ansible.builtin.include_tasks:
file: check-new-configs.yml
tags:
- always

39
roles/proxmox-upgrade/tasks/prepare.yml Executable file
View File

@ -0,0 +1,39 @@
---
# get basic info and set facts
- name: check if proxmox pve role is installed
ansible.builtin.stat:
path: "{{ proxmox_pveversion }}"
register: pve_version_info
tags:
- backup
- name: check if proxmox-backup-manager role is installed
ansible.builtin.stat:
path: "{{ proxmox_backup_manager }}"
register: pbs_manager_info
tags:
- backup
- name: define server as pve role
ansible.builtin.set_fact:
is_pve_role: true
when: pve_version_info.stat.exists
tags:
- backup
- name: define as pbs role
ansible.builtin.set_fact:
is_pbs_role: true
when: pbs_manager_info.stat.exists
tags:
- backup
# check that there is enough free disc space
- name: define free disc space
set_fact:
space_free_gb: "{{ (((ansible_mounts| selectattr('mount', 'equalto', '/')| list)[0].size_available)/1024/1024/1024) | round | int }}"
- name: check that there is enough free space
ansible.builtin.fail:
msg: Stopping because there is not enough free space! There should be at least 5GB free of space.
when: space_free_gb > '5'

50
roles/proxmox-upgrade/tasks/update-repos.yml Executable file
View File

@ -0,0 +1,50 @@
---
# update debian repos
- name: Set new debian sources
ansible.builtin.apt_repository:
repo: "{{ item.line }}"
state: "{{ item.state }}"
update_cache: false
loop: "{{ __deb_debian_repos }}"
- name: Install GPG key
shell: wget https://enterprise.proxmox.com/debian/proxmox-release-bookworm.gpg -O /etc/apt/trusted.gpg.d/proxmox-release-bookworm.gpg
when: ansible_distribution_release == "bookworm"
# update proxmox pve repos
- name: update proxmox pve repos
block:
- name: set new proxmov pve no-subscription repos
ansible.builtin.apt_repository:
repo: "{{ item.line }}"
state: "{{ item.state }}"
update_cache: false
loop: "{{ __deb_proxmox_pve_repos }}"
- name: set new proxmov pve subscription repos
ansible.builtin.apt_repository:
repo: "{{ item.line }}"
state: "{{ item.state }}"
update_cache: false
loop: "{{ __deb_proxmox_pve_ent_repos }}"
when: use_enterprise_repos
when: is_pve_role
# update proxmox pbs repos
- name: update proxmox pbs repos
block:
- name: set new proxmox pbs no-subscription repos
ansible.builtin.apt_repository:
repo: "{{ item.line }}"
state: "{{ item.state }}"
update_cache: false
loop: "{{ __deb_proxmox_pbs_repos }}"
- name: set new proxmox pbs subscription repos
ansible.builtin.apt_repository:
repo: "{{ item.line }}"
state: "{{ item.state }}"
update_cache: false
loop: "{{ __deb_proxmox_pbs_ent_repos }}"
when: use_enterprise_repos
when: is_pbs_role

91
roles/proxmox-upgrade/tasks/upgrade-pbs.yml Executable file
View File

@ -0,0 +1,91 @@
---
- name: Perform tasks for Proxmox Backup Server
block:
- name: Remove pbs-enterprise subscription repo
ansible.builtin.file:
path: "{{ proxmox_pbs_enterprise_list }}"
state: absent
when: use_enterprise_repos == false
- name: Make sure that pbs-no-subscription repos are active
ansible.builtin.apt_repository:
repo: "{{ item.line }}"
state: present
update_cache: false
loop: "{{ __deb_proxmox_pbs_repos | selectattr('release', '==', ansible_distribution_release) }}"
- name: Update packages to make sure we are at version "{{ proxmox_pbs_min_version }}" or higher
ansible.builtin.apt:
name: "*"
state: latest
update_cache: yes
notify:
- reboot server
- name: flush handlers to reboot instantly
ansible.builtin.meta: flush_handlers
- name: get Proxmox Backup Server version
ansible.builtin.command:
cmd: proxmox-backup-manager versions
register: proxmox_pbs_version_info
changed_when: false
- name: define Proxmox Backup Server version
ansible.builtin.set_fact:
proxmox_pbs_version: "{{ proxmox_pbs_version_info.stdout | regex_search('running version\\:\\s(\\d+\\.\\d+.\\d+)') | split | last }}"
- name: fail when Proxmox Backup Server version is not met
ansible.builtin.fail:
msg: "Proxmox Backup Server version too low, at least {{ proxmox_pbs_min_version }} is needed."
when: proxmox_pbs_version <= proxmox_pbs_min_version
# backup configuration
- name: create backup of configuration
community.general.archive:
path: /etc/proxmox-backup
dest: "{{ backup_path }}proxmox-pbs-backup-{{ ansible_date_time.date }}.tar.gz"
mode: 600
format: gz
tags:
- backup
# put datastores in maintenance mode
- name: get datastore infos
ansible.builtin.command:
cmd: proxmox-backup-manager datastore list --output-format json
register: datastores_info
changed_when: false
- name: define datastores
ansible.builtin.set_fact:
datastores: "{{ datastores_info.stdout | from_json }}"
- name: put datastores in readonly mode
ansible.builtin.command:
cmd: proxmox-backup-manager datastore update "{{ item |json_query('name') }}" --maintenance-mode read-only
loop: "{{ datastores }}"
register: maintenance_output
changed_when: maintenance_output.rc != 0
# update repos
- name: update repos
ansible.builtin.include_tasks:
file: update-repos.yml
# do dist-upgrade
- name: upgrade Proxmox Backup Server
ansible.builtin.apt:
update_cache: yes
upgrade: dist
# reactivate datastores
- name: put datastores back in write mode
ansible.builtin.command:
cmd: proxmox-backup-manager datastore update "{{ item |json_query('name') }}" --delete maintenance-mode
loop: "{{ datastores }}"
register: maintenance_output
changed_when: maintenance_output.rc != 0
notify:
- reboot server
- check proxmox-backup-services

93
roles/proxmox-upgrade/tasks/upgrade-pve-Debian10.yml Executable file
View File

@ -0,0 +1,93 @@
---
- name: remove pve-enterprise subscription repo
ansible.builtin.file:
path: "{{ proxmox_pve_enterprise_list }}"
state: absent
when: use_enterprise_repos == false
- name: make sure that pve-no-subscription repos are active
ansible.builtin.apt_repository:
repo: "{{ item.line }}"
state: present
update_cache: false
loop: "{{ __deb_proxmox_pve_repos | selectattr('release', '==', ansible_distribution_release) }}"
- name: Update packages on Debian 10
ansible.builtin.apt:
name: "*"
state: latest
update_cache: true
notify:
- reboot server
- name: flush handlers to reboot instantly
ansible.builtin.meta: flush_handlers
- name: create backup of configuration
community.general.archive:
path: /etc
dest: "{{ backup_path }}proxmox-pve-backup-{{ ansible_date_time.date }}.tar.gz"
mode: 600
format: gz
tags:
- backup
- name: get running virtual maschines
ansible.builtin.shell:
cmd: "{{ proxmox_get_vms_cmd }}"
register: proxmox_running_vms_info
changed_when: false
- name: shutting down virtual maschines
ansible.builtin.command:
cmd: "{{ proxmox_qm_manager }} shutdown {{ item }} {{ proxmox_shutdown_vms_opts }}"
loop: "{{ proxmox_running_vms_info.stdout_lines }}"
when: shutdown_vms
- name: update repos
ansible.builtin.include_tasks:
file: update-repos.yml
- name: Update packages to Debian 11
ansible.builtin.apt:
name: "*"
state: latest
update_cache: true
notify:
- reboot server
- name: flush handlers to reboot instantly
ansible.builtin.meta: flush_handlers
# - name: Exit if nothing to upgrade
# fail: msg="Only manual upgrade!"
- name: Upgrade Proxmox Virtual Environment Server on Debian 10
ansible.builtin.apt:
update_cache: true
upgrade: dist
force: true
notify:
- reboot server
- name: Remove useless packages from the cache
ansible.builtin.apt:
autoclean: true
- name: Remove dependencies that are no longer required and purge their configuration files
ansible.builtin.apt:
autoremove: true
purge: true
- name: flush handlers to reboot instantly
ansible.builtin.meta: flush_handlers
- name: check if proxmox pve role is installed
ansible.builtin.stat:
path: "{{ proxmox_pveversion }}"
register: pve_version_info
tags:
- backup
notify:
- reboot server

84
roles/proxmox-upgrade/tasks/upgrade-pve.yml Executable file
View File

@ -0,0 +1,84 @@
---
- name: perform tasks for Proxmox Virtual Environment
block:
- name: Run upgrade Debian 10
ansible.builtin.include_tasks:
file: upgrade-pve-Debian10.yml
when: ansible_distribution_release == "buster"
- name: remove pve-enterprise subscription repo
ansible.builtin.file:
path: "{{ proxmox_pve_enterprise_list }}"
state: absent
when: use_enterprise_repos == false
- name: make sure that pve-no-subscription repos are active
ansible.builtin.apt_repository:
repo: "{{ item.line }}"
state: present
update_cache: false
loop: "{{ __deb_proxmox_pve_repos | selectattr('release', '==', ansible_distribution_release) }}"
- name: update packages to make sure we are at version "{{ proxmox_pve_min_version }}" or higher
ansible.builtin.apt:
name: "*"
state: latest
update_cache: true
notify:
- reboot server
- name: flush handlers to reboot instantly
ansible.builtin.meta: flush_handlers
- name: get Proxmox Virtual Environment version
ansible.builtin.command:
cmd: "{{ proxmox_pveversion }}"
register: proxmox_pveversion_info
changed_when: false
- name: define Proxmox Virtual Environment version
ansible.builtin.set_fact:
proxmox_pve_version: "{{ proxmox_pveversion_info.stdout | regex_search('pve-manager\/(\\d+\\.\\d+-\\d+)') | split('/') | last }}"
- name: update packages to make sure we are at version "{{ proxmox_pve_min_version }}" or higher
ansible.builtin.apt:
name: "*"
state: latest
update_cache: true
when: proxmox_pve_version <= proxmox_pve_min_version
# backup configuration
- name: create backup of configuration
community.general.archive:
path: /etc
dest: "{{ backup_path }}proxmox-pve-backup-{{ ansible_date_time.date }}.tar.gz"
mode: 600
format: gz
tags:
- backup
# shutdown virtual maschines
- name: get running virtual maschines
ansible.builtin.shell:
cmd: "{{ proxmox_get_vms_cmd }}"
register: proxmox_running_vms_info
changed_when: false
- name: shutting down virtual maschines
ansible.builtin.command:
cmd: "{{ proxmox_qm_manager }} shutdown {{ item }} {{ proxmox_shutdown_vms_opts }}"
loop: "{{ proxmox_running_vms_info.stdout_lines }}"
when: shutdown_vms
# update repos
- name: update repos
ansible.builtin.include_tasks:
file: update-repos.yml
# do dist-upgrade
- name: upgrade Proxmox Virtual Environment Server
ansible.builtin.apt:
update_cache: true
upgrade: dist
notify:
- reboot server

83
roles/proxmox-upgrade/vars/Debian-10.yml Executable file
View File

@ -0,0 +1,83 @@
---
# when Debian 10 (buster) is detected, change repos to Debian 12 (bullseye)
__deb_debian_repos:
- {
"line": "deb http://ftp.ru.debian.org/debian bullseye main contrib",
"release": "bullseye",
"state": "present",
}
- {
"line": "deb http://ftp.ru.debian.org/debian bullseye-updates main contrib",
"release": "bullseye",
"state": "present",
}
- {
"line": "deb http://security.debian.org bullseye-security main contrib",
"release": "bullseye",
"state": "present",
}
- {
"line": "deb http://ftp.ru.debian.org/debian buster main contrib",
"release": "buster",
"state": "absent",
}
- {
"line": "deb http://ftp.ru.debian.org/debian buster-updates main contrib",
"release": "buster",
"state": "absent",
}
- {
"line": "deb http://security.debian.org buster/updates main contrib",
"release": "buster",
"state": "absent",
}
__deb_proxmox_pve_repos:
- {
"line": "deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription",
"release": "bullseye",
"state": "present",
}
- {
"line": "deb http://download.proxmox.com/debian/pve buster pve-no-subscription",
"release": "buster",
"state": "absent",
}
__deb_proxmox_pve_ent_repos:
- {
"line": "deb https://enterprise.proxmox.com/debian/pve bullseye pve-enterprise",
"release": "bullseye",
"state": "present",
}
- {
"line": "deb https://enterprise.proxmox.com/debian/pve buster pve-enterprise",
"release": "buster",
"state": "absent",
}
__deb_proxmox_pbs_repos:
- {
"line": "deb http://download.proxmox.com/debian/pbs bullseye pbs-no-subscription",
"release": "bullseye",
"state": "present",
}
- {
"line": "deb http://download.proxmox.com/debian/pbs buster pbs-no-subscription",
"release": "buster",
"state": "absent",
}
__deb_proxmox_pbs_ent_repos:
- {
"filename": "pbs-enterprise",
"line": "deb https://enterprise.proxmox.com/debian/pbs bullseye pbs-enterprise",
"release": "bullseye",
"state": "absent",
}
- {
"filename": "pbs-enterprise",
"line": "deb https://enterprise.proxmox.com/debian/pbs buster pbs-enterprise",
"release": "buster",
"state": "absent",
}

83
roles/proxmox-upgrade/vars/Debian-11.yml Executable file
View File

@ -0,0 +1,83 @@
---
# when Debian 11 (bullseye) is detected, change repos to Debian 12 (bookworm)
__deb_debian_repos:
- {
"line": "deb http://ftp.ru.debian.org/debian bookworm main contrib",
"release": "bookworm",
"state": "present",
}
- {
"line": "deb http://ftp.ru.debian.org/debian bookworm-updates main contrib",
"release": "bookworm",
"state": "present",
}
- {
"line": "deb http://security.debian.org bookworm-security main contrib",
"release": "bookworm",
"state": "present",
}
- {
"line": "deb http://ftp.ru.debian.org/debian bullseye main contrib",
"release": "bullseye",
"state": "absent",
}
- {
"line": "deb http://ftp.ru.debian.org/debian bullseye-updates main contrib",
"release": "bullseye",
"state": "absent",
}
- {
"line": "deb http://security.debian.org bullseye-security main contrib",
"release": "bullseye",
"state": "absent",
}
__deb_proxmox_pve_repos:
- {
"line": "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription",
"release": "bookworm",
"state": "present",
}
- {
"line": "deb http://download.proxmox.com/debian/pve bullseye pve-no-subscription",
"release": "bullseye",
"state": "absent",
}
__deb_proxmox_pve_ent_repos:
- {
"line": "deb https://enterprise.proxmox.com/debian/pve bookworm pve-enterprise",
"release": "bookworm",
"state": "present",
}
- {
"line": "deb https://enterprise.proxmox.com/debian/pve bullseye pve-enterprise",
"release": "bullseye",
"state": "absent",
}
__deb_proxmox_pbs_repos:
- {
"line": "deb http://download.proxmox.com/debian/pbs bookworm pbs-no-subscription",
"release": "bookworm",
"state": "present",
}
- {
"line": "deb http://download.proxmox.com/debian/pbs bullseye pbs-no-subscription",
"release": "bullseye",
"state": "absent",
}
__deb_proxmox_pbs_ent_repos:
- {
"filename": "pbs-enterprise",
"line": "deb https://enterprise.proxmox.com/debian/pbs bookworm pbs-enterprise",
"release": "bookworm",
"state": "absent",
}
- {
"filename": "pbs-enterprise",
"line": "deb https://enterprise.proxmox.com/debian/pbs bullseye pbs-enterprise",
"release": "bullseye",
"state": "absent",
}