Technical Note: Using CirrOS Images with CloudInit on Proxmox

The technote describes why you might want to run CirrOS on Proxmox and how to overcome the challenges presented by CloudInit.

Definitions

What is Proxmox?

Proxmox VE is a complete, open-source server management platform for enterprise virtualization. It tightly integrates the KVM hypervisor and Linux Containers (LXC), software-defined storage and networking functionality, on a single platform. - https://www.proxmox.com/en/proxmox-ve

What is CirrOS?

The CirrOS project provides linux disk and kernel/initramfs images that are small, boot quickly, and optimized for testing cloud infrastructure. - https://github.com/cirros-dev/cirros

CirrOS is part of Openstack CI. CirrOS images include useful tools and functions for debugging or developing cloud infrastructure. CirrOS images come with CloudInit pre-installed.

Warning: CirrOS images are for testing only and have well known login information. Users can log in with ‘cirros:letsgocubs’ locally or remotely and have passwordless sudo access to root. They should not be used in production.

What is CloudInit?

Cloud-init is the industry standard multi-distribution method for cross-platform cloud instance initialization. It is supported across all major public cloud providers, provisioning systems for private cloud infrastructure, and bare-metal installations. - https://cloudinit.readthedocs.io/en/latest/

CirrOS On Proxmox

Motivation: Why use CirrOS on Proxmox?

From development through quality assurance, there is a continuous need for integration testing of software and drivers. In our release testing environment alone, there are hundreds of tests. Therefore, we want VMs to boot as fast as possible, especially for functional storage tests such as attaching storage. We also want to minimize the storage footprint for operations like storage migration, full clones, backup, etc. Standard Cloud Images are often much larger than CirrOS, and this slows down acceptance testing.

Challenges

The CloudInit version built into CirrOS requires metadata and user data file formats that are different from the default Proxmox format. The metadata format must be JSON, and the user data must be in executable form. If CirrOS does not find the cloud-init files in a compatible form, instances take a very long time to boot due to retries.

Note: When using ‘–citype configdrive2’ the meta information is generated as JSON, but the hostname is not set.

Solution

Proxmox provides the ability to specify custom CloudInit data using the -cicustom option. Proxmox generates an ISO during startup as the delivery mechanism for the CloudInit data: this includes the meta, user, and network files.

Note: It is our experience that the CloudInit in CirrOS ignores the network information. We set the network options using standard Proxmox commands.

Define the content of the meta and user files as follows.

The meta file must contain a JSON object. Name it cirros.meta and store it as a snippet.

{
"instance-id": "INSTANCEID",
"local-hostname": "HOSTNAME"
}

The user file must be a script. Name it cirros.user and store it as a snippet.

#!/bin/sh
mkdir /root/.ssh
mkdir /home/cirros/.ssh
chmod 700 /root/.ssh
chmod 700 /home/cirros/.ssh
chown cirros /home/cirros/.ssh
echo "ssh-rsa AAAA...blockbridgetest"  >> /home/cirros/.ssh/authorized_keys
echo "ssh-rsa AAAA...blockbridgetest"  >> /root/.ssh/authorized_keys

Below is an example that shows how to specify CloudInit data for a VM.

qm create 100 --memory 256 --name vm100 --boot order=scsi0 --socket 1 --onboot no
qm importdisk vm100 /data/iso/tempate/cirros-0.5.2-x86_64-disk.img local --format raw
qm set 100 --scsihw virtio-scsi-pci --scsi0 local:100/vm-100-disk-0.raw
qm set 100 -net0 e1000,bridge=vmbr0,firewall=1
qm set 100 -ipconfig0 ip=dhcp
qm set 100 --serial0 socket --vga serial0
qm set 100 --ide2 local:cloudinit
qm set 100 -cicustom "meta=local:snippets/cirros.meta,user=local:snippets/cirros.user"