arch-desktop/notes/libvirt.md

libvirt

I would rather not be beholden to Oracle so one of my goals is to transition away from virtualbox and towards the messy combination of libvirt, kvm, qemu, and ovmf uefi (eg. edk2-ovmf).

A first-attempt went fairly well, but in addition to adding four separate packages to replace two, it also seems to introduce platform-dependent paths and extra plugins required per-tool. These are just the expenses of using open sourced code without first-party integration support.

In addition to lacking integration support, it also lacks decent documentation as you'll be scouring dozens of sources and praying what you are reading is up to date.

packages

You'll need all of these packages:

• libvirt
• qemu
• libguestfs
• virt-install
• edk2-ovmf

The edk2-ovmf will install a UEFI disk image to /usr/share/ovmf/x64/OVMF.fd, and both the package and path may vary depending on which linux distribution you are using.

vagrant

For vagrant to work you will need to add a plugin:

vagrant plugin install vagrant-libvirt

After this it will be able to communicate with qemu/kvm.

A Vagrantfile may look like this:

ENV['VAGRANT_DEFAULT_PROVIDER'] = 'libvirt'

Vagrant.configure("2") do |config|
    config.vm.box_url = 'file://' + File.dirname(__FILE__) + '/dist/arch-desktop.box'
    config.vm.box = 'arch-desktop'
    config.vm.boot_timeout = 1
    config.vm.graceful_halt_timeout = 1
    config.vm.synced_folder '.', '/vagrant', disabled: true
    config.ssh.insert_key = false

    config.vm.provider :libvirt do |v|
        v.memory = 2048
        v.cpus = 2
        v.driver = "kvm"
        v.machine_arch = 'x86_64'
        v.loader = '/usr/share/ovmf/x64/OVMF.fd'
    end
end

libvirt

Libvirt needs to run as a service:

systemctl enable libvirtd.service

This service requires some control of the network and so you'll have to ensure your network utility does not have a dnsproxy.

For example if using connman then you need to add /etc/systemd/system/connman.service.d/disable_dns_proxy.conf with:

[Service]
ExecStart=
ExecStart=/usr/bin/connmand -n --nodnsproxy

Finally, you need to create a polkit policy at /etc/polkit-1/rules.d/50-libvirt.rules for users to operate it:

polkit.addRule(function(action, subject) {
    if (action.id == "org.libvirt.unix.manage" &&
        subject.isInGroup("sudo")) {
            return polkit.Result.YES;
    }
});

This example allows only users with sudo group access to do so, but you could use looser restrictions.

packer

While packer does work without much effort, it does require you to point to the host path of OVMF:

{
    "variables": {
        "iso_url": "https://mirrors.kernel.org/archlinux/iso/{{isotime \"2006.01\"}}.01/archlinux-{{isotime \"2006.01\"}}.01-x86_64.iso",
        "iso_checksum_url": "https://mirrors.kernel.org/archlinux/iso/{{isotime \"2006.01\"}}.01/sha1sums.txt",
        "efi_bios": "/usr/share/ovmf/x64/OVMF.fd",
        "root_password": "arch",
        "username": "vagrant",
        "password": "vagrant"
    },
    "builders": [
        {
            "headless": true,
            "type": "qemu",
            "iso_url": "{{ user `iso_url` }}",
            "iso_checksum": "file:{{ user `iso_checksum_url` }}",
            "vm_name": "arch-desktop",
            "format": "qcow2",
            "accelerator": "kvm",
            "output_directory": "dist/arch-desktop",
            "firmware": "{{ user `efi_bios` }}",
            "memory": 1024,
            "disk_size": "20G",
            "disk_interface": "virtio",
            "net_device": "virtio-net",
            "ssh_username": "root",
            "ssh_password": "{{user `root_password`}}",
            "ssh_timeout": "20m",
            "shutdown_command": "systemctl poweroff",
            "boot_wait": "5s",
            "boot_command": [
                "<enter><wait90s>",
                "printf \"{{user `root_password`}}\\n{{user `root_password`}}\\n\" | passwd<enter>",
                "systemctl start sshd.service<enter>"
            ]
        }
    ],
    "provisioners": [
        {
            "type": "file",
            "source": "arch.sh",
            "destination": "arch.sh"
        },
        {
            "type": "file",
            "source": "install",
            "destination": "install"
        },
        {
            "type": "shell",
            "skip_clean": true,
            "expect_disconnect": true,
            "environment_vars": [
                "DEBUG=y",
                "enable_hibernation=y",
                "disk=vda",
                "root_password={{user `root_password`}}",
                "username={{user `username`}}",
                "password={{user `password`}}"
            ],
            "script": "setup/install.sh"
        }
    ],
    "post-processors": [
        {
            "type": "vagrant",
            "compression_level": 9,
            "output": "dist/arch-desktop.img"
        }
    ]
}

conclusion

It works, but it's not as simple nor well integrated.

To begin with you need 4-6 packages to get things working. Whether this is better or worse than a single monolithic package is debatable, but more packages means more complexity in terms of knowing what you need to get started.

With regards to packer support, it works relatively well actually. The few problems I ran into were related to disk names based on the types of disk drivers specified.

With vagrant you need to install an extra plugin. Further, I have been unable to get vagrant to launch a GUI regardless of the graphics_type or video_type used.

It also requires significantly more work to clear when testing a new build. Apparently vagrant box is not linked to virsh (eg. libvirt cli) storage, so you have to delete the image in root storage using virsh or virt-manager, and while it is possible to establish userspace storage that's even more undocumented complexity that I haven't had time to get working.

Overall, it's nice to have a purely open sourced build option, but it's way more complicated with less documentation and fewer integrations.