Sunday, January 20, 2013

VM pt 2

6.4. Installing guest virtual machines with PXE

Requirements

PXE guest installation requires a PXE server running on the same subnet as the guest virtual machines you wish to install. The method of accomplishing this depends on how the virtual machines are connected to the network. Contact Support if you require assistance setting up a PXE server.

PXE installation with virt-install

virt-install PXE installations require both the --network=bridge:installation parameter, where installation is the name of your bridge, and the --pxe parameter.By default, if no network is found, the guest virtual machine attempts to boot from alternative bootable devices. If there is no other bootable device found, the guest pauses. You can use the qemu-kvm boot parameter reboot-timeout to allow the guest to retry booting if no bootable device is found, like so:# qemu-kvm -boot reboot-timeout=1000

Example 6.2. Fully-virtualized PXE installation with virt-install

# virt-install --hvm --connect qemu:///system \ --network=bridge:installation --pxe --graphics spice \ --name rhel6-machine --ram=756 --vcpus=4 \ --os-type=linux --os-variant=rhel6 \ --disk path=/var/lib/libvirt/images/rhel6-machine.img,size=10Note that the command above cannot be executed in a text-only environment. A fully-virtualized (--hvm) guest can only be installed in a text-only environment if the --location and --extra-args "console=console_type" are provided instead of the --graphics spice parameter.

Procedure 6.2. PXE installation with virt-manager

Select PXESelect PXE as the installation method and follow the rest of the steps to configure the OS type, memory, CPU and storage settings.

Figure 6.8. Selecting the installation method

Figure 6.9. Selecting the installation type

Figure 6.10. Specifying virtualized hardware details

Figure 6.11. Specifying storage details

Start the installationThe installation is ready to start.

Figure 6.12. Finalizing virtual machine details

A DHCP request is sent and if a valid PXE server is found the guest virtual machine's installation processes will start.

Chapter 7. Installing a Red Hat Enterprise Linux 6 guest virtual machine on a Red Hat Enterprise Linux 6 host

7.1. Creating a Red Hat Enterprise Linux 6 guest with local installation media7.2. Creating a Red Hat Enterprise Linux 6 guest with a network installation tree7.3. Creating a Red Hat Enterprise Linux 6 guest with PXEThis chapter covers how to install a Red Hat Enterprise Linux 6 guest virtual machine on a Red Hat Enterprise Linux 6 host.These procedures assume that the KVM hypervisor and all other required packages are installed and the host is configured for virtualization.

Note

For more information on installing the virtualization packages, refer to Chapter 5, Installing the virtualization packages.

7.1. Creating a Red Hat Enterprise Linux 6 guest with local installation media

This procedure covers creating a Red Hat Enterprise Linux 6 guest virtual machine with a locally stored installation DVD or DVD image. DVD images are available from http://access.redhat.com for Red Hat Enterprise Linux 6.

Procedure 7.1. Creating a Red Hat Enterprise Linux 6 guest virtual machine with virt-manager

Optional: PreparationPrepare the storage environment for the virtual machine. For more information on preparing storage, refer to the Red Hat Enterprise Linux 6 Virtualization Administration Guide.

Important

Various storage types may be used for storing guest virtual machines. However, for a virtual machine to be able to use migration features the virtual machine must be created on networked storage.Red Hat Enterprise Linux 6 requires at least 1GB of storage space. However, Red Hat recommends at least 5GB of storage space for a Red Hat Enterprise Linux 6 installation and for the procedures in this guide.Open virt-manager and start the wizardOpen virt-manager by executing the virt-manager command as root or opening Applications → System Tools → Virtual Machine Manager.

Figure 7.1. The Virtual Machine Manager window

Click on the Create a new virtual machine button to start the new virtualized guest wizard.

Figure 7.2. The Create a new virtual machine button

The New VM window opens.Name the virtual machineVirtual machine names can contain letters, numbers and the following characters: '_', '.' and '-'. Virtual machine names must be unique for migration and cannot consist only of numbers.Choose the Local install media (ISO image or CDROM) radio button.

Figure 7.3. The New VM window - Step 1

Click Forward to continue.Select the installation mediaSelect the appropriate radio button for your installation media.

Figure 7.4. Locate your install media

If you wish to install from a CD-ROM or DVD, select the Use CDROM or DVD radio button, and select the appropriate disk drive from the drop-down list of drives available.If you wish to install from an ISO image, select Use ISO image, and then click the Browse... button to open the Locate media volume window.Select the installation image you wish to use, and click Choose Volume.If no images are displayed in the Locate media volume window, click on the Browse Local button to browse the host machine for the installation image or DVD drive containing the installation disk. Select the installation image or DVD drive containing the installation disk and click Open; the volume is selected for use and you are returned to the Create a new virtual machine wizard.

Important

For ISO image files and guest storage images, the recommended location to use is /var/lib/libvirt/images/. Any other location may require additional configuration by SELinux. Refer to the Red Hat Enterprise Linux 6 Virtualization Administration Guide for more details on configuring SELinux.Select the operating system type and version which match the installation media you have selected.

Figure 7.5. The New VM window - Step 2

Click Forward to continue.Set RAM and virtual CPUsChoose appropriate values for the virtual CPUs and RAM allocation. These values affect the host's and guest's performance. Memory and virtual CPUs can be overcommitted. For more information on overcommitting, refer to the Red Hat Enterprise Linux 6 Virtualization Administration Guide.Virtual machines require sufficient physical memory (RAM) to run efficiently and effectively. Red Hat supports a minimum of 512MB of RAM for a virtual machine. Red Hat recommends at least 1024MB of RAM for each logical core.Assign sufficient virtual CPUs for the virtual machine. If the virtual machine runs a multithreaded application, assign the number of virtual CPUs the guest virtual machine will require to run efficiently.You cannot assign more virtual CPUs than there are physical processors (or hyper-threads) available on the host system. The number of virtual CPUs available is noted in the Up to X available field.

Figure 7.6. The new VM window - Step 3

Click Forward to continue.StorageEnable and assign storage for the Red Hat Enterprise Linux 6 guest virtual machine. Assign at least 5GB for a desktop installation or at least 1GB for a minimal installation.

Note

Live and offline migrations require virtual machines to be installed on shared network storage. For information on setting up shared storage for virtual machines, refer to the Red Hat Enterprise Linux Virtualization Administration Guide.With the default local storageSelect the Create a disk image on the computer's hard drive radio button to create a file-based image in the default storage pool, the /var/lib/libvirt/images/ directory. Enter the size of the disk image to be created. If the Allocate entire disk now check box is selected, a disk image of the size specified will be created immediately. If not, the disk image will grow as it becomes filled.

Figure 7.7. The New VM window - Step 4

Click Forward to create a disk image on the local hard drive. Alternatively, select Select managed or other existing storage, then select Browse to configure managed storage.With a storage poolIf you selected Select managed or other existing storage in the previous step to use a storage pool and clicked Browse, the Locate or create storage volume window will appear.

Figure 7.8. The Locate or create storage volume window

Select a storage pool from the Storage Pools list.Optional: Click on the New Volume button to create a new storage volume. The Add a Storage Volume screen will appear. Enter the name of the new storage volume.Choose a format option from the Format dropdown menu. Format options include raw, cow, qcow, qcow2, qed, vmdk, and vpc. Adjust other fields as desired.

Figure 7.9. The Add a Storage Volume window

Click Finish to continue.Verify and finishVerify there were no errors made during the wizard and everything appears as expected.Select the Customize configuration before install check box to change the guest's storage or network devices, to use the para-virtualized drivers or to add additional devices.Click on the Advanced options down arrow to inspect and modify advanced options. For a standard Red Hat Enterprise Linux 6 installation, none of these options require modification.

Figure 7.10. The New VM window - local storage

Click Finish to continue into the Red Hat Enterprise Linux installation sequence. For more information on installing Red Hat Enterprise Linux 6 refer to the Red Hat Enterprise Linux 6 Installation Guide.A Red Hat Enterprise Linux 6 guest virtual machine is now created from an ISO installation disc image.

7.2. Creating a Red Hat Enterprise Linux 6 guest with a network installation tree

Procedure 


12.4. Detaching an assigned PCI deviceRed Hat Enterprise Linux 6 exposes three classes of device to its virtual machines:Emulated devices are purely virtual devices that mimic real hardware, allowing unmodified guest operating systems to work with them using their standard in-box drivers.Virtio devices are purely virtual devices designed to work optimally in a virtual machine. Virtio devices are similar to emulated devices, however, non-Linux virtual machines do not include the drivers they require by default. Virtualization management software like the Virtual Machine Manager (virt-manager) and the Red Hat Enterprise Virtualization Hypervisor install these drivers automatically for supported non-Linux guest operating systems.Assigned devices are physical devices that are exposed to the virtual machine. This method is also known as 'passthrough'. Device assignment allows virtual machines exclusive access to PCI devices for a range of tasks, and allows PCI devices to appear and behave as if they were physically attached to the guest operating system.Device assignment is supported on PCI Express devices, except graphics cards. Parallel PCI devices may be supported as assigned devices, but they have severe limitations due to security and system configuration conflicts.Red Hat Enterprise Linux 6 supports 32 PCI device slots per virtual machine, and 8 PCI functions per device slot. This gives a theoretical maximum of 256 configurable PCI functions per guest.However, this theoretical maximum is subject to the following limitations:Each virtual machine supports a maximum of 8 assigned device functions.4 PCI device slots are configured with emulated devices by default. However, users can explicitly remove 2 of the emulated devices that are configured by default (the video adapter device in slot 2, and the memory balloon driver device in slot 3). This gives users a supported functional maximum of 30 PCI device slots per virtual machine.Red Hat Enterprise Linux 6.0 and newer supports hot plugging assigned PCI devices into virtual machines. However, PCI device hot plugging operates at the slot level and therefore does not support multi-function PCI devices. Multi-function PCI devices are recommended for static device configuration only.

Note

Red Hat Enterprise Linux 6.0 limited guest operating system driver access to a device's standard and extended configuration space. Limitations that were present in Red Hat Enterprise Linux 6.0 are significantly reduced in Red Hat Enterprise Linux 6.1, and enable a much larger set of PCI Express devices to be successfully assigned to KVM guests.Secure device assignment also requires interrupt remapping support. If a platform does not support interrupt remapping, device assignment will fail. To use device assignment without interrupt remapping support in a development environment, set the allow_unsafe_assigned_interrupts KVM module parameter to 1.PCI device assignment is only available on hardware platforms supporting either Intel VT-d or AMD IOMMU. These Intel VT-d or AMD IOMMU specifications must be enabled in BIOS for PCI device assignment to function.

Procedure 12.1. Preparing an Intel system for PCI device assignment

Enable the Intel VT-d specificationsThe Intel VT-d specifications provide hardware support for directly assigning a physical device to a virtual machine. These specifications are required to use PCI device assignment with Red Hat Enterprise Linux.The Intel VT-d specifications must be enabled in the BIOS. Some system manufacturers disable these specifications by default. The terms used to refer to these specifications can differ between manufacturers; consult your system manufacturer's documentation for the appropriate terms.Activate Intel VT-d in the kernelActivate Intel VT-d in the kernel by adding the intel_iommu=on parameter to the kernel line in the /boot/grub/grub.conf file.The example below is a modified grub.conf file with Intel VT-d activated.default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux Server (2.6.32-330.x86_645) root (hd0,0) kernel /vmlinuz-2.6.32-330.x86_64 ro root=/dev/VolGroup00/LogVol00 rhgb quiet intel_iommu=on initrd /initrd-2.6.32-330.x86_64.imgReady to useReboot the system to enable the changes. Your system is now capable of PCI device assignment.

Procedure 12.2. Preparing an AMD system for PCI device assignment

Enable the AMD IOMMU specificationsThe AMD IOMMU specifications are required to use PCI device assignment in Red Hat Enterprise Linux. These specifications must be enabled in the BIOS. Some system manufacturers disable these specifications by default.Enable IOMMU kernel supportAppend amd_iommu=on to the kernel command line in /boot/grub/grub.conf so that AMD IOMMU specifications are enabled at boot.

12.1. Assigning a PCI device with virsh

These steps cover assigning a PCI device to a virtual machine on a KVM hypervisor.This example uses a PCIe network controller with the PCI identifier code, pci_0000_01_00_0, and a fully virtualized guest machine named guest1-rhel6-64.

Procedure 12.3. Assigning a PCI device to a guest virtual machine with virsh

Identify the deviceFirst, identify the PCI device designated for device assignment to the virtual machine. Use the lspci command to list the available PCI devices. You can refine the output of lspci with grep.This example uses the Ethernet controller highlighted in the following output:# lspci | grep Ethernet 00:19.0 Ethernet controller: Intel Corporation 82567LM-2 Gigabit Network Connection 01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)This Ethernet controller is shown with the short identifier 00:19.0. We need to find out the full identifier used by virsh in order to assign this PCI device to a virtual machine.To do so, combine the virsh nodedev-list command with the grep command to list all devices of a particular type (pci) that are attached to the host machine. Then look at the output for the string that maps to the short identifier of the device you wish to use.This example highlights the string that maps to the Ethernet controller with the short identifier 00:19.0. Note that the : and . characters are replaced with underscores in the full identifier.# virsh nodedev-list --cap pci pci_0000_00_00_0 pci_0000_00_01_0 pci_0000_00_03_0 pci_0000_00_07_0 pci_0000_00_10_0 pci_0000_00_10_1 pci_0000_00_14_0 pci_0000_00_14_1 pci_0000_00_14_2 pci_0000_00_14_3 pci_0000_00_19_0 pci_0000_00_1a_0 pci_0000_00_1a_1 pci_0000_00_1a_2 pci_0000_00_1a_7 pci_0000_00_1b_0 pci_0000_00_1c_0 pci_0000_00_1c_1 pci_0000_00_1c_4 pci_0000_00_1d_0 pci_0000_00_1d_1 pci_0000_00_1d_2 pci_0000_00_1d_7 pci_0000_00_1e_0 pci_0000_00_1f_0 pci_0000_00_1f_2 pci_0000_00_1f_3 pci_0000_01_00_0 pci_0000_01_00_1 pci_0000_02_00_0 pci_0000_02_00_1 pci_0000_06_00_0 pci_0000_07_02_0 pci_0000_07_03_0Record the PCI device number that maps to the device you want to use; this is required in other steps.Review device informationInformation on the domain, bus, and function are available from output of the virsh nodedev-dumpxml command:virsh nodedev-dumpxml pci_0000_00_19_0 <device>






Offload.Rx.ChecksumSpecifies the RX checksum offloading mode.In Red Hat Enterprise Linux 6.4 and onward, the valid values for this parameter are All (the default), which enables IP, TCP and UDP checksum offloading for both IPv4 and IPv6; TCP/UDP(v4,v6), which enables TCP and UDP checksum offloading for both IPv4 and IPv6; TCP/UDP(v4), which enables TCP and UDP checksum offloading for IPv4 only; and TCP(v4), which enables only TCP checksum offloading for IPv4 only.In Red Hat Enterprise Linux 6.3 and earlier, the valid values are Disable (the default), which disables RX checksum offloading; All, which enables TCP, UDP, and IP checksum offloading; TCP/UDP, which enables TCP and UDP checksum offloading; and TCP, which enables only TCP checksum offloading.

Test and debug parameters

Important

Test and debug parameters should only be used for testing or debugging; they should not be used in production.TestOnly.DelayConnect(ms)The period for which to delay connection upon startup, in milliseconds. The default value is 0.TestOnly.DPCCheckingSets the DPC checking mode. 0 (the default) disables DPC checking. 1 enables DPC checking; each hang test verifies DPC activity and acts as if the DPC was spawned. 2 clears the device interrupt status and is otherwise identical to 1.TestOnly.Scatter-GatherA Boolean value that determines whether scatter-gather functionality is enabled. The default value is 1 (enabled). Setting this value to 0 disables scatter-gather functionality and all dependent capabilities.TestOnly.InterruptRecoveryA Boolean value that determines whether interrupt recovery is enabled. The default value is 1 (enabled).TestOnly.PacketFilterA Boolean value that determines whether packet filtering is enabled. The default value is 1(enabled).TestOnly.BatchReceiveA Boolean value that determines whether packets are received in batches, or singularly. The default value is 1, which enables batched packet receipt.TestOnly.PromiscuousA Boolean value that determines whether promiscuous mode is enabled. The default value is 0 (disabled).TestOnly.AnalyzeIPPacketsA Boolean value that determines whether the checksum fields of outgoing IP packets are tested and verified for debugging purposes. The default value is 0 (no checking).TestOnly.RXThrottleAn integer that determines the number of receive packets handled in a single DPC. The default value is 1000.TestOnly.UseSwTxChecksumA Boolean value that determines whether hardware checksumming is enabled. The default value is 0 (disabled).

Common libvirt errors and troubleshooting

This appendix documents common libvirt-related problems and errors along with instructions for dealing with them.Locate the error on the table below and follow the corresponding link under Solutionfor detailed troubleshooting information.

Table B.1. Common libvirt errors

ErrorDescription of problemSolutionlibvirtd failed to startThe libvirt daemon failed to start. However, there is no information about this error in /var/log/messages.Section B.1, "libvirtd failed to start"Cannot read CA certificateThis is one of several errors that occur when the URI fails to connect to the hypervisor.Section B.2, "The URI failed to connect to the hypervisor"Failed to connect socket ... : Permission deniedThis is one of several errors that occur when the URI fails to connect to the hypervisor.Section B.2, "The URI failed to connect to the hypervisor"Other connectivity errorsThese are other errors that occur when the URI fails to connect to the hypervisor.Section B.2, "The URI failed to connect to the hypervisor"Internal error guest CPU is not compatible with host CPUThe guest virtual machine cannot be started because the host and guest processors are different.Section B.3, "The guest virtual machine cannot be started: internal error guest CPU is not compatible with host CPU"Failed to create domain from vm.xml error: monitor socket did not show up.: Connection refusedThe guest virtual machine (or domain) starting fails and returns this error or similar.Section B.4, "Guest starting fails with error: monitor socket did not show up"Internal error cannot find character device (null)This error can occur when attempting to connect a guest's console. It reports that there is no serial console configured for the guest virtual machine.Section B.5, "Internal error cannot find character device (null)"No boot deviceAfter building a guest virtual machine from an existing disk image, the guest booting stalls. However, the guest can start successfully using the QEMUcommand directly.Section B.6, "Guest virtual machine booting stalls with error: No boot device"The virtual network "default" has not been startedIf the default network (or other locally-created network) is unable to start, any virtual machine configured to use that network for its connectivity will also fail to start.Section B.7, "Virtual network default has not been started"PXE boot (or DHCP) on guest failedA guest virtual machine starts successfully, but is unable to acquire an IP address from DHCP, boot using the PXE protocol, or both. This is often a result of a long forward delay time set for the bridge, or when the iptables package and kernel do not support checksum mangling rules.Section B.8, "PXE boot (or DHCP) on guest failed"Guest can reach outside network, but cannot reach host when using macvtap interfaceA guest can communicate with other guests, but cannot connect to the host machine after being configured to use a macvtap (or type='direct') network interface.This is actually not an error — it is the defined behavior of macvtap.Section B.9, "Guest can reach outside network, but cannot reach host when using macvtap interface"Could not add rule to fixup DHCP response checksums on network 'default'This warning message is almost always harmless, but is often mistakenly seen as evidence of a problem.Section B.10, "Could not add rule to fixup DHCP response checksums on network 'default'"Unable to add bridge br0 port vnet0: No such deviceThis error message or the similar Failed to add tap interface to bridge 'br0': No such devicereveal that the bridge device specified in the guest's (or domain's) <interface>definition does not exist.Section B.11, "Unable to add bridge br0 port vnet0: No such device"Warning: could not open /dev/net/tun: no virtual network emulation qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could not be initializedThe guest virtual machine does not start after configuring a type='ethernet' (or 'generic ethernet') interface in the host system. This error or similar appears either in libvirtd.log, /var/log/libvirt/qemu/name_of_guest.log, or in both.Section B.12, "Guest is unable to start with error: warning: could not open /dev/net/tun"Unable to resolve address name_of_host service '49155': Name or service not knownQEMU guest migration fails and this error message appears with an unfamiliar hostname.Section B.13, "Migration fails with Error: unable to resolve address"Unable to allow access for disk path /var/lib/libvirt/images/qemu.img: No such file or directoryA guest virtual machine cannot be migrated because libvirtcannot access the disk image(s).Section B.14, "Migration fails with Unable to allow access for disk path: No such file or directory"No guest virtual machines are present when libvirtd is startedThe libvirt daemon is successfully started, but no guest virtual machines appear to be present when running virsh list --all.Section B.15, "No guest virtual machines are present when libvirtd is started"Unable to connect to server at 'host:16509': Connection refused ... error: failed to connect to the hypervisorWhile libvirtd should listen on TCP ports for connections, the connection to the hypervisor fails.Section B.16, "Unable to connect to server at 'host:16509': Connection refused ... error: failed to connect to the hypervisor"Common XML errorslibvirt uses XML documents to store structured data. Several common errors occur with XML documents when they are passed to libvirt through the API. This entry provides instructions for editing guest XML definitions, and details common errors in XML syntax and configuration.Section B.17, "Common XML errors"

B.1. libvirtd failed to start

SymptomThe libvirt daemon does not start automatically. Starting the libvirt daemon manually fails as well:# /etc/init.d/libvirtd start * Caching service dependencies ... [ ok ] * Starting libvirtd ... /usr/sbin/libvirtd: error: Unable to initialize network sockets. Check /var/log/messages or run without --daemon for more info. * start-stop-daemon: failed to start `/usr/sbin/libvirtd' [ !! ] * ERROR: libvirtd failed to startMoreover, there is not 'more info' about this error in /var/log/messages.InvestigationChange libvirt's logging in /etc/libvirt/libvirtd.conf by uncommenting the line below. To uncomment the line, open the /etc/libvirt/libvirtd.conf file in a text editor, remove the hash (or #) symbol from the beginning of the following line, and save the change:log_outputs="3:syslog:libvirtd"

Note

This line is commented out by default to prevent libvirt from producing excessive log messages. After diagnosing the problem, it is recommended to comment this line again in the /etc/libvirt/libvirtd.conf file.Restart libvirt to determine if this has solved the problem.If libvirtd still does not start successfully, an error similar to the following will be shown in the /var/log/messages file:Feb 6 17:22:09 bart libvirtd: 17576: info : libvirt version: 0.9.9 Feb 6 17:22:09 bart libvirtd: 17576: error : virNetTLSContextCheckCertFile:92: Cannot read CA certificate '/etc/pki/CA/cacert.pem': No such file or directory Feb 6 17:22:09 bart /etc/init.d/libvirtd[17573]: start-stop-daemon: failed to start `/usr/sbin/libvirtd' Feb 6 17:22:09 bart /etc/init.d/libvirtd[17565]: ERROR: libvirtd failed to startThe libvirtd man page shows that the missing cacert.pem file is used as TLS authority when libvirt is run in Listen for TCP/IP connections mode. This means the --listen parameter is being passed.SolutionConfigure the libvirt daemon's settings with one of the following methods:Install a CA certificate.

Note

For more information on CA certificates and configuring system authentication, refer to the Configuring Authentication chapter in the Red Hat Enterprise Linux 6 Deployment Guide.Do not use TLS; use bare TCP instead. In /etc/libvirt/libvirtd.conf set listen_tls = 0 and listen_tcp = 1. The default values are listen_tls = 1 and listen_tcp = 0.Do not pass the --listen parameter. In /etc/sysconfig/libvirtd.confchange the LIBVIRTD_ARGS variable.

B.2. The URI failed to connect to the hypervisor

Several different errors can occur when connecting to the server (for example, when running virsh).

B.2.1. Cannot read CA certificate

SymptomWhen running a command, the following error (or similar) appears:$ virsh -c name_of_uri list error: Cannot read CA certificate '/etc/pki/CA/cacert.pem': No such file or directory error: failed to connect to the hypervisorInvestigation
★★

15.1. Preparing the boot server

To perform the steps in this chapter you will need:A PXE Server (DHCP and TFTP) - This can be a libvirt internal server, manually-configured dhcpd and tftpd, dnsmasq, a server configured by Cobbler, or some other server.Boot images - for example, PXELINUX configured manually or by Cobbler.

15.1.1. Setting up a PXE boot server on a private libvirt network

This example uses the default network. Perform the following steps:

Procedure 15.1. Configuring the PXE boot server

Place the PXE boot images and configuration in /var/lib/tftp.Run the following commands:# virsh net-destroy default # virsh net-edit defaultEdit the <ip> element in the configuration file for the default network to include the appropriate address, network mask, DHCP address range, and boot file, where BOOT_FILENAME represents the file name you are using to boot the guest virtual machine.<ip address='192.168.122.1' netmask='255.255.255.0'> <tftp root='/var/lib/tftp' /> <dhcp> <range start='192.168.122.2' end='192.168.122.254' /> <bootp file='BOOT_FILENAME' /> </dhcp> </ip>Boot the guest using PXE (refer to Section 15.2, "Booting a guest using PXE").

15.2. Booting a guest using PXE

This section demonstrates how to boot a guest virtual machine with PXE.

15.2.1. Using bridged networking

Procedure 15.2. Booting a guest using PXE and bridged networking

Ensure bridging is enabled such that the PXE boot server is available on the network.Boot a guest virtual machine with PXE booting enabled. You can use the virt-install command to create a new virtual machine with PXE booting enabled, as shown in the following example command:virt-install --pxe --network bridge=breth0 --promptAlternatively, ensure that the guest network is configured to use your bridged network, and that the XML guest configuration file has a <boot dev='network'/> element inside the <os> element, as shown in the following example:<os> <type arch='x86_64' machine='rhel6.2.0'>hvm</type> <boot dev='network'/> <boot dev='hd'/> </os> <interface type='bridge'> <mac address='52:54:00:5a:ad:cb'/> <source bridge='breth0'/> <target dev='vnet0'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface>

15.2.2. Using a private libvirt network

Procedure 15.3. Using a private libvirt network

Configure PXE booting on libvirt as shown in Section 15.1.1, "Setting up a PXE boot server on a private libvirt network".Boot a guest virtual machine using libvirt with PXE booting enabled. You can use the virt-install command to create/install a new virtual machine using PXE:virt-install --pxe --network network=default --promptAlternatively, ensure that the guest network is configured to use your bridged network, and that the XML guest configuration file has a <boot dev='network'/> element inside the <os> element, as shown in the following example:<os> <type arch='x86_64' machine='rhel6.2.0'>hvm</type> <boot dev='network'/> <boot dev='hd'/> </os>Also ensure that the guest virtual machine is connected to the private network:<interface type='network'> <mac address='52:54:00:66:79:14'/> <source network='default'/> <target dev='vnet0'/> <alias name='net0'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/> </interface>

Chapter 16. QEMU Guest Agent

16.1. Set Up Communication between Guest Agent and HostThe QEMU Guest Agent allows the host machine to issue commands to the guest operating system. The guest operating system then responds to those commands asynchronously.This section covers the options and commands available to the guest agent in detail. It also covers how to run the guest agent in the foreground, or as a daemon in the background.

16.1. Set Up Communication between Guest Agent and Host

The host machine communicates with the guest agent through a VirtIO serial connection between the host and guest machines. A VirtIO serial channel is connected to the host via a character device driver (typically a Unix socket), and the guest listens on this serial channel. The following procedure shows how to set up the host and guest machines for guest agent use.

Procedure 16.1. Set Up Host-Agent Communication

Launch QEMU with a character device driverLaunch QEMU as usual, with additional definitions for the character device driver required to communicate with the guest agent.The following example launches QEMU to communicate over the Unix socket /tmp/qga.sock./usr/libexec/qemu-kvm [...] -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga0 \ -device virtio-serial \ -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0Start the Guest AgentOn the guest, run the following command to start the Guest Agent:qemu-ga --path device_path --method methodThe guest agent now parses incoming QMP messages for commands, and acts upon them if valid.If no other method or path is specified with the --method or --path options respectively, the Guest Agent listens over virtio-serial, through the /dev/virtio-ports/org.qemu.guest_agent.0 device path.You can now communicate with the guest by sending valid QMP commands over the established character device driver.For further information about the guest agent, refer to the Red Hat Enterprise Linux 6 Virtualization Administration Guide.

NetKVM Driver Parameters

After the NetKVM driver is installed, you can configure it to better suit your environment. The parameters listed in this section can be configured in the Windows Device Manager (devmgmt.msc).

Important

Modifying the driver's parameters causes Windows to re-load that driver. This interrupts existing network activity.

Procedure A.1. Configuring NetKVM Parameters

Open Device ManagerClick on the Start button. In the right-hand pane, right-click on Computer, and click Manage. If prompted, click Continue on the User Account Control window. This opens the Computer Management window.In the left-hand pane of the Computer Management window, click Device Manager.Locate the correct deviceIn the central pane of the Computer Management window, click on the + symbol beside Network adapters.Under the list of Red Hat VirtIO Ethernet Adapter devices, double-click on NetKVM. This opens the Properties window for that device.View device parametersIn the Properties window, click on the Advanced tab.Modify device parametersClick on the parameter you wish to modify to display the options for that parameter.Modify the options as appropriate, then click on OK to save your changes.

A.1. Configurable parameters for NetKVM

Logging parameters

Logging.EnableA Boolean value that determines whether logging is enabled. The default value is 1(enabled).Logging.LevelAn integer that defines the logging level. As the integer increases, so does the verbosity of the log. The default value is 0 (errors only). 1-2 adds configuration messages. 3-4 adds packet flow information. 5-6 adds interrupt and DPC level trace information.

Important

High logging levels will slow down your guest virtual machine.Logging.Statistics(sec)An integer that defines whether log statistics are printed, and the time in seconds between each periodical statistics printout. The default value is 0 (no logging statistics).

Initial parameters

Assign MACA string that defines the locally-administered MAC address for the para-virtualized NIC. This is not set by default.Init.ConnectionRate(Mb)An integer that represents the connection rate in megabytes. The default value for Windows 2008 and later is 10000.Init.Do802.1PQA Boolean value that enables Priority/VLAN tag population and removal support. The default value is 1 (enabled).Init.UseMergedBuffersA Boolean value that enables merge-able RX buffers. The default value is 1 (enabled).Init.UsePublishEventsA Boolean value that enables published event use. The default value is 1 (enabled).Init.MTUSizeAn integer that defines the maximum transmission unit (MTU). The default value is 1500. Any value from 500 to 65500 is acceptable.Init.IndirectTxControls whether indirect ring descriptors are in use. The default value is Disable, which disables use of indirect ring descriptors. Other valid values are Enable, which enables indirect ring descriptor usage; and Enable*, which enables conditional use of indirect ring descriptors.Init.MaxTxBuffersAn integer that represents the amount of TX ring descriptors that will be allocated. The default value is 1024. Valid values are: 16, 32, 64, 128, 256, 512, or 1024.Init.MaxRxBuffersAn integer that represents the amount of RX ring descriptors that will be allocated. The default value is 256. Valid values are: 16, 32, 64, 128, 256, 512, or 1024.Offload.Tx.ChecksumSpecifies the TX checksum offloading mode.In Red Hat Enterprise Linux 6.4 and onward, the valid values for this parameter are All (the default), which enables IP, TCP and UDP checksum offloading for both IPv4 and IPv6; TCP/UDP(v4,v6), which enables TCP and UDP checksum offloading for both IPv4 and IPv6; TCP/UDP(v4), which enables TCP and UDP checksum offloading for IPv4 only; and TCP(v4), which enables only TCP checksum offloading for IPv4 only.In Red Hat Enterprise Linux 6.3 and earlier, the valid values for this parameter are TCP/UDP(the default value), which enables TCP and UDP checksum offload; TCP, which enables only TCP checksum offload; or Disable, which disables TX checksum offload.Offload.Tx.LSOA Boolean value that enables TX TCP Large Segment Offload (LSO). The default value is 1(enabled).

★★

Red Hat Enterprise Linux 6 Security-Enhanced Linux User Guide.

B.13. Migration fails with Error: unable to resolve address

SymptomQEMU guest migration fails and this error message appears:# virsh migrate qemu qemu+tcp://192.168.122.12/system error: Unable to resolve address name_of_host service '49155': Name or service not knownFor example, if the destination hostname is "newyork", the error message will appear as:# virsh migrate qemu qemu+tcp://192.168.122.12/system error: Unable to resolve address 'newyork' service '49155': Name or service not knownHowever, this error looks strange as we did not use "newyork" hostname anywhere.InvestigationDuring migration, libvirtd running on the destination host creates a URI from an address and port where it expects to receive migration data and sends it back to libvirtd running on the source host.In this case, the destination host (192.168.122.12) has its name set to 'newyork'. For some reason, libvirtd running on that host is unable to resolve the name to an IP address that could be sent back and still be useful. For this reason, it returned the 'newyork'hostname hoping the source libvirtd would be more successful with resolving the name. This can happen if DNS is not properly configured or /etc/hosts has the hostname associated with local loopback address (127.0.0.1).Note that the address used for migration data cannot be automatically determined from the address used for connecting to destination libvirtd (for example, from qemu+tcp://192.168.122.12/system). This is because to communicate with the destination libvirtd, the source libvirtd may need to use network infrastructure different from that which virsh (possibly running on a separate machine) requires.SolutionThe best solution is to configure DNS correctly so that all hosts involved in migration are able to resolve all host names.If DNS cannot be configured to do this, a list of every host used for migration can be added manually to the /etc/hosts file on each of the hosts. However, it is difficult to keep such lists consistent in a dynamic environment.If the host names cannot be made resolvable by any means, virsh migrate supports specifying the migration host:# virsh migrate qemu qemu+tcp://192.168.122.12/system tcp://192.168.122.12Destination libvirtd will take the tcp://192.168.122.12 URI and append an automatically generated port number. If this is not desirable (because of firewall configuration, for example), the port number can be specified in this command:# virsh migrate qemu qemu+tcp://192.168.122.12/system tcp://192.168.122.12:12345Another option is to use tunnelled migration. Tunnelled migration does not create a separate connection for migration data, but instead tunnels the data through the connection used for communication with destination libvirtd (for example, qemu+tcp://192.168.122.12/system):# virsh migrate qemu qemu+tcp://192.168.122.12/system --p2p --tunnelled

B.14. Migration fails with Unable to allow access for disk path: No such file or directory

SymptomA guest virtual machine (or domain) cannot be migrated because libvirt cannot access the disk image(s):# virsh migrate qemu qemu+tcp://name_of_host/system error: Unable to allow access for disk path /var/lib/libvirt/images/qemu.img: No such file or directoryFor example, if the destination hostname is "newyork", the error message will appear as:# virsh migrate qemu qemu+tcp://newyork/system error: Unable to allow access for disk path /var/lib/libvirt/images/qemu.img: No such file or directoryInvestigationBy default, migration only transfers the in-memory state of a running guest (such as memory or CPU state). Although disk images are not transferred during migration, they need to remain accessible at the same path by both hosts.SolutionSet up and mount shared storage at the same location on both hosts. The simplest way to do this is to use NFS:

Procedure B.5. Setting up shared storage

Set up an NFS server on a host serving as shared storage. The NFS server can be one of the hosts involved in the migration, as long as all hosts involved are accessing the shared storage through NFS.# mkdir -p /exports/images # cat >>/etc/exports <<EOF /exports/images 192.168.122.0/24(rw,no_root_squash) EOFMount the exported directory at a common location on all hosts running libvirt. For example, if the IP address of the NFS server is 192.168.122.1, mount the directory with the following commands:# cat >>/etc/fstab <<EOF 192.168.122.1:/exports/images /var/lib/libvirt/images nfs auto 0 0 EOF # mount /var/lib/libvirt/images

Note

It is not possible to export a local directory from one host using NFS and mount it at the same path on another host — the directory used for storing disk images must be mounted from shared storage on both hosts. If this is not configured correctly, the guest virtual machine may lose access to its disk images during migration, because the source host's libvirt daemon may change the owner, permissions, and SELinux labels on the disk images after it successfully migrates the guest to its destination.If libvirt detects that the disk images are mounted from a shared storage location, it will not make these changes.

B.15. No guest virtual machines are present when libvirtd is started

SymptomThe libvirt daemon is successfully started, but no guest virtual machines appear to be present.# virsh list --all Id Name State ---------------------------------------------------- # InvestigationThere are various possible causes of this problem. Performing these tests will help to determine the cause of this situation:Verify KVM kernel modulesVerify that KVM kernel modules are inserted in the kernel:# lsmod | grep kvm kvm_intel 121346 0 kvm 328927 1 kvm_intelIf you are using an AMD machine, verify the kvm_amd kernel modules are inserted in the kernel instead, using the similar command lsmod | grep kvm_amd in the root shell.If the modules are not present, insert them using the modprobe <modulename>command.

Note

Although it is uncommon, KVM virtualization support may be compiled into the kernel. In this case, modules are not needed.Verify virtualization extensionsVerify that virtualization extensions are supported and enabled on the host:# egrep "(vmx|svm)" /proc/cpuinfo flags : fpu vme de pse tsc ... svm ... skinit wdt npt lbrv svm_lock nrip_save flags : fpu vme de pse tsc ... svm ... skinit wdt npt lbrv svm_lock nrip_saveEnable virtualization extensions in your hardware's firmware configuration within the BIOS setup. Refer to your hardware documentation for further details on this.Verify client URI configurationVerify that the URI of the client is configured as desired:# virsh uri vbox:///systemFor example, this message shows the URI is connected to the VirtualBoxhypervisor, not QEMU, and reveals a configuration error for a URI that is otherwise set to connect to a QEMU hypervisor. If the URI was correctly connecting to QEMU, the same message would appear instead as:# virsh uri qemu:///systemThis situation occurs when there are other hypervisors present, which libvirt may speak to by default.SolutionAfter performing these tests, use the following command to view a list of guest virtual machines:# virsh list --all

B.16. Unable to connect to server at 'host:16509': Connection refused ... error: failed to connect to the hypervisor

SymptomWhile libvirtd should listen on TCP ports for connections, the connections fail:# virsh -c qemu+tcp://host/system error: unable to connect to server at 'host:16509': Connection refused error: failed to connect to the hypervisorThe libvirt daemon is not listening on TCP ports even after changing configuration in /etc/libvirt/libvirtd.conf:# grep listen_ /etc/libvirt/libvirtd.conf listen_tls = 1 listen_tcp = 1 listen_addr = "0.0.0.0"However, the TCP ports for libvirt are still not open after changing configuration:# netstat -lntp | grep libvirtd #InvestigationThe libvirt daemon was started without the --listen option. Verify this by running this command:# ps aux | grep libvirtd root 27314 0.0 0.0 1000920 18304 ? Sl Feb16 1:19 libvirtd --daemonThe output does not contain the --listen option.SolutionStart the daemon with the --listen option.To do this, modify the /etc/sysconfig/libvirtd file and uncomment the following line:#LIBVIRTD_ARGS="--listen"Then restart the libvirtd service with this command:# /etc/init.d/libvirtd restart

B.17. Common XML errors

The libvirt tool uses XML documents to store structured data. A variety of common errors occur with XML documents when they are passed to libvirt through the API. Several common XML errors — including misformatted XML, inappropriate values, and missing elements — are detailed below.

B.17.1. Editing domain definition

Although it is not recommended, it is sometimes necessary to edit a guest virtual machine's (or a domain's) XML file manually. To access the guest's XML for editing, use the following command:# virsh edit name_of_guest.xmlThis command opens the file in a text editor with the current definition of the guest virtual machine. After finishing the edits and saving the changes, the XML is reloaded and parsed by libvirt. If the XML is correct, the following message is displayed:# virsh edit name_of_guest.xml Domain name_of_guest.xml XML configuration edited.

Important

When using the edit command in virsh to edit an XML document, save all changes before exiting the editor.After saving the XML file, use the xmllint command to validate that the XML is well-formed, or the virt-xml-validate command to check for usage problems:# xmllint --noout config.xml# virt-xml-validate config.xmlIf no errors are returned, the XML description is well-formed and matches the libvirtschema. While the schema does not catch all constraints, fixing any reported errors will further troubleshooting.XML documents stored by libvirtThese documents contain definitions of states and configurations for the guests. These documents are automatically generated and should not be edited manually. Errors in these documents contain the file name of the broken document. The file name is valid only on the host machine defined by the URI, which may refer to the machine the command was run on.Errors in files created by libvirt are rare. However, one possible source of these errors is a downgrade of libvirt — while newer versions of libvirt can always read XML generated by older versions, older versions of libvirt may be confused by XML elements added in a newer version.

B.17.2. XML syntax errors

Syntax errors are caught by the XML parser. The error message contains information for identifying the problem.This example error message from the XML parser consists of three lines — the first line denotes the error message, and the two following lines contain the context and location of the XML code containing the error. The third line contains an indicator showing approximately where the error lies on the line above it:error: (name_of_guest.xml):6: StartTag: invalid element name <vcpu>2</vcpu>< -----------------^Information contained in this message:(name_of_guest.xml)This is the file name of the document that contains the error. File names in parentheses are symbolic names to describe XML documents parsed from memory, and do not directly correspond to files on disk. File names that are not contained in parentheses are local files that reside on the target of the connection.6This is the line number in the XML file that contains the error.StartTag: invalid element nameThis is the error message from the libxml2 parser, which describes the specific XML error.

B.17.2.1. Stray < in the document

SymptomThe following error occurs:error: (name_of_guest.xml):6: StartTag: invalid element name <vcpu>2</vcpu>< -----------------^InvestigationThis error message shows that the parser expects a new element name after the < symbol on line 6 of a guest's XML file.Ensure line number display is enabled in your text editor. Open the XML file, and locate the text on line 6:<domain type='kvm'> <name>name_of_guest</name> <memory>524288</memory> <vcpu>2</vcpu>< This snippet of a guest's XML file contains an extra < in the document:SolutionRemove the extra < or finish the new element.

B.17.2.2. Unterminated attribute

SymptomThe following error occurs:error: (name_of_guest.xml):2: Unescaped '<' not allowed in attributes values <name>name_of_guest</name> --^InvestigationThis snippet of a guest's XML file contains an unterminated element attribute value:<domain type='kvm> <name>name_of_guest</name>In this case, 'kvm' is missing a second quotation mark. Strings of attribute values, such as quotation marks and apostrophes, must be opened and closed, similar to XML start and end tags.SolutionCorrectly open and close all attribute value strings.

B.17.2.3. Opening and ending tag mismatch

SymptomThe following error occurs:error: (name_of_guest.xml):61: Opening and ending tag mismatch: clock line 16 and domain </domain> ---------^InvestigationThe error message above contains three clues to identify the offending tag:The message following the last colon, clock line 16 and domain, reveals that <clock> contains a mismatched tag on line 16 of the document. The last hint is the pointer in the context part of the message, which identifies the second offending tag.Unpaired tags must be closed with />. The following snippet does not follow this rule and has produced the error message shown above:<domain type='kvm'> ... <clock offset='utc'>This error is caused by mismatched XML tags in the file. Every XML tag must have a matching start and end tag.Other examples of mismatched XML tagsThe following examples produce similar error messages and show variations of mismatched XML tags.This snippet contains an unended pair tag for <features>:<domain type='kvm'> ... <features> <acpi/> <pae/> ... </domain>This snippet contains an end tag (</name>) without a corresponding start tag:<domain type='kvm'> </name> ... </domain>SolutionEnsure all XML tags start and end correctly.

B.17.2.4. Typographical errors in tags

SymptomThe following error message appears:error: (name_of_guest.xml):1: Specification mandate value for attribute ty <domain ty pe='kvm'> -----------^InvestigationXML errors are easily caused by a simple typographical error. This error message highlights the XML error — in this case, an extra white space within the word type — with a pointer.<domain ty pe='kvm'>These XML examples will not parse correctly because of typographical errors such as a missing special character, or an additional character:<domain type 'kvm'><dom#ain type='kvm'>SolutionTo identify the problematic tag, read the error message for the context of the file, and locate the error with the pointer. Correct the XML and save the changes.

B.17.3. Logic and configuration errors

A well-formatted XML document can contain errors that are correct in syntax but libvirtcannot parse. Many of these errors exist, with two of the most common cases outlined below.

B.17.3.1. Vanishing parts

SymptomParts of the change you have made do not show up and have no effect after editing or defining the domain. The define or edit command works, but when dumping the XML once again, the change disappears.InvestigationThis error likely results from a broken construct or syntax that libvirt does not parse. The libvirt tool will generally only look for constructs it knows but ignore everything else, resulting in some of the XML changes vanishing after libvirt parses the input.SolutionValidate the XML input before passing it to the edit or define commands. The libvirtdevelopers maintain a set of XML schemas bundled with libvirt which define the majority of the constructs allowed in XML documents used by libvirt.Validate libvirt XML files using the following command:# virt-xml-validate libvirt.xmlIf this command passes, libvirt will likely understand all constructs from your XML, except if the schemas cannot detect options which are valid only for a given hypervisor. Any XML generated by libvirt as a result of a virsh dump command, for example, should validate without error.

B.17.3.2. Incorrect drive device type

SymptomThe definition of the source image for the CD-ROM virtual drive is not present, despite being added:# virsh dumpxml domain <domain type='kvm'> ... <disk type='block' device='cdrom'> <driver name='qemu' type='raw'/> <target dev='hdc' bus='ide'/> <readonly/> </disk> ... </domain>SolutionCorrect the XML by adding the missing <source> parameter as follows:<disk type='block' device='cdrom'> <driver name='qemu' type='raw'/> <source file='/path/to/image.iso'/> <target dev='hdc' bus='ide'/> <readonly/> </disk>A type='block' disk device expects that the source is a physical device. To use the disk with an image file, use type='file' instead.

Rev .8. PXE boot (or DHCP) on guest failed

SymptomA guest virtual machine starts successfully, but is then either unable to acquire an IP address from DHCP or boot using the PXE protocol, or both. There are two common causes of this error: having a long forward delay time set for the bridge, and when the iptablespackage and kernel do not support checksum mangling rules.Long forward delay time on bridgeInvestigationThis is the most common cause of this error. If the guest network interface is connecting to a bridge device that has STP (Spanning Tree Protocol) enabled, as well as a long forward delay set, the bridge will not forward network packets from the guest virtual machine onto the bridge until at least that number of forward delay seconds have elapsed since the guest connected to the bridge. This delay allows the bridge time to watch traffic from the interface and determine the MAC addresses behind it, and prevent forwarding loops in the network topology.If the forward delay is longer than the timeout of the guest's PXE or DHCP client, then the client's operation will fail, and the guest will either fail to boot (in the case of PXE) or fail to acquire an IP address (in the case of DHCP).SolutionIf this is the case, change the forward delay on the bridge to 0, disable STP on the bridge, or both.

Note

This solution applies only if the bridge is not used to connect multiple networks, but just to connect multiple endpoints to a single network (the most common use case for bridges used by libvirt).If the guest has interfaces connecting to a libvirt-managed virtual network, edit the definition for the network, and restart it. For example, edit the default network with the following command:# virsh net-edit defaultAdd the following attributes to the <bridge> element:<name_of_bridge='virbr0' delay='0' stp='on'/>

Note

delay='0' and stp='on' are the default settings for virtual networks, so this step is only necessary if the configuration has been modified from the default.If the guest interface is connected to a host bridge that was configured outside of libvirt, change the delay setting.Add or edit the following lines in the /etc/sysconfig/network-scripts/ifcfg-name_of_bridge file to turn STP on with a 0 second delay:STP=on DELAY=0After changing the configuration file, restart the bridge device:/sbin/ifdown name_of_bridge /sbin/ifup name_of_bridge

Note

If name_of_bridge is not the root bridge in the network, that bridge's delay will eventually reset to the delay time configured for the root bridge. In this case, the only solution is to disable STP completely on name_of_bridge.The iptables package and kernel do not support checksum mangling rulesInvestigationThis message is only a problem if all four of the following conditions are true:The guest is using virtio network devices.If so, the configuration file will contain model type='virtio'The host has the vhost-net module loaded.This is true if ls /dev/vhost-net does not return an empty result.The guest is attempting to get an IP address from a DHCP server that is running directly on the host.The iptables version on the host is older than 1.4.10.iptables 1.4.10 was the first version to add the libxt_CHECKSUM extension. This is the case if the following message appears in the libvirtd logs:warning: Could not add rule to fixup DHCP response checksums on network default warning: May need to update iptables package and kernel to support CHECKSUM rule.

Important

Unless all of the other three conditions in this list are also true, the above warning message can be disregarded, and is not an indicator of any other problems.When these conditions occur, UDP packets sent from the host to the guest have uncomputed checksums. This makes the host's UDP packets seem invalid to the guest's network stack.SolutionTo solve this problem, invalidate any of the four points above. The best solution is to update the host iptables and kernel to iptables-1.4.10 or newer where possible. Otherwise, the most specific fix is to disable the vhost-net driver for this particular guest. To do this, edit the guest configuration with this command:virsh edit name_of_guestChange or add a <driver> line to the <interface> section:<interface type='network'> <model type='virtio'/> <driver name='qemu'/> ... </interface>Save the changes, shut down the guest, and then restart it.If this problem is still not resolved, the issue may be due to a conflict between firewalld and the default libvirt network.To fix this, stop firewalld with the service firewalld stopcommand, then restart libvirt with the service libvirtd restartcommand.

B.9. Guest can reach outside network, but cannot reach host when using macvtap interface

SymptomA guest virtual machine can communicate with other guests, but cannot connect to the host machine after being configured to use a macvtap (also known as type='direct') network interface.InvestigationEven when not connecting to a Virtual Ethernet Port Aggregator (VEPA) or VN-Link capable switch, macvtap interfaces can be useful. Setting the mode of such an interface to bridgeallows the guest to be directly connected to the physical network in a very simple manner without the setup issues (or NetworkManager incompatibility) that can accompany the use of a traditional host bridge device.However, when a guest virtual machine is configured to use a type='direct' network interface such as macvtap, despite having the ability to communicate with other guests and other external hosts on the network, the guest cannot communicate with its own host.This situation is actually not an error — it is the defined behavior of macvtap. Due to the way in which the host's physical Ethernet is attached to the macvtap bridge, traffic into that bridge from the guests that is forwarded to the physical interface cannot be bounced back up to the host's IP stack. Additionally, traffic from the host's IP stack that is sent to the physical interface cannot be bounced back up to the macvtap bridge for forwarding to the guests.SolutionUse libvirt to create an isolated network, and create a second interface for each guest virtual machine that is connected to this network. The host and guests can then directly communicate over this isolated network, while also maintaining compatibility with NetworkManager.

Procedure B.3. Creating an isolated network with libvirt

Add and save the following XML in the /tmp/isolated.xml file. If the 192.168.254.0/24 network is already in use elsewhere on your network, you can choose a different network.<network> <name>isolated</name> <ip address='192.168.254.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.254.2' end='192.168.254.254' /> </dhcp> </ip> </network>Create the network with this command: virsh net-define /tmp/isolated.xmlSet the network to autostart with the virsh net-autostart isolatedcommand.Start the network with the virsh net-start isolated command.Using virsh edit name_of_guest, edit the configuration of each guest that uses macvtap for its network connection and add a new <interface> in the <devices> section similar to the following (note the <model type='virtio'/>line is optional to include):<interface type='network'> <source network='isolated'/> <model type='virtio'/> </interface>Shut down, then restart each of these guests.The guests are now able to reach the host at the address 192.168.254.1, and the host will be able to reach the guests at the IP address they acquired from DHCP (alternatively, you can manually configure the IP addresses for the guests). Since this new network is isolated to only the host and guests, all other communication from the guests will use the macvtap interface.

B.10. Could not add rule to fixup DHCP response checksums on network 'default'

SymptomThis message appears:Could not add rule to fixup DHCP response checksums on network 'default'InvestigationAlthough this message appears to be evidence of an error, it is almost always harmless.SolutionUnless the problem you are experiencing is that the guest virtual machines are unable to acquire IP addresses through DHCP, this message can be ignored.If this is the case, refer to Section B.8, "PXE boot (or DHCP) on guest failed" for further details on this situation.

B.11. Unable to add bridge br0 port vnet0: No such device

SymptomThe following error message appears:Unable to add bridge name_of_bridge port vnet0: No such deviceFor example, if the bridge name is br0, the error message will appear as:Unable to add bridge br0 port vnet0: No such deviceIn libvirt versions 0.9.6 and earlier, the same error appears as:Failed to add tap interface to bridge name_of_bridge: No such deviceOr for example, if the bridge is named br0:Failed to add tap interface to bridge 'br0': No such deviceInvestigationBoth error messages reveal that the bridge device specified in the guest's (or domain's) <interface> definition does not exist.To verify the bridge device listed in the error message does not exist, use the shell command ifconfig br0.A message similar to this confirms the host has no bridge by that name:br0: error fetching interface information: Device not foundIf this is the case, continue to the solution.However, if the resulting message is similar to the following, the issue exists elsewhere:br0 Link encap:Ethernet HWaddr 00:00:5A:11:70:48 inet addr:10.22.1.5 Bcast:10.255.255.255 Mask:255.0.0.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:249841 errors:0 dropped:0 overruns:0 frame:0 TX packets:281948 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:106327234 (101.4 MiB) TX bytes:21182634 (20.2 MiB)SolutionEdit the existing bridge or create a new bridge with virshUse virsh to either edit the settings of an existing bridge or network, or to add the bridge device to the host system configuration.Edit the existing bridge settings using virshUse virsh edit name_of_guest to change the <interface>definition to use a bridge or network that already exists.For example, change type='bridge' to type='network', and <source bridge='br0'/> to <source network='default'/>.Create a host bridge using virshFor libvirt version 0.9.8 and later, a bridge device can be created with thevirsh iface-bridge command. This will create a bridge device br0with eth0, the physical network interface which is set as part of a bridge, attached:virsh iface-bridge eth0 br0Optional: If desired, remove this bridge and restore the original eth0configuration with this command:virsh iface-unbridge br0Create a host bridge manuallyFor older versions of libvirt, it is possible to manually create a bridge device on the host. Refer to Section 11.3, "Bridged networking with libvirt" for instructions.

B.12. Guest is unable to start with error: warning: could not open /dev/net/tun

SymptomThe guest virtual machine does not start after configuring a type='ethernet' (also known as 'generic ethernet') interface in the host system. An error appears either in libvirtd.log, /var/log/libvirt/qemu/name_of_guest.log, or in both, similar to the below message:warning: could not open /dev/net/tun: no virtual network emulation qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could not be initializedInvestigationUse of the generic ethernet interface type (<interface type='ethernet'>) is discouraged, because using it requires lowering the level of host protection against potential security flaws in QEMU and its guests. However, it is sometimes necessary to use this type of interface to take advantage of some other facility that is not yet supported directly in libvirt. For example, openvswitch was not supported in libvirt until libvirt-0.9.11, so in older versions of libvirt, <interface type='ethernet'> was the only way to connect a guest to an openvswitch bridge.However, if you configure a <interface type='ethernet'> interface without making any other changes to the host system, the guest virtual machine will not start successfully.The reason for this failure is that for this type of interface, a script called by QEMU needs to manipulate the tap device. However, with type='ethernet' configured, in an attempt to lock down QEMU, libvirt and SELinux have put in place several checks to prevent this. (Normally, libvirt performs all of the tap device creation and manipulation, and passes an open file descriptor for the tap device to QEMU.)SolutionReconfigure the host system to be compatible with the generic ethernet interface.

Procedure B.4. Reconfiguring the host system to use the generic ethernet interface

Set SELinux to permissive by configuring SELINUX=permissive in /etc/selinux/config:# This file controls the state of SELinux on the system. # SELINUX= can take one of these three values: # enforcing - SELinux security policy is enforced. # permissive - SELinux prints warnings instead of enforcing. # disabled - No SELinux policy is loaded. SELINUX=permissive # SELINUXTYPE= can take one of these two values: # targeted - Targeted processes are protected, # mls - Multi Level Security protection. SELINUXTYPE=targetedFrom a root shell, run the command setenforce permissive.In /etc/libvirt/qemu.conf add or edit the following lines:clear_emulator_capabilities = 0user = "root"group = "root"cgroup_device_acl = [ "/dev/null", "/dev/full", "/dev/zero", "/dev/random", "/dev/urandom", "/dev/ptmx", "/dev/kvm", "/dev/kqemu", "/dev/rtc", "/dev/hpet", "/dev/net/tun",Restart libvirtd.

Important

Since each of these steps significantly decreases the host's security protections against QEMU guest domains, this configuration should only be used if there is no alternative to using <interface type='ethernet'>.

Note

For more information on SELinux, refer to the Red Hat Enterprise Linux 6 Security-Enhanced Linux User Guide.

★☆ Offload.Rx.ChecksumSpecifies the RX checksum offloading mode.In Red Hat Enterprise Linux 6.4 and onward, the valid values for this parameter are All (the default), which enables IP, TCP and UDP checksum offloading for both IPv4 and IPv6; TCP/UDP(v4,v6), which enables TCP and UDP checksum offloading for both IPv4 and IPv6; TCP/UDP(v4), which enables TCP and UDP checksum offloading for IPv4 only; and TCP(v4), which enables only TCP checksum offloading for IPv4 only.In Red Hat Enterprise Linux 6.3 and earlier, the valid values are Disable (the default), which disables RX checksum offloading; All, which enables TCP, UDP, and IP checksum offloading; TCP/UDP, which enables TCP and UDP checksum offloading; and TCP, which enables only TCP checksum offloading.

Test and debug parameters

Important

Test and debug parameters should only be used for testing or debugging; they should not be used in production.TestOnly.DelayConnect(ms)The period for which to delay connection upon startup, in milliseconds. The default value is 0.TestOnly.DPCCheckingSets the DPC checking mode. 0 (the default) disables DPC checking. 1 enables DPC checking; each hang test verifies DPC activity and acts as if the DPC was spawned. 2 clears the device interrupt status and is otherwise identical to 1.TestOnly.Scatter-GatherA Boolean value that determines whether scatter-gather functionality is enabled. The default value is 1 (enabled). Setting this value to 0 disables scatter-gather functionality and all dependent capabilities.TestOnly.InterruptRecoveryA Boolean value that determines whether interrupt recovery is enabled. The default value is 1 (enabled).TestOnly.PacketFilterA Boolean value that determines whether packet filtering is enabled. The default value is 1(enabled).TestOnly.BatchReceiveA Boolean value that determines whether packets are received in batches, or singularly. The default value is 1, which enables batched packet receipt.TestOnly.PromiscuousA Boolean value that determines whether promiscuous mode is enabled. The default value is 0 (disabled).TestOnly.AnalyzeIPPacketsA Boolean value that determines whether the checksum fields of outgoing IP packets are tested and verified for debugging purposes. The default value is 0 (no checking).TestOnly.RXThrottleAn integer that determines the number of receive packets handled in a single DPC. The default value is 1000.TestOnly.UseSwTxChecksumA Boolean value that determines whether hardware checksumming is enabled. The default value is 0 (disabled).

Common libvirt errors and troubleshooting

This appendix documents common libvirt-related problems and errors along with instructions for dealing with them.Locate the error on the table below and follow the corresponding link under Solutionfor detailed troubleshooting information.

Table B.1. Common libvirt errors

ErrorDescription of problemSolutionlibvirtd failed to startThe libvirt daemon failed to start. However, there is no information about this error in /var/log/messages.Section B.1, "libvirtd failed to start"Cannot read CA certificateThis is one of several errors that occur when the URI fails to connect to the hypervisor.Section B.2, "The URI failed to connect to the hypervisor"Failed to connect socket ... : Permission deniedThis is one of several errors that occur when the URI fails to connect to the hypervisor.Section B.2, "The URI failed to connect to the hypervisor"Other connectivity errorsThese are other errors that occur when the URI fails to connect to the hypervisor.Section B.2, "The URI failed to connect to the hypervisor"Internal error guest CPU is not compatible with host CPUThe guest virtual machine cannot be started because the host and guest processors are different.Section B.3, "The guest virtual machine cannot be started: internal error guest CPU is not compatible with host CPU"Failed to create domain from vm.xml error: monitor socket did not show up.: Connection refusedThe guest virtual machine (or domain) starting fails and returns this error or similar.Section B.4, "Guest starting fails with error: monitor socket did not show up"Internal error cannot find character device (null)This error can occur when attempting to connect a guest's console. It reports that there is no serial console configured for the guest virtual machine.Section B.5, "Internal error cannot find character device (null)"No boot deviceAfter building a guest virtual machine from an existing disk image, the guest booting stalls. However, the guest can start successfully using the QEMUcommand directly.Section B.6, "Guest virtual machine booting stalls with error: No boot device"The virtual network "default" has not been startedIf the default network (or other locally-created network) is unable to start, any virtual machine configured to use that network for its connectivity will also fail to start.Section B.7, "Virtual network default has not been started"PXE boot (or DHCP) on guest failedA guest virtual machine starts successfully, but is unable to acquire an IP address from DHCP, boot using the PXE protocol, or both. This is often a result of a long forward delay time set for the bridge, or when the iptables package and kernel do not support checksum mangling rules.Section B.8, "PXE boot (or DHCP) on guest failed"Guest can reach outside network, but cannot reach host when using macvtap interfaceA guest can communicate with other guests, but cannot connect to the host machine after being configured to use a macvtap (or type='direct') network interface.This is actually not an error — it is the defined behavior of macvtap.Section B.9, "Guest can reach outside network, but cannot reach host when using macvtap interface"Could not add rule to fixup DHCP response checksums on network 'default'This warning message is almost always harmless, but is often mistakenly seen as evidence of a problem.Section B.10, "Could not add rule to fixup DHCP response checksums on network 'default'"Unable to add bridge br0 port vnet0: No such deviceThis error message or the similar Failed to add tap interface to bridge 'br0': No such devicereveal that the bridge device specified in the guest's (or domain's) <interface>definition does not exist.Section B.11, "Unable to add bridge br0 port vnet0: No such device"Warning: could not open /dev/net/tun: no virtual network emulation qemu-kvm: -netdev tap,script=/etc/my-qemu-ifup,id=hostnet0: Device 'tap' could not be initializedThe guest virtual machine does not start after configuring a type='ethernet' (or 'generic ethernet') interface in the host system. This error or similar appears either in libvirtd.log, /var/log/libvirt/qemu/name_of_guest.log, or in both.Section B.12, "Guest is unable to start with error: warning: could not open /dev/net/tun"Unable to resolve address name_of_host service '49155': Name or service not knownQEMU guest migration fails and this error message appears with an unfamiliar hostname.Section B.13, "Migration fails with Error: unable to resolve address"Unable to allow access for disk path /var/lib/libvirt/images/qemu.img: No such file or directoryA guest virtual machine cannot be migrated because libvirtcannot access the disk image(s).Section B.14, "Migration fails with Unable to allow access for disk path: No such file or directory"No guest virtual machines are present when libvirtd is startedThe libvirt daemon is successfully started, but no guest virtual machines appear to be present when running virsh list --all.Section B.15, "No guest virtual machines are present when libvirtd is started"Unable to connect to server at 'host:16509': Connection refused ... error: failed to connect to the hypervisorWhile libvirtd should listen on TCP ports for connections, the connection to the hypervisor fails.Section B.16, "Unable to connect to server at 'host:16509': Connection refused ... error: failed to connect to the hypervisor"Common XML errorslibvirt uses XML documents to store structured data. Several common errors occur with XML documents when they are passed to libvirt through the API. This entry provides instructions for editing guest XML definitions, and details common errors in XML syntax and configuration.Section B.17, "Common XML errors"

B.1. libvirtd failed to start

SymptomThe libvirt daemon does not start automatically. Starting the libvirt daemon manually fails as well:# /etc/init.d/libvirtd start * Caching service dependencies ... [ ok ] * Starting libvirtd ... /usr/sbin/libvirtd: error: Unable to initialize network sockets. Check /var/log/messages or run without --daemon for more info. * start-stop-daemon: failed to start `/usr/sbin/libvirtd' [ !! ] * ERROR: libvirtd failed to startMoreover, there is not 'more info' about this error in /var/log/messages.InvestigationChange libvirt's logging in /etc/libvirt/libvirtd.conf by uncommenting the line below. To uncomment the line, open the /etc/libvirt/libvirtd.conf file in a text editor, remove the hash (or #) symbol from the beginning of the following line, and save the change:log_outputs="3:syslog:libvirtd"

Note

This line is commented out by default to prevent libvirt from producing excessive log messages. After diagnosing the problem, it is recommended to comment this line again in the /etc/libvirt/libvirtd.conf file.Restart libvirt to determine if this has solved the problem.If libvirtd still does not start successfully, an error similar to the following will be shown in the /var/log/messages file:Feb 6 17:22:09 bart libvirtd: 17576: info : libvirt version: 0.9.9 Feb 6 17:22:09 bart libvirtd: 17576: error : virNetTLSContextCheckCertFile:92: Cannot read CA certificate '/etc/pki/CA/cacert.pem': No such file or directory Feb 6 17:22:09 bart /etc/init.d/libvirtd[17573]: start-stop-daemon: failed to start `/usr/sbin/libvirtd' Feb 6 17:22:09 bart /etc/init.d/libvirtd[17565]: ERROR: libvirtd failed to startThe libvirtd man page shows that the missing cacert.pem file is used as TLS authority when libvirt is run in Listen for TCP/IP connections mode. This means the --listen parameter is being passed.SolutionConfigure the libvirt daemon's settings with one of the following methods:Install a CA certificate.

Note

For more information on CA certificates and configuring system authentication, refer to the Configuring Authentication chapter in the Red Hat Enterprise Linux 6 Deployment Guide.Do not use TLS; use bare TCP instead. In /etc/libvirt/libvirtd.conf set listen_tls = 0 and listen_tcp = 1. The default values are listen_tls = 1 and listen_tcp = 0.Do not pass the --listen parameter. In /etc/sysconfig/libvirtd.confchange the LIBVIRTD_ARGS variable.

B.2. The URI failed to connect to the hypervisor

Several different errors can occur when connecting to the server (for example, when running virsh).

B.2.1. Cannot read CA certificate

SymptomWhen running a command, the following error (or similar) appears:$ virsh -c name_of_uri list error: Cannot read CA certificate '/etc/pki/CA/cacert.pem': No such file or directory error: failed to connect to the hypervisorInvestigation

No comments:

Post a Comment