How to make Intel GVT-g work

Hello! Intel offered an excellent solution to the perennial problem: "I have a laptop on Linux and I need to run Windows with hardware acceleration, but I do not have a heavy laptop with two GPUs and liquid cooling." Using the architecture of their own GPUs or something else, they managed to make it so that you can split your onboard Intel GPU into two or more GPUs.

Unfortunately, this is by no means so simple ... The documentation is slightly outdated, and some things break in an incomprehensible way without an obvious reason. Therefore, in this post I will tell you how to set up a hardware accelerated Windows virtual machine with cool fast virtio and Intel GVT-g drivers.

To do this, you will need a more or less modern GPU ( Translator's Note : according to official documentation , GVT-g supports integrated graphics cards starting from the fifth generation of Intel Core and from the fourth generation of Xeon ).

Step 1: configure the kernel

Check that you have a fresh version of the kernel. It seems that the options for GVT-g were included before version 4.8, but then they definitely worked worse, so I recommend using the latest available kernel. If you are original enough to build your kernel, enable these options . Also disable the removal of unused ksyms, since this option causes a bug .

Now we need to correct the kernel command line arguments. Important options are:

i915.enable_gvt=1 kvm.ignore_msrs=1 intel_iommu=on i915.enable_guc=0 

Make sure you don’t redefine enable_guc=0 with something that will turn on GuC booting, as this will lead to the depressing crash of the i915 driver. When you boot, log into your work environment and check for the presence of the /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types/ directory. If it does not exist, GVT-g does not work. Check the logs and / or cry in the pillow.

As a solution to the problem, you can add these modules to the initramfs and remove the i915 from there.

For more detailed logging, you can set the drm.debug variable to some value, for example, setting it to 0x02 will include messages from drivers.

Step 2: create a virtual friend

Inside mdev_supported_types you can find a whole set of directories. This set is determined by the amount of your graphics memory, each subdirectory corresponds to a type of virtual GPU. The description file in it contains information about the memory and permissions supported by this virtual GPU. If creating a virtual GPU with large memory using the UUID output in the /create file gives you an incomprehensible error, then you have several options. First, go to the BIOS and add video memory, if possible. If this does not work, you can stop your DM, switch to the framebuffer, create the desired vGPU from there, and then return to x11. Unfortunately, this method leads to many bugs and does not allow to achieve 60 FPS on my laptop. An alternative is to create a smaller vGPU, and use a special resolution increase program (CRU). In this way, I managed to achieve 60 FPS and there were much less bugs and hangs.

You can create vGPU with the following command:

 $ echo ${vGPU_UUID} | sudo tee /sys/bus/pci/devices/0000:00:02.0/mdev_supported_types/${vGPU_TYPE}/create 

And delete - this:

 $ echo 1 | sudo tee /sys/bus/mdev/devices/${vGPU_UUID}/remove 

Translator's Note :
You can generate a UUID for vGPU using the uuidgen command uuidgen no arguments. The variable $ {vGPU_TYPE} denotes one of the types listed in the mdev_supported_types directory. It is also worth noting that vGPU must be recreated every time it is rebooted, they are not saved between OS launches.

Step 3: Cortana screams at you

The next step is a much better supported and significantly slower and more painful thing - installing Windows 10. You should not use torrents or unofficial downloads, or an old version of Windows, the correct link here . It is also worth downloading the virtio driver disk image, designed specifically to speed up guests, here . Install libvirt and virt-manager and run libvirtd :

 # systemctl start libvirtd 

Start virt-manager and make sure that you are connected to the libvirt system session, and not the user session:

Translator's Note :
In order for virt-manager to connect to the system session, you can run it on behalf of the superuser, but it is better to set up authorization, for example, as suggested in the Arch Wiki .

Once you have done this, you can start creating a virtual machine. In the settings dialog, select the download from the local iso-image and find the downloaded image. If virt-manager did not recognize it as a Windows 10 image, select it manually, as this will speed up Windows, since virt-manager in this case provides some Microsoft virtualization interfaces. Create a disk image or LVM partition and configure the configuration as you need. The setup interface is very limited before installation, so I usually start the installation and immediately stop it in order to fully configure everything. Here are some settings:

Translator's Note :
Despite the fact that the post is about setting up a virtual machine on a laptop, for some reason, the question about the distribution of the network using a wireless adapter is omitted. The point is that the default network settings in virt-manager are not suitable for a wireless network. In this situation, an answer to this question and comments on it can help. The author also suggests using BIOS instead of UEFI, probably due to the fact that UEFI requires additional configuration. Also, tianocore does not seem to work with GVT-g yet, see bug 935 . In my case, however, VM started, but Windows did not recognize the monitor to which the integrated video card is connected.

If you need fast disk access or even the ability of a virtual machine to compress a disk image when deleting virtual machine files (TRIM forwarding, you need to create an image with the qemu-img create -f qcow2 -o preallocation=metadata,lazy_refcounts , complete instructions here ), configure the primary disk to use SCSI. The virtual machine will need drivers for Windows to understand this format, so connect the previously downloaded virtio driver disk for Windows 10. By default, they use IDE, but you can speed up the installation many times and use less legacy code if you use it instead for dvd scsi. Windows supports it out of the box. You can also:

• Make USB bus use USB 3.0
• Add spice, spice-webdav and qemu-ga channels for copy and paste and file sharing between the VM and the host
• Remove unused virtual hardware
• Switch the emulated video card to QXL and switch the display to SPICE, NOT listening to the network (even the loopback) ( Translator's note: otherwise the VM simply crashes ).
• Switch the chip type to Q35, the setting is next to the BIOS settings.
• Feed the dog

You can also start getting used to your new virsh edit friend. If you run it with sudo -E , your environment variables, in particular EDITOR, will be used for editing, and you will use the libvirt system session, not the user session. In this file, you can, for example, assign a correspondence between physical and virtual processors, making the processor caches more consistent and the scheduler behaves not so strange. Here is an example of XML that you can put there:

 <vcpu placement='static'>6</vcpu> <cputune> <vcpupin vcpu='0' cpuset='1'/> <vcpupin vcpu='1' cpuset='2'/> <vcpupin vcpu='2' cpuset='3'/> <vcpupin vcpu='3' cpuset='5'/> <vcpupin vcpu='4' cpuset='6'/> <vcpupin vcpu='5' cpuset='7'/> </cputune> <features> <hyperv> <relaxed state='on'/> <vapic state='on'/> <spinlocks state='on' retries='8191'/> <runtime state='on'/> <synic state='on'/> <stimer state='on'/> </hyperv> </features> <cpu mode='host-passthrough' check='none'> <topology sockets='1' cores='3' threads='2'/> </cpu> 

In this fragment, I configure the VM so that it sees a processor with three physical cores, each of which has two hyper-threads. Further, each processor / hyperflow is attached to its own hyperflow, and this correspondence does not change. The Windows scheduler knows about hyper-threads and can use them correctly, not counting them as separate processors. I also include some Hyper-V interfaces that are turned off by default and may have no effect. If you use SPICE, you can add the following lines to disable compression, because the external network is still not used to access the VM.

 <graphics type='spice'> <listen type='none'/> <image compression='off'/> <jpeg compression='never'/> <zlib compression='never'/> <playback compression='off'/> <streaming mode='off'/> <!-- <gl enable='yes' rendernode='/dev/dri/by-path/pci-0000:00:02.0-render'/> пока что это не работает, этот комментарий будет удалён libvirt --> </graphics> 

Now you can customize the boot order and start installing Windows. If you are using virtio or a SCSI disk, Windows will not find it. You will need to install the SCSI driver from the disk that you connected, they are in the virtscsi/amd64 directory. Everything should go smoothly, and Windows should boot in slow and wretched, not accelerated mode. Cortana will start screaming at you, and your network will not work. Break through it all to the desktop. There, launch the device manager, find all unidentified devices, and update the drivers for them from the disk that you connected. You will get some faster windows.

Step 4: The fun part

There are three ways to get an accelerated virtual VM display with Windows on the screen of your machine.

• VNC or some other remote access protocol (usually a very bad solution). In this version, you only need to connect the vGPU and disconnect all other displays and video cards. Also set the display='off' setting to display='off' . You do not need the igd-opregion , shown later.
• SPICE (I didn’t manage to achieve 30 FPS or higher, but the shared clipboard and file transfer between the VM and the host are working).
• Built-in QEMU interface on GTK + (shared clipboard and file transfer do not work, but you can achieve 60 FPS using a patch).

Whatever you are going to use, you still have to use the second option to install the drivers for the GPU. Microsoft's built-in drivers do not work very well with GVT-g at the time of writing the post, and often break. Before you connect the vGPU to the VM, it is advisable to download the latest version of the driver from Intel (Apparently, Intel changes the approach to distributing the drivers , so in the future this step may be different, or it may not be necessary at all). Now make sure you have vGPU created. Open virt-manager and replace good fast QXL with slow Cirrus to avoid conflicts. To connect a vGPU to a VM, you need to open virsh edit and add the following fragment somewhere:

 <domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'> <hostdev mode='subsystem' type='mdev' managed='no' model='vfio-pci' display='on'> 

 <address uuid='fff6f017-3417-4ad3-b05e-17ae3e1a4615'/> 

  <address type='pci' domain='0x0000' bus='0x00' slot='0x09' function='0x0'/> <rom enabled='no'/> </hostdev> <graphics type='spice'> <listen type='none'/> <image compression='off'/> <jpeg compression='never'/> <zlib compression='never'/> <playback compression='off'/> <streaming mode='off'/> <gl enable='yes' rendernode='/dev/dri/by-path/pci-0000:00:02.0-render'/> <!-- Ваш путь к устройству может отличаться, virt-manager может проставить это за вас, если вы отметите чекбокс с аппаратным ускорением GL. Убедитесь, что тут прописан путь к устройству, а не auto. --> </graphics> <qemu:commandline> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.x-igd-opregion=on'/> <!-- libvirt пока не поддерживает эту опцию, поэтому её надо выставить вручную --> </qemu:commandline> </domain> 

Note: When I provide an XML fragment like this, you should, if possible, add it to the current one, without replacing anything.

Check that you have created unique UUIDs for all the vGPUs you are using, and that the slot numbers do not conflict with any of the other PCI devices. If the slot number is after the Cirrus GPU, the virtual machine will fall. Now you can start the virtual machine. You need to install virt-viewer to see both displays! You can connect to the VM using the command

 $ sudo -E virt-viewer --attach 

One of the displays will be empty or not initialized, the second will be the familiar little non-accelerated display. Open it and, upon entering, install the driver for the GPU. If you are lucky, everything will work right away. Otherwise, it is necessary to turn off and restart the VM (do not reboot) using the running screen. Now it's time to open the terminal and run dmesg -w inside. This command will give you some useful information about the problems and the general progress of work using vGPU. For example, when booting, KVM will complain about blocked MSRs, then you should receive several messages about incorrect access when vGPU is initialized. If there are too many of them, something is wrong.

If the system is booted, you can open the display settings and turn off the non-accelerated screen. A blank screen can be hidden through the virt-viewer View menu. In principle, the VM can already be used, but there are a couple more things that can be done to achieve higher resolution and higher speed.

The CRU utility is quite useful. You can play with it, and even if you stumble upon some kind of graphical artifacts or even an almost completely black screen, as I did, you can run Restart64.exe , which comes with the program, to restart the Windows graphics subsystem. I personally use this utility to use a higher resolution on a more modest vGPU.

To achieve excellent 60 FPS, you need to switch to the built-in QEMU monitor on GTK + without supporting the common clipboard with the host and similar buns, and also change one line in it and rebuild QEMU. You will also need to add a bunch of nasty command line arguments to your XML. Remove the SPICE display and the Cirrus video card and set the display attribute on your vGPU to off (libvirt does not support the display on GTK + and will not allow to boot from display='on' without display).

  <qemu:commandline> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.x-igd-opregion=on'/> <qemu:arg value='-set'/> <qemu:arg value='device.hostdev0.display=on'/> <qemu:arg value='-display'/> <qemu:arg value='gtk,gl=on'/> <qemu:env name='DISPLAY' value=':1'/> <qemu:env name='GDK_SCALE' value='1.0'/> </qemu:commandline> 

Scaling for HiDPI at the QEMU monitor works very badly, so we will disable it. Also, you will need to set the DISPLAY variable to the number of the display you are using. To give the user running qemu access rights to the X server, use the command:

 # xhost si:localuser:nobody 

If that doesn't work, try xhost + , but make sure you use a firewall. Otherwise, try a more secure method.

With such tricks, you still will not get above 30 FPS because of this stupid bug in QEMU, unless you patch it by changing the line, as indicated in the comments link. Make sure you build only the QEMU for x86-64, unless you are going to use it on another platform. I attached my PKGBUILD, which does not change the line, but only collects QEMU for x86_64 without support for network storage here .

If you get lost somewhere along the way, you can view my current xml for libvirt .

useful links

Official GVT-g Setup Guide
Dma-buf User Guide
An article on configuring Intel GVT-g on the NixOS Wiki
An article on the Arch Wiki about libvirt
Network configuration in KVM on the wireless interface
Intel GVT-g dedicated website

PS: Thank you aNNiMON for help in reading the translation text and correcting errors.