From 49f23ddea74e0f977bb1556548f8199560593020 Mon Sep 17 00:00:00 2001 From: yurialek Date: Sat, 20 Oct 2018 18:46:51 +0200 Subject: [PATCH] Less sleep --- Hardware configurations/README.md | 2 +- Install.md | 12 ++++---- README.md | 23 +++++++++------ scripts/iommu.sh | 0 scripts/windows-install.sh | 43 ++++------------------------ scripts/windows.sh | 47 +++++-------------------------- 6 files changed, 33 insertions(+), 94 deletions(-) mode change 100644 => 100755 scripts/iommu.sh diff --git a/Hardware configurations/README.md b/Hardware configurations/README.md index e39f581..a8009de 100644 --- a/Hardware configurations/README.md +++ b/Hardware configurations/README.md @@ -38,7 +38,7 @@ Ryzen 5 2600 - GTX 770 - @YuriAlek.md ### How I did it For extracting the `vBIOS` I used the 1st method in Linux and edited it. -You can add anything that you consider useful like the step needed in your distro for installing everything. The more information, and solutions, the better. +You can add anything that you consider useful like the steps needed in your distro for installing everything or a link to your script. The more information, and solutions, the better. ### Files to modify #### `scripts/windows.sh` diff --git a/Install.md b/Install.md index 653942b..5f01375 100644 --- a/Install.md +++ b/Install.md @@ -6,7 +6,7 @@ ``` ### Enabling IOMMU support on boot -For `systemd-boot` edit `/boot/loader/entries/arch.conf` and add `intel_iommu=on` **OR** `amd_iommu=on` and `iommu=pt`. +#### For `systemd-boot` edit `/boot/loader/entries/arch.conf` and add `intel_iommu=on` **OR** `amd_iommu=on` and `iommu=pt`. ``` # AMD options root=/dev/sda2 amd_iommu=on iommu=pt @@ -15,8 +15,8 @@ options root=/dev/sda2 amd_iommu=on iommu=pt options root=/dev/sda2 intel_iommu=on iommu=pt ``` Reboot. ----- -For `GRUB` edit `/etc/default/grub` and append your kernel options to the `GRUB_CMDLINE_LINUX_DEFAULT`. + +#### For `GRUB` edit `/etc/default/grub` and append your kernel options, `intel_iommu=on` **OR** `amd_iommu=on` and `iommu=pt`, to the `GRUB_CMDLINE_LINUX_DEFAULT`. ``` # AMD GRUB_CMDLINE_LINUX_DEFAULT="amd_iommu=on iommu=pt" @@ -29,10 +29,10 @@ And then automatically re-generate the grub.cfg file with: # grub-mkconfig -o /boot/grub/grub.cfg ``` Reboot. ----- -After reboot IOMMU should be working + +After reboot IOMMU should be working: ``` -[yu@ryzen ~]$ dmesg | grep -e DMAR -e IOMMU +$ dmesg | grep -e DMAR -e IOMMU -------------------------------------------- [ 0.492684] AMD-Vi: IOMMU performance counters supported [ 0.494370] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40 diff --git a/README.md b/README.md index b881644..171cd7a 100644 --- a/README.md +++ b/README.md @@ -13,20 +13,23 @@ 7. [Known problems](#known-problems) 8. [TODO](#todo) +## Last Update +2018/10/20 + ## What this does In one command it kills X, frees the GPU from drivers and console, detaches the GPU from the host, starts the VM with the GPU, waits until the VM is off, reattaches the GPU to the host and starts lightdm. ## What you need * An IOMMU enabled motherboard. Check your motherboard manual for an option in BIOS to enable IOMMU. * CPU support for AMD-v/VT-x and AMD-Vi/VT-d (AMD/Intel). -* Virtualization support enabled on BIOS; check your motherboard manual if you need help. +* Virtualization support enabled on BIOS. Check your motherboard manual if you need help. * One GPU that supports UEFI and its BIOS. All GPUs from 2012 and later should support this, some may have issues. If the GPU does not support UEFI you may be able to make it work, but you won't see anything in the screen until the drivers inside Windows kick in. * QEMU, OVMF UEFI and VIRTIO drivers for Windows. [If you need to install, refer to the Install guide](Install.md) -* [Optional] HDD only for Windows +* [Optional] An hard drive only for Windows. * [Recommended] Another computer to login remotely with `ssh` for convenience, at least until you have everything working. ## Submit your own configuration -[Create a pull merge](https://gitlab.com/YuriAlek/vfio/merge_requests/new) with [a file explaining how you got it working](Hardware configurations/README.md) +[Create a pull merge](https://gitlab.com/YuriAlek/vfio/merge_requests/new) with [a file explaining how you got it working](Hardware configurations/README.md). ## My system ``` @@ -42,14 +45,14 @@ In one command it kills X, frees the GPU from drivers and console, detaches the [Software] Linux Distro: ArchLinux - Linux Kernel: 4.17.14 vanilla + Linux Kernel: 4.17.14 Nvidia divers: 396.51-1 QEMU version: 2.12.1-1 OVMF version: r24021 [Guests] - Windows: Windows 10 Pro 1709 x64 - MacOS: MacOS High Sierra 10.13.3 + Windows 10 Pro 1709 x64 + MacOS High Sierra 10.13.3 ``` ## vBIOS @@ -119,6 +122,7 @@ $ qemu-img create -f raw /path/to/image/windows.raw 60G ``` 6. Edit the config in `scripts/config.sh` to convenience. If you use systemd to start the VM you have to edit `EnvironmentFile` in `qemu@.service` to point to your config file. Variables you may have to edit: + 1. PCI devices. `IOMMU_GPU`; `IOMMU_USB`. 2. User. 3. Location of HDD/IMG, ISO, vBIOS and OVMF image. @@ -135,6 +139,7 @@ $ qemu-img create -f raw /path/to/image/windows.raw 60G ``` 8. When installing Windows, in the section `Where do you want to install Windows?` there will be no hard drives to install to; to fix it: + 1. Load driver 2. Browse 3. CD Drive (E:) virtio-win-0.1.1 @@ -192,17 +197,17 @@ Windows 10 Pro 1709 works for me, but 1803 does not (may be the UEFI). [I have h ## TODO - [x] Unbind GPU without `virsh`. -- [x] Update macos script. - [x] Run QEMU as user. - [x] Try if is necessary to edit `/etc/mkinitcpio.conf`. No need to load the kernel modules at boot. - [x] Extract the vBIOS in Linux. - [x] Install guide. -- [ ] Network. +- [x] Fix the race condition. +- [ ] Network. [I am too lazy, check this](https://yurialek.gitlab.io/gitbook/docs/qemu/network.html). I will add it to the scripts at some point. +- [ ] Update macos script. - [ ] IOMMU guide. - [ ] Audio. - [ ] Troubleshooting guide. - [ ] How to edit the `windows.sh` script. -- [ ] Fix the race condition. - [ ] Create scripts for install and use (Without DVD images). - [ ] Improve the script with multiple options for HDD/IMG; network; PCI devices, etc. - [ ] ACS Patch (Does not work for me). diff --git a/scripts/iommu.sh b/scripts/iommu.sh old mode 100644 new mode 100755 diff --git a/scripts/windows-install.sh b/scripts/windows-install.sh index 546e403..c78a1de 100644 --- a/scripts/windows-install.sh +++ b/scripts/windows-install.sh @@ -10,8 +10,8 @@ fi source config # Memory lock limit -if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != 12884900 ]; then - ulimit -l 12884900 +if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != $(( $(echo $RAM | tr -d 'G')*1048576+10 )) ]; then + ulimit -l $(( $(echo $RAM | tr -d 'G')*1048576+10 )) fi ## Kill X and related @@ -21,58 +21,35 @@ sleep 2 # Kill the console to free the GPU echo 0 > /sys/class/vtconsole/vtcon0/bind -sleep 1 echo 0 > /sys/class/vtconsole/vtcon1/bind -sleep 1 echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind -sleep 1 # Unload the Kernel Modules that use the GPU modprobe -r nvidia_drm -sleep 1 modprobe -r nvidia_modeset -sleep 1 modprobe -r nvidia -sleep 1 modprobe -r snd_hda_intel -sleep 2 # Load the kernel module modprobe vfio -sleep 1 modprobe vfio_iommu_type1 -sleep 1 modprobe vfio-pci -sleep 1 # Detach the GPU from drivers and attach to vfio. Also the usb. echo $videoid > /sys/bus/pci/drivers/vfio-pci/new_id -sleep 1 echo $videobusid > /sys/bus/pci/devices/$videobusid/driver/unbind -sleep 1 echo $videobusid > /sys/bus/pci/drivers/vfio-pci/bind -sleep 1 echo $videoid > /sys/bus/pci/drivers/vfio-pci/remove_id -sleep 1 echo $audioid > /sys/bus/pci/drivers/vfio-pci/new_id -sleep 1 echo $audiobusid > /sys/bus/pci/devices/$audiobusid/driver/unbind -sleep 1 echo $audiobusid > /sys/bus/pci/drivers/vfio-pci/bind -sleep 1 echo $audioid > /sys/bus/pci/drivers/vfio-pci/remove_id -sleep 1 echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id -sleep 1 echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind -sleep 1 echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind -sleep 1 echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id -#ls -la /sys/bus/pci/devices/$usbbusid/ -sleep 1 # QEMU (VM) command qemu-system-x86_64 -runas $USER -enable-kvm \ @@ -99,31 +76,23 @@ qemu-system-x86_64 -runas $USER -enable-kvm \ # Wait for QEMU to finish before continue wait -sleep 5 +sleep 1 # Unload the vfio module. I am lazy, this leaves the GPU without drivers modprobe -r vfio-pci -sleep 2 modprobe -r vfio_iommu_type1 -sleep 2 modprobe -r vfio -sleep 2 # Reload the kernel modules. This loads the drivers for the GPU modprobe snd_hda_intel -sleep 5 modprobe nvidia_drm -sleep 2 modprobe nvidia_modeset -sleep 2 modprobe nvidia -sleep 5 # Bind the usb #echo $usbid > /sys/bus/pci/drivers/xhci_hcd/new_id echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind -sleep 10 #echo $usbid > /sys/bus/pci/drivers/xhci_hcd/remove_id #ls -la /sys/bus/pci/devices/$usbbusid/ @@ -132,13 +101,11 @@ sleep 10 echo 1 > /sys/class/vtconsole/vtcon0/bind sleep 1 echo 1 > tee /sys/class/vtconsole/vtcon1/bind -sleep 5 -#echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind -#sleep 1 +sleep 1 # Reload the Display Manager to access X systemctl start lightdm -sleep 5 +sleep 2 # Restore the Frame Buffer echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind diff --git a/scripts/windows.sh b/scripts/windows.sh index 6b35870..f53446c 100644 --- a/scripts/windows.sh +++ b/scripts/windows.sh @@ -7,11 +7,11 @@ if [ "$EUID" -ne 0 ] fi # END Check if you are sudo -source /home/yu/scripts/config +source config -# Memory lock limit -if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != 12884900 ]; then - ulimit -l 12884900 +# Memory lock limit. +if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != $(( $(echo $RAM | tr -d 'G')*1048576+10 )) ]; then + ulimit -l $(( $(echo $RAM | tr -d 'G')*1048576+10 )) fi ## Kill X and related @@ -21,58 +21,35 @@ sleep 2 # Kill the console to free the GPU echo 0 > /sys/class/vtconsole/vtcon0/bind -sleep 1 echo 0 > /sys/class/vtconsole/vtcon1/bind -sleep 1 echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind -sleep 1 # Unload the Kernel Modules that use the GPU modprobe -r nvidia_drm -sleep 1 modprobe -r nvidia_modeset -sleep 1 modprobe -r nvidia -sleep 1 modprobe -r snd_hda_intel -sleep 2 # Load the kernel module modprobe vfio -sleep 1 modprobe vfio_iommu_type1 -sleep 1 modprobe vfio-pci -sleep 1 # Detach the GPU from drivers and attach to vfio. Also the usb. echo $videoid > /sys/bus/pci/drivers/vfio-pci/new_id -sleep 1 echo $videobusid > /sys/bus/pci/devices/$videobusid/driver/unbind -sleep 1 echo $videobusid > /sys/bus/pci/drivers/vfio-pci/bind -sleep 1 echo $videoid > /sys/bus/pci/drivers/vfio-pci/remove_id -sleep 1 echo $audioid > /sys/bus/pci/drivers/vfio-pci/new_id -sleep 1 echo $audiobusid > /sys/bus/pci/devices/$audiobusid/driver/unbind -sleep 1 echo $audiobusid > /sys/bus/pci/drivers/vfio-pci/bind -sleep 1 echo $audioid > /sys/bus/pci/drivers/vfio-pci/remove_id -sleep 1 echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id -sleep 1 echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind -sleep 1 echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind -sleep 1 echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id -#ls -la /sys/bus/pci/devices/$usbbusid/ -sleep 1 # QEMU (VM) command qemu-system-x86_64 -runas $USER -enable-kvm \ @@ -95,31 +72,23 @@ qemu-system-x86_64 -runas $USER -enable-kvm \ # Wait for QEMU to finish before continue wait -sleep 5 +sleep 1 # Unload the vfio module. I am lazy, this leaves the GPU without drivers modprobe -r vfio-pci -sleep 2 modprobe -r vfio_iommu_type1 -sleep 2 modprobe -r vfio -sleep 2 # Reload the kernel modules. This loads the drivers for the GPU modprobe snd_hda_intel -sleep 5 modprobe nvidia_drm -sleep 2 modprobe nvidia_modeset -sleep 2 modprobe nvidia -sleep 5 # Bind the usb #echo $usbid > /sys/bus/pci/drivers/xhci_hcd/new_id echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind -sleep 10 #echo $usbid > /sys/bus/pci/drivers/xhci_hcd/remove_id #ls -la /sys/bus/pci/devices/$usbbusid/ @@ -128,13 +97,11 @@ sleep 10 echo 1 > /sys/class/vtconsole/vtcon0/bind sleep 1 echo 1 > tee /sys/class/vtconsole/vtcon1/bind -sleep 5 -#echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind -#sleep 1 +sleep 1 # Reload the Display Manager to access X systemctl start lightdm -sleep 5 +sleep 2 # Restore the Frame Buffer echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind