Archives
/
vfio-guide-2
mirror of https://gitlab.com/YuriAlek/vfio
2
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Less sleep

Browse Source
testing-auto
yurialek 6 years ago
parent d6a1f7d1c6
commit 49f23ddea7
6 changed files with 33 additions and 94 deletions
Show Stats Download Patch File Download Diff File

2
Hardware configurations/README.md
Unescape Escape View File

@ -38,7 +38,7 @@ Ryzen 5 2600 - GTX 770 - @YuriAlek.md
### How I did it
For extracting the `vBIOS` I used the 1st method in Linux and edited it.
You can add anything that you consider useful like the step needed in your distro for installing everything. The more information, and solutions, the better.
You can add anything that you consider useful like the steps needed in your distro for installing everything or a link to your script. The more information, and solutions, the better.
### Files to modify
#### `scripts/windows.sh`

12
Install.md
Unescape Escape View File

@ -6,7 +6,7 @@
```
### Enabling IOMMU support on boot
For `systemd-boot` edit `/boot/loader/entries/arch.conf` and add `intel_iommu=on` **OR** `amd_iommu=on` and `iommu=pt`.
#### For `systemd-boot` edit `/boot/loader/entries/arch.conf` and add `intel_iommu=on` **OR** `amd_iommu=on` and `iommu=pt`.
```
# AMD
options root=/dev/sda2 amd_iommu=on iommu=pt
@ -15,8 +15,8 @@ options root=/dev/sda2 amd_iommu=on iommu=pt
options root=/dev/sda2 intel_iommu=on iommu=pt
```
Reboot.
----
For `GRUB` edit `/etc/default/grub` and append your kernel options to the `GRUB_CMDLINE_LINUX_DEFAULT`.
#### For `GRUB` edit `/etc/default/grub` and append your kernel options, `intel_iommu=on` **OR** `amd_iommu=on` and `iommu=pt`, to the `GRUB_CMDLINE_LINUX_DEFAULT`.
```
# AMD
GRUB_CMDLINE_LINUX_DEFAULT="amd_iommu=on iommu=pt"
@ -29,10 +29,10 @@ And then automatically re-generate the grub.cfg file with:
# grub-mkconfig -o /boot/grub/grub.cfg
```
Reboot.
----
After reboot IOMMU should be working
After reboot IOMMU should be working:
```
[yu@ryzen ~]$ dmesg | grep -e DMAR -e IOMMU
$ dmesg | grep -e DMAR -e IOMMU
--------------------------------------------
[ 0.492684] AMD-Vi: IOMMU performance counters supported
[ 0.494370] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40

23
README.md
Unescape Escape View File

@ -13,20 +13,23 @@
7. [Known problems](#known-problems)
8. [TODO](#todo)
## Last Update
2018/10/20
## What this does
In one command it kills X, frees the GPU from drivers and console, detaches the GPU from the host, starts the VM with the GPU, waits until the VM is off, reattaches the GPU to the host and starts lightdm.
## What you need
* An IOMMU enabled motherboard. Check your motherboard manual for an option in BIOS to enable IOMMU.
* CPU support for AMD-v/VT-x and AMD-Vi/VT-d (AMD/Intel).
* Virtualization support enabled on BIOS; check your motherboard manual if you need help.
* Virtualization support enabled on BIOS. Check your motherboard manual if you need help.
* One GPU that supports UEFI and its BIOS. All GPUs from 2012 and later should support this, some may have issues. If the GPU does not support UEFI you may be able to make it work, but you won't see anything in the screen until the drivers inside Windows kick in.
* QEMU, OVMF UEFI and VIRTIO drivers for Windows. [If you need to install, refer to the Install guide](Install.md)
* [Optional] HDD only for Windows
* [Optional] An hard drive only for Windows.
* [Recommended] Another computer to login remotely with `ssh` for convenience, at least until you have everything working.
## Submit your own configuration
[Create a pull merge](https://gitlab.com/YuriAlek/vfio/merge_requests/new) with [a file explaining how you got it working](Hardware configurations/README.md)
[Create a pull merge](https://gitlab.com/YuriAlek/vfio/merge_requests/new) with [a file explaining how you got it working](Hardware configurations/README.md).
## My system
```
@ -42,14 +45,14 @@ In one command it kills X, frees the GPU from drivers and console, detaches the
[Software]
Linux Distro: ArchLinux
Linux Kernel: 4.17.14 vanilla
Linux Kernel: 4.17.14
Nvidia divers: 396.51-1
QEMU version: 2.12.1-1
OVMF version: r24021
[Guests]
Windows: Windows 10 Pro 1709 x64
MacOS: MacOS High Sierra 10.13.3
Windows 10 Pro 1709 x64
MacOS High Sierra 10.13.3
```
## vBIOS
@ -119,6 +122,7 @@ $ qemu-img create -f raw /path/to/image/windows.raw 60G
```
6. Edit the config in `scripts/config.sh` to convenience. If you use systemd to start the VM you have to edit `EnvironmentFile` in `qemu@.service` to point to your config file. Variables you may have to edit:
1. PCI devices. `IOMMU_GPU`; `IOMMU_USB`.
2. User.
3. Location of HDD/IMG, ISO, vBIOS and OVMF image.
@ -135,6 +139,7 @@ $ qemu-img create -f raw /path/to/image/windows.raw 60G
```
8. When installing Windows, in the section `Where do you want to install Windows?` there will be no hard drives to install to; to fix it:
1. Load driver
2. Browse
3. CD Drive (E:) virtio-win-0.1.1
@ -192,17 +197,17 @@ Windows 10 Pro 1709 works for me, but 1803 does not (may be the UEFI). [I have h
## TODO
- [x] Unbind GPU without `virsh`.
- [x] Update macos script.
- [x] Run QEMU as user.
- [x] Try if is necessary to edit `/etc/mkinitcpio.conf`. No need to load the kernel modules at boot.
- [x] Extract the vBIOS in Linux.
- [x] Install guide.
- [ ] Network.
- [x] Fix the race condition.
- [ ] Network. [I am too lazy, check this](https://yurialek.gitlab.io/gitbook/docs/qemu/network.html). I will add it to the scripts at some point.
- [ ] Update macos script.
- [ ] IOMMU guide.
- [ ] Audio.
- [ ] Troubleshooting guide.
- [ ] How to edit the `windows.sh` script.
- [ ] Fix the race condition.
- [ ] Create scripts for install and use (Without DVD images).
- [ ] Improve the script with multiple options for HDD/IMG; network; PCI devices, etc.
- [ ] ACS Patch (Does not work for me).

0
scripts/iommu.sh
Unescape Escape View File

43
scripts/windows-install.sh
Unescape Escape View File

@ -10,8 +10,8 @@ fi
source config
# Memory lock limit
if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != 12884900 ]; then
ulimit -l 12884900
if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != $(( $(echo $RAM | tr -d 'G')*1048576+10 )) ]; then
ulimit -l $(( $(echo $RAM | tr -d 'G')*1048576+10 ))
fi
## Kill X and related
@ -21,58 +21,35 @@ sleep 2
# Kill the console to free the GPU
echo 0 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 0 > /sys/class/vtconsole/vtcon1/bind
sleep 1
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
sleep 1
# Unload the Kernel Modules that use the GPU
modprobe -r nvidia_drm
sleep 1
modprobe -r nvidia_modeset
sleep 1
modprobe -r nvidia
sleep 1
modprobe -r snd_hda_intel
sleep 2
# Load the kernel module
modprobe vfio
sleep 1
modprobe vfio_iommu_type1
sleep 1
modprobe vfio-pci
sleep 1
# Detach the GPU from drivers and attach to vfio. Also the usb.
echo $videoid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $videobusid > /sys/bus/pci/devices/$videobusid/driver/unbind
sleep 1
echo $videobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $videoid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $audiobusid > /sys/bus/pci/devices/$audiobusid/driver/unbind
sleep 1
echo $audiobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
sleep 1
echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id
#ls -la /sys/bus/pci/devices/$usbbusid/
sleep 1
# QEMU (VM) command
qemu-system-x86_64 -runas $USER -enable-kvm \
@ -99,31 +76,23 @@ qemu-system-x86_64 -runas $USER -enable-kvm \
# Wait for QEMU to finish before continue
wait
sleep 5
sleep 1
# Unload the vfio module. I am lazy, this leaves the GPU without drivers
modprobe -r vfio-pci
sleep 2
modprobe -r vfio_iommu_type1
sleep 2
modprobe -r vfio
sleep 2
# Reload the kernel modules. This loads the drivers for the GPU
modprobe snd_hda_intel
sleep 5
modprobe nvidia_drm
sleep 2
modprobe nvidia_modeset
sleep 2
modprobe nvidia
sleep 5
# Bind the usb
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/new_id
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind
sleep 10
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/remove_id
#ls -la /sys/bus/pci/devices/$usbbusid/
@ -132,13 +101,11 @@ sleep 10
echo 1 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 1 > tee /sys/class/vtconsole/vtcon1/bind
sleep 5
#echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
#sleep 1
sleep 1
# Reload the Display Manager to access X
systemctl start lightdm
sleep 5
sleep 2
# Restore the Frame Buffer
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind

47
scripts/windows.sh
Unescape Escape View File

@ -7,11 +7,11 @@ if [ "$EUID" -ne 0 ]
fi
# END Check if you are sudo
source /home/yu/scripts/config
source config
# Memory lock limit
if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != 12884900 ]; then
ulimit -l 12884900
# Memory lock limit.
if [ $(ulimit -a | grep "max locked memory" | awk '{print $6}') != $(( $(echo $RAM | tr -d 'G')*1048576+10 )) ]; then
ulimit -l $(( $(echo $RAM | tr -d 'G')*1048576+10 ))
fi
## Kill X and related
@ -21,58 +21,35 @@ sleep 2
# Kill the console to free the GPU
echo 0 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 0 > /sys/class/vtconsole/vtcon1/bind
sleep 1
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
sleep 1
# Unload the Kernel Modules that use the GPU
modprobe -r nvidia_drm
sleep 1
modprobe -r nvidia_modeset
sleep 1
modprobe -r nvidia
sleep 1
modprobe -r snd_hda_intel
sleep 2
# Load the kernel module
modprobe vfio
sleep 1
modprobe vfio_iommu_type1
sleep 1
modprobe vfio-pci
sleep 1
# Detach the GPU from drivers and attach to vfio. Also the usb.
echo $videoid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $videobusid > /sys/bus/pci/devices/$videobusid/driver/unbind
sleep 1
echo $videobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $videoid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $audiobusid > /sys/bus/pci/devices/$audiobusid/driver/unbind
sleep 1
echo $audiobusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $audioid > /sys/bus/pci/drivers/vfio-pci/remove_id
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/new_id
sleep 1
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
sleep 1
echo $usbbusid > /sys/bus/pci/drivers/vfio-pci/bind
sleep 1
echo $usbid > /sys/bus/pci/drivers/vfio-pci/remove_id
#ls -la /sys/bus/pci/devices/$usbbusid/
sleep 1
# QEMU (VM) command
qemu-system-x86_64 -runas $USER -enable-kvm \
@ -95,31 +72,23 @@ qemu-system-x86_64 -runas $USER -enable-kvm \
# Wait for QEMU to finish before continue
wait
sleep 5
sleep 1
# Unload the vfio module. I am lazy, this leaves the GPU without drivers
modprobe -r vfio-pci
sleep 2
modprobe -r vfio_iommu_type1
sleep 2
modprobe -r vfio
sleep 2
# Reload the kernel modules. This loads the drivers for the GPU
modprobe snd_hda_intel
sleep 5
modprobe nvidia_drm
sleep 2
modprobe nvidia_modeset
sleep 2
modprobe nvidia
sleep 5
# Bind the usb
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/new_id
echo $usbbusid > /sys/bus/pci/devices/$usbbusid/driver/unbind
echo $usbbusid > /sys/bus/pci/drivers/xhci_hcd/bind
sleep 10
#echo $usbid > /sys/bus/pci/drivers/xhci_hcd/remove_id
#ls -la /sys/bus/pci/devices/$usbbusid/
@ -128,13 +97,11 @@ sleep 10
echo 1 > /sys/class/vtconsole/vtcon0/bind
sleep 1
echo 1 > tee /sys/class/vtconsole/vtcon1/bind
sleep 5
#echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
#sleep 1
sleep 1
# Reload the Display Manager to access X
systemctl start lightdm
sleep 5
sleep 2
# Restore the Frame Buffer
echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind

Write Preview
Loading…
Cancel
Save