How To: Debug the WRK on Mac OS X Using VirtualBox

Windows Research Kernel @ HPI

This article describes how to configure VirtualBox on a Mac OS X system that allows both developing and debugging the WRK at the same time.

Working with the WRK usually requires at least two machines: The development machine, where all the source and the build environment are located, and second, the test machine, where you run the modified kernel. This is in particular necessary when the newly built kernel has bugs and requires improvement.

Thanks to virtualization technologies, our students use the following setup: They use their normal Windows workstation as development environment and deploy the test machine in a virtual machine. On Windows systems, both VMware and Microsoft provide suitable virtualization technologies.

Usual setup when working with the WRK on Windows operating systems.
Figure 1: Usual setup when working with the WRK on Windows operating systems.

However, some students do not want to relinquish their beloved Mac OS, which is why we were looking for alternative approaches. On Apple systems, two major problems arise: First, early virtualization vendors, i.e. Parallels and later VMware, require a fee for their license. Second, using Mac OS also requires to virtualize the development system, which means that both the development virtual machine and the test virtual machine run in parallel on the same host. So, if performance matters, we would not recommend using our approach. For simple debugging sessions, we propose the following architecture:

Proposed setup on Mac OS X operating systems
Figure 2: Proposed setup on Mac OS X operating systems.

To overcome the license fee, we suggest using VirtualBox (VB) from Sun, that runs quite well on Mac OS. We then created two virtual machines: (1) the WRK Guest system, which serves as test environment, and (2) the Debugger Guest that hosts the build environment as well as the kernel debugger. While the WRK Guest obviously has to be Windows Server 2003 Enterprise Edition with Service Pack 1, it's up to you what Windows version to use. In this article we are using on both machines Windows Server 2003 as it allows for faster VM creation.

Also, please note that it is not a good idea to migrate the virtual machine from one VM vendor to VirtualBox. Although Sun provides some hints for the migration, you may be faster with installing a fresh OS in VirtualBox.

Now, in order to let the Debugger Guest talk to the WRK Guest, we first need to create serial ports for both VMs. In order to let both machines communicate, choose the option Host-Pipe when creating the serial connection. This means that VB creates a named pipe on the host system and both VMs connect to that pipe for communication. Figure 3 shows the serial port configuration for the Debugger Guest, which we decided should create the pipe upon startup.

Configuration for the serial port in the Debugger Guest VM.
Figure 3: Configuration for the serial port in the Debugger Guest VM.

Figure 4 shows the serial port configuration for the WRK Guest, which differs only in the check-box "Create Pipe". After configuring the serial ports, start both virtual machines.

Configuration for the serial port in the WRK Guest VM.
Figure 4: Configuration for the serial port in the WRK Guest VM.

Unfortunately, according to a VB bug tracker entry (thanks to Jan Brunnert), there are known issues on creating a serial port on Windows Server 2003 virtual machines, which results in the COM1 port not showing up after starting the VM.

Here is a short summary on how to circumvent the issue:

  1. Go to Control Panel, Add Hardware
  2. Select "Yes, I have already connected the hardware"
  3. In the list, select the last item: Add a new hardware device
  4. Select "Search for and install the hardware automatically"
  5. Windows found a Communications Port

Now the COM1 port should show up on (both) the virtual machines. It is however necessary, due to our configuration, that you first start the Debugger Guest and then the WRK Guest, because otherwise the pipe does not exist when starting the WRK Guest.

You can now start to deploy the build environment and the debugger to the Debugger guest as well as deploy the WRK, after being built, to the WRK Guest.

For the communication over the virtual serial port, it is necessary that both ends use the same baud rate for transmission. On the Debugger Guest you can do this before starting the "Kernel Debug …" session with applying the following settings:

Configuration for the kernel debug session on the Debugger Guest.
Figure 5: Configuration for the kernel debug session on the Debugger Guest.

For the WRK Guest, it is necessary to configure the boot.ini file a little bit more. If you already have the WRK deployed on this machine, you should have at least the following two [operating systems] entries:

[operating systems]
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Server 2003 Enterprise"
/noexecute=optout /fastdetect
multi(0)disk(0)rdisk(0)partition(1)\WINDOWS="Windows Research Kernel"
/kernel=wrkx86.exe /baudrate=115200 /debug /hal=halacpim.dll

You should be familiar with the /hal, /debugport and /debug switch. You must also specify the /baudrate switch and specify the same baud rate as you did with WinDbg, in our case 115200.

That is it. You should now be able to do both, developing and testing the WRK when running Mac OS.


4 Responses to "How To: Debug the WRK on Mac OS X Using VirtualBox"

  1. Robert Kornmesser on November 18th, 2009 13:49

    Leider funktioniert die Anleitung so nicht.
    Die Com-Port Ressouce wird als belegt angezeigt, sobald der /debugport switch in der boot.ini gesetzt ist.
    Ich komme genau an dieser Stelle auch nicht weiter.

    Die Serielle-Kommunikation konnte ich im normalen Modus mit Hilfe von Putty erfolgreich teste.

  2. How To: Debug the WRK on Mac OS X Using VMware Fusion on January 3rd, 2011 12:12

    [...] hosts the kernel debugger. Both VMs will use their serial port to communicate with each other. In a previous post, we have shown for VirtualBox how to connect both serial ports over a named pipe. Now, we are going [...]

  3. How To: Boot the WRK in a Virtual Machine : Windows Research Kernel @ HPI on June 27th, 2011 12:12

    [...] a previous post, we showed you how to debug the WRK on a Mac OS system using Sun's VirtualBox. However, in [...]