Porting virt tools to Windows

So you've written your nice Gtk-based GUI with C library back end and it runs really well on the GNOME desktop. The only problem is that a few diehards insist on using this thing called Windows. This is an essay about how to make your program run on Windows, and in particular it's about how I made our virtualisation library and tools run as Windows programs.

Libvirt — porting a C library

Libvirt is our abstraction library that hides the differences between virtualisation technologies.

libvirt making POSIX calls to Linux

Technically speaking it's a library, written in the C programming language, which assumes a POSIX- / Linux-like API. It uses a GNU toolchain to build, so requires programs like gcc, GNU make, and the autotools such as autoconf and automake. So this is a good place to start if you want to port such a program or library.

Porting this involves getting replacements for:

  1. the C compiler
  2. the POSIX API calls
  3. the build environment (GNU make, autotools)

Visual C++

We could start with Microsoft's C compiler offering, called Visual C++ which has, for some time, been free (as in beer) software. It's a competent C compiler and will certainly compile most of our code. Well except for the GCC extensions we've used, but perhaps it's not too much trouble to go back to strict ANSI C. What it cannot do is compile any external API calls we make which aren't part of a subset of standard C, so any calls to fork, ioctl, getuid etc etc won't compile.

Our other problem with Visual C++ is that the build environment is really nothing at all like the GNU toolchain. They have a very primitive make program, but nothing at all for the rest of the toolchain. You can create a VC++ "project" but that will splatter proprietary binary files into your source tree and you'll be left maintaining two completely separate build systems in parallel.

MinGW/MSYS

However we can install the GNU toolchain (gcc, GNU make, autotools, bash, sed, grep, etc.) under Windows using MinGW and MSYS. This gives us gcc, so we can compile the code with GCC extensions. It gives us enough GNU tools that we can run ./configure and make, probably with only rather minimal changes to the build system. The changes are equivalent to the changes you might make if you wanted to port to an obscure and rather strange old version of Unix (say, IRIX 4).

However MinGW does not solve the missing POSIX API.

Cygwin

Cygwin library emulates POSIX calls on the NT kernel

For that we need to consider Cygwin. Cygwin is at its heart a library which emulates POSIX calls on Windows. Cygwin means that we can compile and run libvirt and its applications directly on Windows with barely any changes from the Linux version (by "barely any", I really mean it too — a couple of dozen lines of code, most of which were actually bugfixes).

At one point, Cygwin and MinGW forked from each other and it appears they still share code and fixes. Cygwin gives you the complete GNU toolchain as with MinGW/MSYS.

Cygwin catches

There are two big catches about using Cygwin: The first is that emulation is quite slow. As an example, Windows has no real concept of a fork as in Unix, so Cygwin needs to emulate it, and since fork is very common (particularly in shell scripts) and emulation of all the details is slow, certain things are much slower than under a real Unix. One of these are the shell scripts used by autotools, so expect that a run of ./configure will take tens of minutes instead of tens of seconds.

The second big catch is that the CYGWIN1.DLL which all your programs must link with is (intentionally) licensed under the GPL. This means that proprietary programs cannot be linked against it, and possibly it stops proprietary programs from linking to our libvirt as well, even though libvirt is under the LGPL.

Introducing #ifdef WIN32 ...

Because of the above catches, we may now be interested in what it would take to port libvirt "for real" to the NT API. How much of libvirt (or your program) really depends on POSIX calls?

The Win32 API is really very different from POSIX. For example to open a file you use a function called NtCreateFile which takes about a dozen very strange parameters and is generally quite alien to Linux.

However Win32 also has a selection of Unix-like calls and wrappers. For example when Microsoft desperately wanted to join the internet revolution, they wanted to port existing BSD programs quickly so their API called Winsock largely contains what looks like the usual BSD socket calls (socket, bind, accept etc.). Also you'll find many Unix-like calls which are a part of older C implementations such as open, fopen etc.

The practical upshot of this is that simple network programs with no UI port quite easily. Unfortunately the differences will start to get to you. One example I well remember is that Winsock functions don't return error codes in errno. Instead there is a "special" Windows function called WSAGetLastError which returns the same thing. Oh, and error numbers have different names and meanings. No big deal though, we'll just start to #ifdef those cases:

int get_error ()
{
#ifdef WIN32
  return WSAGetLastError ();
#else
  return errno;
#endif
}

You'll discover other differences too. Such as select being replaced by WaitForMultipleObjects, and after a while your code will start to look like the source code for Apache 1.3, where we see stuff like this:

    } while ((select_rv > 0) &&
#if defined(WIN32) || defined(NETWARE)
  (recv(lsd, dummybuf, sizeof(dummybuf), 0) > 0));
#else
  (read(lsd, dummybuf, sizeof(dummybuf)) > 0));
#endif

I can read that and imagine the pain they had before they discovered exactly why read in Winsock didn't quite have the same behaviour as under Unix ...

After a while of doing this (if your program is significantly large or important enough to justify it) you're maybe thinking about writing a portability library to isolate all these changes. Apache developers had this moment of realisation in around 1996 and they came up with the Apache Portable Runtime as used in Apache ≥ 2.0, and Netscape has the NSPR, GNOME folk have Glib, and there are plenty of others.

Conclusions

By the way, if you were expecting me to come up with a brilliant and definitive method for solving this, then sorry I'm going to disappoint. Porting a program to use the "#ifdef method" is intrusive, but so is changing your program to use APR. It would have been better (possibly?) if we'd used a portability library from the start, but as it turns out we didn't.

virt-manager — porting high level languages

Perl functions are turned into NT kernel calls by the Perl runtime

If your program is written mostly or entirely in a modern high level language, and by that I'm including Java, Perl, Python etc., then the situation is usually much better. The reason is simply because all these high level languages were developed with portability in mind. In general terms, all the portability issues above are confined to the language's implementation and base libraries, and you program directly in what is in effect a portable runtime. (Often the language implementation uses the "#ifdef method" of portability, certainly that's the case in Perl and OCaml).

As an example, take the Perl function listen, defined as:

  listen SOCKET,QUEUESIZE

In Perl this eventually turns into a call into C, and the call is routed to one of several different implementations. The Win32 one looks like this:

int
win32_listen(SOCKET s, int backlog)
{
    int r;

    SOCKET_TEST_ERROR(r = listen(TO_SOCKET(s), backlog));
    return r;
}

where SOCKET_TEST_ERROR is a convoluted macro to deal with the errno / WSAGetLastError problem in Winsock.

Java is of course the mother of all portable languages, containing a huge library and claiming that any 100% pure Java program will run anywhere, a claim which is often disputed.

Python and PyGTK

If the program in question, virt-manager, was a pure Python command-line program (it isn't), then it's likely it would run almost immediately under the Windows native port of Python. In fact several of our virt tools are command-line Python programs (for example, virt-install) and they compile without any problem. Although there will still be a few incorrect assumptions they make, such as the location of system files, so those will need to be resolved over time.

Unfortunately virt-manager has a long list of dependencies, and a key one might appear to be PyGTK.

Actually, GTK isn't so much of a problem as it sounds. Gtk on Windows is mature software, there are solid Python bindings (PyGTK) which also run on Windows, and it even has an almost-Windows look and feel. I've written about this before so I won't go into further detail.

Building virt-manager

The real list of implicit and explicit dependencies for virt-manager looks like this:

  1. C compiler (for some custom Gtk widgets)
  2. Python
  3. GNU make and autotools for the C stuff
  4. Python toolchain for the Python bits
  5. GTK and PyGTK
  6. urlgrabber
  7. rarian
  8. gtk-vnc
  9. ... and don't forget libvirt itself.

Using Cygwin, all of those dependencies + virt-manager compile and runs [well, as at time of writing, almost runs but there is a tiny bug related to GTK which I'm confident I can fix]. That's a success for Cygwin because the patch required to make it work is just 83 lines long and as with libvirt most of it represents bugs that we've revealed by trying to use virt-manager in a different, but "Unix-y enough" environment.

Conclusions

My conclusions for this are: do where possible use a high level language, where the language designers have already solved the portability problems for you. You'll find it far simpler than in C to build your program across multiple platforms.

Further reading


rjones AT redhat DOT com

$Id: index.html,v 1.1 2007/11/30 16:03:32 rjones Exp $