Writing a cross-platform ELF binary

Just for fun (actually, I was bored at work) I came up with the crazy idea of writing a cross-platform ELF binary that I could run on Linux, FreeBSD (without the Linux compatibility module), and potentially other Unix. I'm sure at the time I justified it by thinking that it would somehow be easier than writing portable C that I just compiled on different platforms, though in retrospect I was just looking for an excuse to do something a bit weird.

My early experiments with making portable ELF files weren't going so well. Trying to run a Linux executable on FreeBSD (without Linux compat) was harder than I thought. The first problem is the ABI version in the ELF header. The first byte of the padding in the ELF header is interpreted as an ABI version under FreeBSD. Linux just sets it to 0 (and ignores any other value you set it to), but FreeBSD recognizes:

> brandelf -l
known ELF types are: FreeBSD(9) Linux(3) Solaris(6) SVR4(0)

...and refuses to run an ELF type of 0. Supposedly, setting to 3 works if you have the Linux compat library installed, but I was avoiding that.

Before I gave up, I got the idea that if I could strip the program down to the basics, I'd have a better chance of making it work. I found a great article on making tiny ELF files, which reduced the problem down to:

• Define _start instead of main
• Make the exit system call (without calling _exit)

It turns out that an even bigger problem than the ELF header is making system calls cross-platform. FreeBSD system calls use a different calling convention: arguments are passed in the stack instead of registers like on Linux. Fortunately, I'm only making one system call (exit). Setting the registers AND pushing values on to the stack will work on both platforms! Doing something complicated that requires more system calls could get messy rather quickly, though.

The final piece is just doing something useful. I found an article describing using the SLDT instruction on x86 to detect virtualization. In summary, if both the high and low bytes returned by the SLDT instruction are nonzero, there's a high probability you're running in a VM. There are lots of other ways to detect virtualization, but SLDT is easy.

The code I wrote is probably non-optimal (I haven't written much x86 assembly), but at least is easy enough to read. Anyway, the "main" assembly part (not including ELF headers, etc) of the program is as follows:

_start:
    mov     eax, 1      ; system call 1 (exit)
    mov     ebx, 0      ; default return value of 0
    sldt    edx         ; SLDT to detect VMWare
    cmp     dl, 0       ; if first byte is 0
    je      .L2         ; jump to end (return 0)
    cmp     dh, 0       ; if second byte is 0
    je      .L2         ; jump to end (return 0)
    mov     ebx, 2      ; otherwise, set return value of 2
.L2:
    push    ebx         ; push return value arg onto stack
    push    eax         ; push syscall number onto stack
    int     0x80        ; syscall interrupt

This gives a return value of 2 if the machine is running in a VM, otherwise 0 (not using 1 because that might be the return value if it doesn't execute properly). It should run on any UNIX platform if the ABI in the ELF header is set correctly, though I've only tested on Linux and FreeBSD. Full assembly on our Google Code project (direct download link here). Since I was following the tiny ELF file article, the assembly includes the ELF header and is 116 bytes when assembled. Build with nasm -f bin -o vmware_small vmware_small.asm. It runs on Linux as-is, but you need to run brandelf -t FreeBSD vmware_small on FreeBSD to set the ABI byte to 9 and make it run there (afterwords, it can still run on Linux!).