<-- Chapter 15 | Back to Index | Chapter 17 -->
| Be warned! Most of today's chapter deals with gcc. If you're using another compiler, you may hit some issues. Most of the concepts should be almost identical with clang, but even then, some things will be slightly different. |
You may have noticed some warning messages appearing when we compile our code. Indeed, there are some warnings in our code at the moment. Let's run build.sh and look at them:
../handmade/code/handmade.cpp:67:1: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
char *Filename = __FILE__;
^
../handmade/code/handmade.cpp:72:86: warning: deprecated conversion from string constant to ‘char*’ [-Wwrite-strings]
DEBUGPlatformWriteEntireFile("test.out", File.ContentsSize, File.Contents);
^
As we can see, we have two warnings. Each of them are represented by three lines. The first line is the important one. It starts with the name of the file (../handmade/code/handmade.cpp), followed by the line number 67 and the column number 1 of the issue. We then find out that is is a warning, which states:
deprecated conversion from string constant to 'char*'This particular warning is telling us that we're passing a constant string as a "char*" rather than a "const char*". gcc doesn't let you modify string literals once they've been initialised, so it's probably a good thing to be aware of. We aren't using const in handmade hero (and the errors are in handmade.cpp, which we're not modifying for compatibility reasons), so let's disable the warning. The last thing on the line is the flag which enables the warning: [-Wwrite-strings]. This means that if we want to specifically enable the warning, we pass -Wwrite-strings to the compiler, and if we want to disable it (as we do), we want to pass -Wno-write-strings. Let's add that to our build.sh file.
Our code now compiles with no warnings! Let's fix that. gcc (and other compilers) have lots of warnings that aren't enabled by sefault. Let's use -Wall to enable them. We'll see a number of errors, but most of them aren't too important. For example, we can get rid of the unused variable messages with -Wno-unused-variable.
The other issue, warning: comparison between signed and unsiged integer expressions actually reveals a bug in our file code (oh no!): we're storing the return value from read() (and write()) in an unsigned integer, but it can return -1 as an error. We'll change it to use a ssize_t (the given return type in the documentation) instead of uint32.
We're back down to no warnings! Let's make sure we look at them and treat them properly. The -Werror option (the equivalent of the MSVC -Wx) makes the compiler treat warnings as errors: the game won't compile at all until we've fixed or disabled all of the warnings.
| Be warned! Not enough warnings for you? You can also enable -Wextra for even more warnings. There's also -pedantic and -pedantic-errors if you really want to follow the standard carefully. If even that isn't enough, there are tools like the clang static analyser, which will track down tiny pieces of undefined behaviour. These are all great ways of finding subtle bugs, but they can require a lot of hard work and discipline to maintain: you can end up spending a lot of time working around harmless warnings! |
Since we're not going to be using advanced C++ features like Run-time type information and exceptions, we can turn these off as well. The -fno-rtti and -fno-exceptions options do just that.
| Be warned! This bit is pretty specific to Debian based distributions (including (K)Ubuntu and Linux Mint). If you're using a different distribution, some of the principles will still apply, but you'd be strongly advised to do some research before doing this. |
Not everyone is lucky(?) enough to have a 64-bit capable machine. These poor souls deserve to play Handmade Hero, too! Clearly we need to be able to compile a 32-bit version of our code as well as a 64-bit version. The bad news is that this can get monumentally difficult on Linux. The simplest way of compiling a 32-bit program is simply to compile it on a 32-bit machine. Indeed, this is one of the normal ways this is done: simply have a separate 32-bit machine (or virtual machine) to compile 32-bit code on.
Yeah, that sounds wasteful to me, too: if our 64-bit machines can run 32-bit code, they should at the very least be able to compile it. What we're looking for is a cross-compiler. Indeed, we can install cross-compilers for many different architectures: to build programs for ARM or PowerPC CPUs, or even to compile Windows programs on Linux with mingw. When installed, you'll simply replace your c++ command with something like i686-w64-mingw32-c++. There's only one thing you might notice: for reasons unknown to man, there's no x86 compiler listed in the Debian repositories.
The secret is that the version of gcc that's installed on Debian based systems is compiled so that it can generate both 64-bit and 32-bit code already. Simply pass the -m32 option to make it compile 32-bit code. But it's not quite that simple: we need to install 32-bit, development versions of all of the libraries we're using. Most importantly, we need to install the standard C runtime library. We can install all of the system-critical libraries with
sudo apt-get install gcc-multilib g++-multilibWe'll still, however, need a 32-bit copy of SDL. We can install the library with
sudo apt-get install libsdl2-2.0-0:i386but that only installs the runtime library. We need the development library as well:
sudo apt-get install libsdl2-dev:i386Oh no! It wants to uninstall our 64-bit version. And that's the problem: in Debian, you can't have the development packages for the some libraries installed for multiple architectures. This is so that their include headers don't conflict. We therefore have two choices:
If we try to compile our code with the -m32 flag, it seems to actually compile, and just complain about the missing SDL2 library at the end. (It also tells us it's skipping a bunch of incompatible, 64-bit libraries, but that's harmless.) If you think about it, that means it's finding the includes. The reason that we couldn't have both 32-bit and 64-bit versions installed at once is that, for some stupid reason, their includes live in the same place. What we'll rely on is the fact that, while they aren't compatible for all libraries, you can usually get away with mixing the 32-and 64-bit headers for SDL (and some other libraries like OpenGL). So if we can find the right library file, we're done.\
We know we have the 32-bit library installed, just not the development package. If we look in /usr/lib/i386-linux-gnu we can see it: libSDL2-2.0.so.0. So why can't the compiler find it. The secret is in how libraries are versioned on Linux. (We can be thankful it's not using Windows' Side-by-side assemblies, though.) There are three names for each library: the actual installed version: libSDL2-2.0.so.0.2.0, which has the exact version of the library as a part of its name; the major version: libSDL2-2.0.so.0 which is a name (called the SONAME) shared by all versions of the library which are binary compatible with the installed version — i.e. a program compiled against one will still run against the other — which is a symbolic link to the actual version; and finally the development version: libSDL2.so, which is the name we pass to the compiler. When we compile our program, the linker inserts the SONAME into the executable, so that your program is always linked against a compatible version. It's quite a handful, and there's a good resource from IBM if you're interested.
What this means, is that if we add the development version ourselves: by creating a symbolic link to the installed version, everything will work. This is frowned upon a bit — if the headers are incompatible we could get in a bit of a pickle — but it's done more often than you might think.
cd /usr/lib/i386-linux-gnu/ sudo ln -s libSDL2-2.0.so.0 libSDL2.soNow if we compile our program, it links succesfully and we can run it. Phew!
If you thought that getting the game to run on Windows XP was a pain, wait until you experience the full horror of getting a binary to run across lots of (particularly older) Linux distributions. The problem largely boils down to the fact that most Linux software is open-source, so every distro just recompiles it: there's much less incentive to care about binary compatibility.
There are two things that cause problems, compatibility wise, on Linux:
Let's start by running ldd to see what libraries our program loads.
ldd ./handmadehero
linux-vdso.so.1 => (0x00007fffa37e4000)
libSDL2-2.0.so.0 => /usr/lib/x86_64-linux-gnu/libSDL2-2.0.so.0 (0x00007f99ac647000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f99ac341000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f99abf7b000)
libasound.so.2 => /usr/lib/x86_64-linux-gnu/libasound.so.2 (0x00007f99abc8a000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f99aba86000)
libpulse-simple.so.0 => /usr/lib/x86_64-linux-gnu/libpulse-simple.so.0 (0x00007f99ab880000)
libpulse.so.0 => /usr/lib/x86_64-linux-gnu/libpulse.so.0 (0x00007f99ab632000)
libX11.so.6 => /usr/lib/x86_64-linux-gnu/libX11.so.6 (0x00007f99ab2f9000)
libXext.so.6 => /usr/lib/x86_64-linux-gnu/libXext.so.6 (0x00007f99ab0e6000)
/lib64/ld-linux-x86-64.so.2 (0x00007f99ac982000)
libpulsecommon-4.0.so => /usr/lib/x86_64-linux-gnu/pulseaudio/libpulsecommon-4.0.so (0x00007f99a95cb000)
libjson-c.so.2 => /lib/x86_64-linux-gnu/libjson-c.so.2 (0x00007f99a93bf000)
...and many more. Our program isn't using all of those libraries, surely. We're certainly not linking against things like libjson.
The secret is that we're linking to SDL, which itself links to a whole bunch of libraries, which link to a lot of libraries themselves.
Ouch. If we want to see what libraries we link to directly, we'll use the objdump
utility.
objdump is a crazy program: it can tell you unspeakable things about executables and libraries, including completely disassembling them. What we want to do is to look at the private headers. If we run objdump -p, we'll get a whole lot of information, most of which we don't care about. It you look under the Dynamic Section heading, you'll see a bunch of lines beginning NEEDED. These list the libraries the executable is linked to:
Dynamic Section: NEEDED libSDL2-2.0.so.0 NEEDED libm.so.6 NEEDED libc.so.6That's more like it: only three libraries, and we know what they all are. libSDL2-2.0.so.0 is SDL2, libm.so.6 is the maths part of the C standard library (we use it for sin()) and libc.so.6 is the main part of the C standard library.
If we run objdump -p on the libSDL2-2.0.so.0 file:
Dynamic Section: NEEDED libasound.so.2 NEEDED libm.so.6 NEEDED libdl.so.2 NEEDED libpulse-simple.so.0 NEEDED libpulse.so.0 NEEDED libX11.so.6 NEEDED libXext.so.6 NEEDED libXcursor.so.1 NEEDED libXinerama.so.1 NEEDED libXi.so.6 NEEDED libXrandr.so.2 NEEDED libXss.so.1 NEEDED libXxf86vm.so.1 NEEDED libwayland-egl.so.1 NEEDED libwayland-client.so.0 NEEDED libwayland-cursor.so.0 NEEDED libxkbcommon.so.0 NEEDED libpthread.so.0 NEEDED librt.so.1 NEEDED libc.so.6 SONAME libSDL2-2.0.so.0Wow: SDL needs a lot of libraries. What's particularly concerning is that it has lots of libraries listed that do the same thing on different platforms: if the user is using ALSA for sound, do we need PulseAudio's libpulse.so.0 and libpulse-simple.so.0 files? Or vice-versa: do we need libasound.so.2 if the user is running PulseAudio. Wouldn't it be nice if these could be dynamically loaded if they're already installed, and quietly ignored oterwise (like XInput and DirectSound are in the Windows version). Fortunately, SDL already supports doing this, but the version included with Debian doesn't have it enabled. We'll need to compile our own version of SDL with the --enable-library-shared options. These are the options I use for building SDL:
./configure --disable-esd --disable-arts --disable-video-directfb --disable-rpath --enable-alsa --enable-alsa-shared --enable-pulseaudio --enable-pulseaudio-shared --enable-x11-shared --enable-sdl-dlopen --disable-input-tslibIf we compile with those options, we end up with a much smaller list of required libraries:
Dynamic Section: NEEDED libm.so.6 NEEDED libdl.so.2 NEEDED libpthread.so.0 NEEDED librt.so.1 NEEDED libc.so.6All of these are system libraries, which we don't need to (and shouldn't, for compatibility reasons) include with our program.
This leaves only libSDL2-2.0.so.0 as a required library. Just copying the file into the same directory as our file won't work, though. By default, Linux doesn't look in the current directory (or the same directory as the current binary) for libraries. There are a couple of ways to work around this. The simplest one is to use the LD_LIBRARY_PATH environment variable. You can set this to a list of extra paths to search for libraries. Having a script that sets this and runs your game is one way of fixing the library problem:
#!/bin/sh LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH ./handmadeheroThis is useful, as there are often other things you might want to do in such a script. On the other hand, it's quite an ugly solution, and requires the current directory to be set correctly (though you could always do that in the script). Another technique is to use the RPATH linker option. This embeds an extra search path for libraries into the executable. So, if we were to add -Wl,-rpath,'$ORIGIN' to our build path, it would search the same directory as the binary. The -Wl part lets gcc know that the next commands (separated by commas) should be passed to the linker. -rpath tells the linker that we want to embed a library path, and '$ORIGIN' is a special variable that expands to whichever directory the executable is in.
Compile again, and if we look with objdump at our new executable, there's a new header in the Dynamic Section
RPATH $ORIGINOur program will now try to read libraries from its own directory first. Sweet! One thing we might want to do, if we want to put our 32-bit and 64-bit binaries in the same directory is to give them each a different library directory. One way to handle this is to put the 32-bit libSDL2-2.0.so.0 file in an x86 directory, and give our 32-bit binary the $ORIGIN/x86 RPATH. Similarly, for 64-bit we can have an x86_64 directory and the $ORIGIN/x86_64 RPATH.
If you try to run a program which was compiled against a newer version of the C runtime library than the one you have installed, you might get an error: version `GLIBC_2.15' not found This is because (along with a small number of other system libraries), the GNU C Library uses something called Symbol Versioning, which allows several different versions of functions to be included in the same library. The idea behind this is was to ensure that programs which relied on the older behaviour of functions would still work when linked against newer versions of the library. On the other hand, if our program wants to use newer versions than the ones installed, the program will quit with the above error.
We can check what symbol versions are used with objdump -p as well: they're found under the Version References header:
Version References:
required from libm.so.6:
0x09691a75 0x00 04 GLIBC_2.2.5
required from libc.so.6:
0x06969194 0x00 05 GLIBC_2.14
0x0d696914 0x00 03 GLIBC_2.4
0x09691a75 0x00 02 GLIBC_2.2.5
These days, you're usually safe with requiring functions tagged GLIBC_2.15 or earlier, so we're okay for now. We have to
keep checking, though, as it's entirely possible we'll call functions with newer versions later. Let's look at how we would work
around the problem.
Basically, we'll need to compile against the old symbol versions. The easiest way to do this is just to install and use an old distro. That's a bit of a pain, though, so we often will either install it in a chroot or create a custom cross-compiler.
There are some tools out there for creating cross-compilers, such as crosstool, but it's quite a lot of work to configure and compile entire environments. Valve implemented a system called the Steam Runtime to solve this very problem. The Steam runtime has two parts, a chroot for compiling programs, and what is basically an entire distribution's worth of libraries. The idea is that if you compile code using the Steam runtime, anyone with the corresponding Steam version will have all of the libraries set up. The Steam runtime is basically a copy of Ubuntu 12.04, which is old enough that most people have a compatible version of glibc.
So let's try using the Steam Runtime SDK. Unfortunately, the easy installer for it has been mysteriously replaced on the github repository. You can still get the setup.sh file we're looking for in the Steamwork SDK sdk/tools/linux directory, or, if you're not a Steamworks developer, in the Github history here. Once you've got it, put it in a directroy by itself, mark it exectuable and run it:
cd steam-runtime chmod +x setup.sh ./setup.shIt will ask you what architecture you want ("all supported architectures"), whether you want a debug or release build (debug for now) and whether or not you want to update the "base SDK", "tools" and "runtime". You want to update all three.
Once it's installed, you can use ./shell-amd64.sh (or ./shell-i386.sh) to enter the steam runtime SDK environment. Any compilation you do from within will use the steam runtime compiler and libraries. This nicely solves our symbol versioning problem. However, our users will still need all of the required libraries: either they'll need to have the steam runtime installed on their own machines, or we'll need to provide our own copy of SDL (and any libraries SDL requires).
If you're going to rely on the user having the Steam runtime installed, you either need to distribute your game on steam, or include the Steam runtime with it. Jørgen Tjerno has a couple of great articles about doing this.
The other solution is to use objdump -p to find the libraries we need, and to include them with our program, as above. The trick here is, not only do we need to ensure our program doesn't use any new symbol versions, we also need to ensure that all of the libraries are compiled against older glibc. It's probably pretty safe to do this from within the Steam runtime SDK these days, but if you want to target older distros (or want it more automated), there's another easier (if hackier) solution.
bingcc is a wrapper around gcc that forces it to link against older glibc symbol versions. You basically just need to call bing++ (or bing++32) instead of c++ to compile your programs. Be careful: it only handles glibc symbol versions, and can cause problems if you need to use advanced C standard library features. One of the useful features of bingcc is that it has scripts that compile extra-compatible versions of common libraries, like SDL. bingcc is something of a hack — it just forces the compiler to link against old symbol versions, without understanding the differences between said versions — so the Steam runtime may be better for complex programs, or programs which rely on the C++ standard library.
| Be warned! Because it requires some level of setting up on your own machine, the sample code does not include any of the glibc version management. It does, however, include a version of SDL compiled with bingcc, and the setup script for the Steam runtime, and will build both 64 and 32 bit versions of the game. Since we're not using many advanced C standard library functions (though since the Linux syscalls are also typically a part of libc, we can't get rid of them all), we're currently compatible with most Linux distros anyway. |
Woah: that's a lot of stuff! Linux binary compatibility is, as you can tell, something of a mess and there's not a lot of good information out there. It's the sort of thing that's been passed from Linux game porter to Linux game porter throughout the ages. Hopefully this has given you a good idea of what the issues are. We'll likely touch upon this again later, when we're packaging our game up for distribution.
If you've bought Handmade Hero, the source for the Linux version can be downloaded here. Note that you will require the official source code for handmade.h and handmade.cpp. This will now build a 32-bit version: handmadehero.x86 and a 64-bit version: handmadehero.x86_64.