Call of an Ancient Power — Bringing Civilization into a New Age

Background

In the late 90s, and early 2000s, Loki Entertainment released native Linux versions of a number of contemporary commercial games. The first of these was the (in my opinion, underrated) Civilization: Call to Power, a spin-off of the well-known Civilization series.

The Linux of 1999 is very different to the Linux of today, however, and so it can be quite difficult to get such an old binary working well on modern systems. While the Linux kernel has maintained very good backwards compatibility, the other parts of a Linux distribution have changed, from the X server to libraries like glibc, libstdc++, and SDL.

In fact, the statically linked civctp binary does work on modern systems, but with a number of issues:

• Sound wouldn't work, as OSS is deprecated, and tools like padsp don't work with statically linked binaries.
• Some parts of the window (particularly the video playback) would be translucent, due to changes in how alpha-transparency is handled between X protocol / server versions.
• Fullscreen and/or resolution changes were brittle or broken.
• Wayland support was absolutely non-existent, and some of the above issues were worse with XWayland.

Civilization: Call to Power (at least in its later patches) also provided a dynamically-linked version: civctp.dynamic. Having a dynamically-linked version allows us to replace libraries with newer versions. In some cases, this causes more problems than it solves — libstdc++ has changed enough that the version which is compatible with CivCTP is not compatible with, for example, most graphics drivers — but in other cases, we can replace the library with a much more functional version, such as with SDL.

Traditionally, running the dynamically-linked version could only be done with the appropriately-old versions of all of these support libraries. A wonderful set of such libraries, compiled so that they can be run on modern systems has been maintained by Alan Swanson for years. However, they're not up-to-date enough to support PulseAudio, or any of the more modern parts of the Linux graphics stack, so ultimately don't solve all of these problems.

Additionally, it'd be nice to be able to replace SDL 1.1, which Civilization: Call to Power uses, with a more recent version. SDL 1.2 is actually binary-compatible with SDL 1.1, so it should be possible to use that for (e.g.) PulseAudio support, but the sdl12-compat project provides a binary-compatible reimplementation of SDL 1.2 on top of SDL2, which adds support for hardware-accelerated graphics, screen-resolution scaling, and Wayland support.

It should theoretically be possible to run Civilization with these newer libraries, but it'll take a bit of patching…

Act Ⅰ: Raiders of the Lost Libraries

Our first goal is to get the game binary to start. This will require copying a few libraries around, and configuring the game to use them.

If we try to start the game straight away, we get an error:

./civctp.dynamic
./civctp.dynamic: error while loading shared libraries: libSDL_mixer-1.0.so.0: cannot open shared object file: No such file or directory

This tells us that the libSDL_mixer-1.0.so.0 library cannot be found. In fact: it's not the only library which is missing, but required. There are two ways of getting a list of libraries needed by a program, each with their own advantages and disadvantages: ldd and objdump.

objdump, when used with the correct options, parses the ELF binary, and extracts the list of needed libraries. These then are the libraries that civctp.dynamic directly calls into. (objdump -p shows a lot of other info, too, so we'll just need lines which contain NEEDED)

objdump -p ./civctp.dynamic | grep NEEDED

  NEEDED               libSDL_mixer-1.0.so.0
  NEEDED               libsmpeg-0.4.so.0
  NEEDED               libttf.so.2
  NEEDED               libSDL-1.1.so.0
  NEEDED               libX11.so.6
  NEEDED               libdl.so.2
  NEEDED               libpthread.so.0
  NEEDED               libXext.so.6
  NEEDED               libm.so.6
  NEEDED               libc.so.6

ldd works by setting a magic environment variable (LD_TRACE_LOADED_OBJECTS=1), and then "running" the executable, making the system dynamic linker go through and load all of the required libraries, but then print out where it found them and quit, rather than starting the program. This not only gives the libraries used directly by the game, but also all of the libraries used by those libraries, and so on, recursively. ldd also prints out where the libraries were found (or if they weren't, so it can be quite useful for checking that the right library is loaded. In our case, ldd shows:

ldd ./civctp.dynamic
        linux-gate.so.1 (0xf7f95000)
        libSDL_mixer-1.0.so.0 => not found
        libsmpeg-0.4.so.0 => not found
        libttf.so.2 => not found
        libSDL-1.1.so.0 => not found
        libX11.so.6 => /lib/libX11.so.6 (0xf7e11000)
        libdl.so.2 => /lib/libdl.so.2 (0xf7e0c000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xf7e07000)
        libXext.so.6 => /lib/libXext.so.6 (0xf7df1000)
        libm.so.6 => /lib/libm.so.6 (0xf7ceb000)
        libc.so.6 => /lib/libc.so.6 (0xf7aba000)
        libxcb.so.1 => /lib/libxcb.so.1 (0xf7a8c000)
        /lib/ld-linux.so.2 (0xf7f97000)
        libXau.so.6 => /lib/libXau.so.6 (0xf7a87000)

Both lists (but particularly the list of not found libraries from ldd give us a pretty good starting point. For each of the missing libraries, we can do one of two things: provide a period-appropriate copy (either by, for instance, the one from Loki_Compat or compiling one yourself), or by patching the game to use a different library or version.

Let's go through the missing libraries:

libSDL_mixer-1.0.so.0

SDL_mixer is an audio playback and mixing library built on top of SDL, and it's what Civilization: Call to Power uses for its sound effects. (Music is played via SDL 1.2's CD audio support, which we'll get to later).

CivCTP was built against version 1.0 of SDL_mixer, but is actually compatible with later versions. We can therefore solve our problem here by installing the distro-provided version of SDL_mixer, and patching the CivCTP binary to link against version 1.2.

To update the executable, we can either use a hex editor and patch the bytes manually, or use the patchelf utility. (We'll do this on a copy of the game, just in case!)

cp civctp.dynamic civctp.dynamic_fixed
patchelf --replace-needed libSDL_mixer-1.0.so.0 libSDL_mixer-1.2.so.0 civctp.dynamic_fixed

If we check with ldd again, we can see that SDL_mixer is now found, with its new name:

libSDL_mixer-1.2.so.0 => /lib/libSDL_mixer-1.2.so.0 (0xf7e87000)

libsmpeg-0.4.so.0

This is the MPEG decoder used for the in-game videos. While the library itself has seen very few updates since Civilization was released, it has seen a major upgrade to run under SDL2. Alas, we need to provide the SDL 1.2 version.

It is possible to compile this yourself, the source code is in the SDL-1.2 branch on GitHub. Compiling it can be a pain on modern systems and compilers, though, particularly since you need a 32-bit version.

It's probably easier just to copy the libsmpeg-0.4.so.0 binary from the Loki_Compat libraries, or use a version provided by your distro (if available) though.

libttf.so.2

An ancient version of the freetype font library. It's not compatible with newer versions, and can be a pain to build, so copy the file over from either Loki_Compat, or the packages from an old distro version.

libSDL-1.1.so.0

This is the big one, the Simple DirectMedia Layer. This library handles most of the graphics, sound, input, and general interactions with the OS. Civilization: Call to Power was one of the first big, commercial titles to use it, and it's still going strong, with newer versions being used in everything from Valve's Steam service, most Linux games, and even the Nintendo NES and SNES mini consoles.

Civilization: Call to Power uses version 1.1 of SDL, which is both very old, and rarely available. It has some issues with window transparency on modern versions of X11, and many builds don't even properly support ALSA for sound, never mind something more modern like PulseAudio. Fortunately, the much more common version 1.2 is almost identical. SDL 1.2 was the main version used for about a decade, and during that time, it was updated to support newer systems, including PulseAudio sound support.

We can therefore simply patch the executable to use version 1.2, and most things will magically start working:

cp civctp.dynamic civctp.dynamic_fixed
patchelf --replace-needed libSDL-1.1.so.0 libSDL-1.2.so.0 civctp.dynamic_fixed

But the story doesn't end there. SDL 1.2 was superseded by SDL 2.0 back in 2014 (and, indeed, SDL 3.0 is currently being worked on in 2023). SDL 2.0 is much more suited to modern hardware and operating systems than SDL 1.2 ever was. Not only does it provide native support for newer APIs like Wayland and Pipewire, it also supports newer ways of handling fullscreen windows (without nasty resolution changes or breaking Alt+Tab) and provides a hardware-accelerated 2D API which is much better suited to modern GPUs.

It'd be great if we could take advantage of this, though we do hit a few issues. SDL2's API is, while reminiscent of SDL 1.2's, not compatible, and indeed lacks some features CivCTP uses. Some sort of compatibility layer is needed to adapt the old API into the new one, and to restore or emulate the missing functionality. In the case of Civilization, this missing functionality is CD Audio support: Civilization's music works by playing tracks on an audio CD, which needs some rethinking on modern systems.

Fortunately, such a compatibility layer already exists: sdl12-compat. Indeed, most Linux distros ship sdl12-compat instead of the original SDL 1.2 library, so we're likely already using it. These days, sdl12-compat works with Civilization: Call to Power largely out-of-the-box, but that wasn't always the case (see below). We'll also need to set up a fake CD drive for music later on.

Act Ⅱ: To Crash or not to Crash

We triumphantly start Civilization: Call to Power with all of our library replacements and…

Paths Error: 'civ3.ldl' not found in asset tree.
Paths Error: 'Colors00.txt' not found in asset tree.
Colors00.txt line 0: Error: Could not open Colors00.txt
Colors00.txt line 0: Error: Missing number of colors.

Oh no! The game binary runs, but we get some weird error messages! What are these files, and why aren't they found?

If we look in the game's datafiles, we eventually find the missing files:

• ctp_data/english/uidata/layouts/civ3.ldl
• ctp_data/default/gamedata/colors00.txt

So the files are there: the game just can't seem to find them. One way to debug this is to use strace. strace is a utility which prints out all of the system calls a process uses.

If we run strace -e file ./civctp.dynamic_fixed, and search for the relevant files, we get:

stat64("/home/david/Games/CivCTP/ctp_da/englisish/uidata/layouts/civ3.ldl", 0xff8cfc54) = -1 ENOENT (No such file or directory)
stat64("/home/david/Games/CivCTP/ctp_da/englisish/uidata/layouts/civ3.ldl", 0xff8cfc04) = -1 ENOENT (No such file or directory)
stat64("/home/david/Games/CivCTP/ctp_da/defaulult/uidata/layouts/civ3.ldl", 0xff8cfc54) = -1 ENOENT (No such file or directory)
stat64("/home/david/Games/CivCTP/ctp_da/defaulult/uidata/layouts/civ3.ldl", 0xff8cfc04) = -1 ENOENT (No such file or directory)
stat64("/home/david/.loki/civctp/ctp_da/englisish/uidata/layouts/civ3.ldl", 0xff8cfc54) = -1 ENOENT (No such file or directory)
stat64("/home/david/.loki/civctp/ctp_da/englisish/uidata/layouts/civ3.ldl", 0xff8cfc04) = -1 ENOENT (No such file or directory)
stat64("/home/david/.loki/civctp/ctp_da/defaulult/uidata/layouts/civ3.ldl", 0xff8cfc54) = -1 ENOENT (No such file or directory)
stat64("/home/david/.loki/civctp/ctp_da/defaulult/uidata/layouts/civ3.ldl", 0xff8cfc04) = -1 ENOENT (No such file or directory)
stat64("/home/david/Games/CivCTP/ctp_da/englisish/gamedata/Colors00.txt", 0xff8a5b84) = -1 ENOENT (No such file or directory)
stat64("/home/david/Games/CivCTP/ctp_da/englisish/gamedata/colors00.txt", 0xff8a5b44) = -1 ENOENT (No such file or directory)
stat64("/home/david/Games/CivCTP/ctp_da/defaulult/gamedata/Colors00.txt", 0xff8a5b84) = -1 ENOENT (No such file or directory)
stat64("/home/david/Games/CivCTP/ctp_da/defaulult/gamedata/colors00.txt", 0xff8a5b44) = -1 ENOENT (No such file or directory)
stat64("/home/david/.loki/civctp/ctp_da/englisish/gamedata/Colors00.txt", 0xff8a5b84) = -1 ENOENT (No such file or directory)
stat64("/home/david/.loki/civctp/ctp_da/englisish/gamedata/colors00.txt", 0xff8a5b44) = -1 ENOENT (No such file or directory)
stat64("/home/david/.loki/civctp/ctp_da/defaulult/gamedata/Colors00.txt", 0xff8a5b84) = -1 ENOENT (No such file or directory)
stat64("/home/david/.loki/civctp/ctp_da/defaulult/gamedata/colors00.txt", 0xff8a5b44) = -1 ENOENT (No such file or directory)

When we look past all of the strange duplication, we note a few things:

• The game looks for files in both the directory it's installed to (in my case, /home/david/Games/CivCTP), and in another directory: $HOME/.loki/civctp
• It also looks for all of the files in both a language-specific directory (english), then falls back to a default directory.
• It looks for every file both with its original capitalisation, then entirely in lowercase. (For civ3.ldl, which is already lowercase, it just looks for the same file twice.)
• The path names are all corrupted: instead of looking at ctp_data/english or ctp_data/default, it's looking it ctp_da/englisish and ctp_da/defaulult.

It's the last of these problems which actually stops the game from booting: of course we won't find the files if we're looking somewhere which doesn't exist!

So, how do we find the source of this string corruption. Given that one of the few things that has changed is libc, there's a good chance that something in there (memory management, string manipulation, etc) is the source of the issue. A good way of finding memory issues is with Valgrind, so let's give it a go.

Wow — not only are there are a lot of errors, but the game actually started. And if we look in the logs, there are a few suspicious entries.

Source and destination overlap in strcpy(0xfec5b478, 0xfec5b47a)

Aha! That makes sense. strcpy() is a C standard library function which copies a string from one part of memory to another. Since this happens a lot, the implementation of it in glibc is often updated to run as quickly as possible, taking advantage of newer CPU features for better optimisations. This is all well and good as long as the programs using it are following the rules, but sometimes they'll accidentally depend on some detail of how the underlying implementation works.

Such is the case here: Civilization is calling strcpy() with overlapping arguments — moving a string within a buffer, overwriting part of itself, rather than copying it to somewhere else. In CivCTP's case, this is used to strip unwanted components from the a path. For example, within the game's code, a lot of paths include a "./" for the current directory. CivCTP uses strcpy to copy all of the text after the "./" over the top of the "./", neatly removing it.

Except, this only works if strcpy copies strings forwards. If so, each character will be overwritten only after it's been copied. But newer versions of glibc actually copy memory (including strings) backwards in a lot of situations, as it can be faster. When overlapping strings are copied backwards, the characters are overwritten before they're copied, meaning the wrong value is copied.

We're actually not the first to encounter this problem: it broke the Flash plugin a few years ago, annoying none other than Linus Torvalds himself.

Fortunately, we can work around this by providing our own version of strcpy, which handles the overlapping case properly. We can do this by making a shared library, and using LD_PRELOAD: an environment variable which allows us to force a library to be loaded into a process. LD_PRELOAD is used by a lot of tracing and debug tools, as well as things like the Steam Overlay, and is generally a good way of injecting code into a process on Linux.

Fortunately, it's very easy to write a version of strcpy() which works with overlapping inputs. The memmove() function is a version of memcpy() which handles overlapping inputs, and all we need to do is determine the length of the string, then move it:

char *strcpy(char *dest, const char *src)
{
        size_t len = strlen(src);
        char *ret = memmove(dest, src, len+1);
        return ret;
}

We can then compile this into a shared library, and run it. If we put that code in civctp_wrapper.c, we can build it like so:

cc -o libcivctp_wrapper.so civctp_wrapper.c --shared -fPIC -m32

We can then run Civilization with the LD_PRELOAD=libcivctp_wrapper.so environment variable set. Lo and behold, the game now runs!

This would probably be far enough, but there is some more work we can do…

Act Ⅲ: Silence of the CD-ROMs

So, the game works, but there's no music! While there are those who prefer to play their own music in games, I've always found the score by Michael Harriton & Mark Morgan to be pretty integral to the mood of the game, so it's an obvious omission! As hinted at earlier, this is because SDL2 dropped support for CD audio. This is largely because the way CD audio works on modern computers is actually quite different to how it worked on 1999-era ones.

On older computers, playing music could be quite computationally expensive: the CPU would have to keep reading audio data, performing any processing (e.g. decompression or synthesising), mixing it with sound effects, and writing it to the sound card. This could end up using a fair amount of CPU power, and if the processor was too slow, the resulting "skipping" in sounds was very obvious and annoying.

CD audio was different: the actual audio playback was done on the CD drive (indeed, many CD drives had their own headphone jacks and 'play' buttons), and all the CPU would do is send commands like "play this track" or "pause". The already fully-decoded, analogue audio signal would be sent to the soundcard, which would mix it with any other sound (e.g. sound effects from the CPU) before sending it out to the speakers. This was essentially "free" for the game: playing music would take no CPU at all.

Of course, with the advent of later CD-ROM drives and DVD drives, this feature was eventually removed, replaced with Digital Audio Extraction or DAE. This allowed the CPU to read the digital sound data from an audio CD, and then play it like any other audio. This removed the need for CD drives to have their own DACs, as well as making high-quality ripping of CDs possible (much to the eventual chagrin of the music industry). It does however require much more actual code and support from programs, or the operating system. Alas, CivCTP has no such support, nor does SDL, and while the Linux kernel does support DAE on most CD-ROM drives, it doesn't support emulating the older style of API using it, so programs need to be rewritten.

And to add to all this, many computers (particularly laptops) no longer have optical drives at all. What's a retro gamer with a penchant for music to do?

Fortunately, sdl12-compat comes to our rescue again. It supports emulating the SDL 1.2 CDROM API by using a set of MP3 files in a directory. Indeed, you may have noticed the debug message sdl12-compat prints out to let us know about this feature:

INFO: This app is looking for CD-ROM drives, but no path was specified
INFO: Set the SDL12COMPAT_FAKE_CDROM_PATH environment variable to a directory
INFO: of MP3 files named trackXX.mp3 where XX is a track number in two digits
INFO: from 01 to 99

To get these mp3 files, we'll need a CD drive, a ripping program (such as cdparanoia), and an mp3 encoder (I used ffmpeg). They should be named track02.mp3 to track09.mp3. The missing track01.mp3 is because the hybrid data CD-ROM / audio CD uses Track 1 to store the game data. To let sdl12-compat know about this, we need to create an empty track01.dat file.

We then need to set the SDL12COMPAT_FAKE_CDROM_PATH environment variable to the directory with our mp3 files. If we put the files in the current directory, alongside the libraries we've found and compiled, we can put together a command like:

SDL12COMPAT_FAKE_CDROM_PATH=`pwd` LD_LIBRARY_PATH=. LD_PRELOAD=libcivctp_wrapper.so ./civctp.dynamic_fixed

Act Ⅳ: A Crash Course in Concurrency

The good news is that Civilization: Call to Power now seems to be working. Indeed, you can play for quite a while without noticing any issues whatsoever. All seems well. However…

After a while, we can experience a hang: the mouse cursor still moves, but it's impossible to click on anything. Clearly, something is up!

To find this hang, we'll attach gdb to the game to observe what it's trying to do. We can then take a stack trace to find the code we're haning on. We get the following:

0xf7f37559 in __kernel_vsyscall ()
(gdb) bt
#0  0xf7f37559 in __kernel_vsyscall ()
#1  0xf788534f in __lll_lock_wait () from /lib/libc.so.6
#2  0xf788c1e9 in pthread_mutex_lock@@GLIBC_2.0 () from /lib/libc.so.6
#3  0xf6b9a810 in ?? () from /lib/libSDL2-2.0.so.0
#4  0xf7e1b433 in SDL_mutexP () from /lib/libSDL-1.2.so.0
#5  0x084e2e5c in ?? ()
#6  0x082a3697 in ?? ()
#7  0x082c283e in ?? ()
#8  0x082c2df4 in ?? ()
#9  0x080563c7 in ?? ()
#10 0x0805673e in ?? ()
#11 0x0805209e in ?? ()
#12 0xf7823855 in __libc_start_call_main () from /lib/libc.so.6
#13 0xf7823918 in __libc_start_main_impl () from /lib/libc.so.6
#14 0x0804d861 in ?? ()

This looks pretty horrifying, but it's just a list of what functions were being called when we took the stacktrace. Some of these have names (like __kernel_vsyscall(), which is a function to make a kernel system call), whereas others have just ?? because we don't have debug symbols for that library installed (or, in the case of Civilization itself, no debug symbols or source code are available to us at all).

Looking at this, we see that the game is trying to lock a mutex, and is waiting for the mutex to become available. We can tell from the functions pthread_mutex_lock() and SDL_mutexP(). These are the pthreads and SDL 1.2 functions to lock a mutex respectively (in sdl12-compat, SDL_mutexP() is implemented by calling SDL_LockMutex() from SDL2 (the ?? here, as I didn't have SDL2 debug symbols installed on the machine this crashed on), which in turn calls the standard pthread_mutex_lock() function from pthreads).

A mutex is a "lock", which allows a programmer to ensure that a piece of code and/or data is only accessed by one user (e.g. one thread) at a time. This is necessary to make sure that data doesn't get corrupted if two threads try to write to different parts of it at the same time, or that a half-written result of a calculation can't be confused with the final thing. They're a very subtle and important part of computer science, and often lead to these kinds of hangs. To use a mutex, you lock it before using the data or code it protects, and unlock it afterwards. Only one thread can hold the lock at a time, so attempting to lock an already-locked mutex will wait until that mutex is unlocked. What's happening here is that one of CivCTP's threads is waiting for a mutex to be unlocked, but it's never happening.

The traditional cause if this is called a deadlock, which happens when multiple mutexes (no, the plural isn't mutices, I'm afraid ;-() are needed to access something, and different threads lock them in different orders, leading to thread 1 holding mutex A and waiting for mutex B, and thread 2 holding mutex B and waiting for mutex A: neither thread can continue. But that's not actually what's happening here: only one thread is waiting on a mutex (we can confirm this in gdb by using thread apply all bt).

So, let's look at what the code calling SDL_mutexP() actually is. We can use a disassembler to look at whatever's at memory location 0x084e2e5c. gdb has a built-in disassembler — just use the disas command — but there are better disassemblers for static analysis, like Ghidra and IDA Pro. I put CivCTP through IDA Pro, and got the following function:

.text:084E2E30 ; =============== S U B R O U T I N E ===============
.text:084E2E30
.text:084E2E30 ; Attributes: bp-based frame
.text:084E2E30
.text:084E2E30 sub_84E2E30     proc near      ; CODE XREF: sub_82382A0+13↑p
.text:084E2E30                                ; sub_82A3678+1A↑p
.text:084E2E30
.text:084E2E30 var_18          = dword ptr -18h
.text:084E2E30 var_4           = dword ptr -4
.text:084E2E30 arg_0           = dword ptr  8
.text:084E2E30
.text:084E2E30                 push    ebp
.text:084E2E31                 mov     ebp, esp
.text:084E2E33                 sub     esp, 14h
.text:084E2E36                 push    ebx
.text:084E2E37                 mov     ebx, [ebp+arg_0]
.text:084E2E3A                 call    _getpid
.text:084E2E3F                 mov     eax, eax
.text:084E2E41                 mov     [ebp+var_4], eax
.text:084E2E44                 mov     eax, [ebx+4]
.text:084E2E47                 cmp     eax, [ebp+var_4]
.text:084E2E4A                 jnz     short loc_84E2E51
.text:084E2E4C                 inc     dword ptr [ebx+8]
.text:084E2E4F                 jmp     short loc_84E2E65
.text:084E2E51
.text:084E2E51 loc_84E2E51:                   ; CODE XREF: sub_84E2E30+1A↑j
.text:084E2E51                 add     esp, 0FFFFFFF4h
.text:084E2E54                 mov     eax, [ebx]
.text:084E2E56                 push    eax
.text:084E2E57                 call    _SDL_mutexP
.text:084E2E5C                 add     esp, 10h
.text:084E2E5F                 mov     eax, [ebp+var_4]
.text:084E2E62                 mov     [ebx+4], eax
.text:084E2E65
.text:084E2E65 loc_84E2E65:                   ; CODE XREF: sub_84E2E30+1F↑j
.text:084E2E65                 mov     ebx, [ebp+var_18]
.text:084E2E68                 mov     esp, ebp
.text:084E2E6A                 pop     ebp
.text:084E2E6B                 retn
.text:084E2E6B sub_84E2E30     endp

Phew: that's a fair bit of assembly, but it gives us an idea of what's happening. After a bit of prodding around, this appears to be yet another mutex lock function, wrapping SDL_mutexP. It's a bit more complicated, though, as it is maintaining a number of other values as part of the game's mutex structure, passed in as arg_0.

In particular, it looks like (having dug through a bit more code) the game's mutex structure has three members we care about:

+0h

An SDL_mutex topass to SDL

+4h

The PID of the process which locked the mutex.

+8h

A "depth": a count of how many times the mutex has been locked, minus how many times it's unlocked.

The purpose of this seems to be to implement a recursive mutex. It may seem strange, for instance, that there'd be this "depth" value, as almost by definition, a mutex can only be locked once, by one thread at a time. So the "depth" should always be 1 if the mutex is locked, and 0 otherwise. However, sometimes it's useful to let the same thread "lock" a mutex if it already has it locked. Since the thread already owns the mutex, this is harmless, and it makes reusing code much easier.

Of course, this isn't actually required: SDL mutexes are already recursive, so there's no need for CivCTP to implement these itself. We could probably just get rid of this code.

There's also the question of the PID check. This is supposed to verify that the owner of the mutex is the current thread: if it is, it's safe to just increment the count, rather than trying to re-lock the mutex. But a PID is supposed to be unique per-process, not per-thread. So it would seem we have a bug: the game is calling getpid() instead of gettid(), and therefore assuming that the mutex is already owned by the current thread, even when it isn't.

I'm not 100% sure for the reason behind this, but here's a guess. For a long time (including when CivCTP was released), the Linux kernel didn't actually have native support for threads, only for processes. Because UNIX systems used fork() to split a process, a lot of the things that other OSes did with threads could be done quite conveniently with processes anyway, and Linux added the clone() syscall, which allowed processes to share address space, much like threads. However, there were a number of issues with this approach in those cases where you really wanted a thread, so an implementation of threads for Linux was created called LinuxThreads. This implemented "threads" using clone(), and mostly worked. However, under LinuxThreads, every thread was technically a separate process, and so had its own PID. It wasn't until LinuxThreads was replaced with the NTPL that has better kernel-level support, and does let different threads within a process share a PID. If, as I suspect, CivCTP was written assuming that LinuxThreads was in use (and, as NPTL wouldn't exist for several years, this is likely), then getpid() would've worked.

So, now that we understand the problem, how do we fix it? We could patch the game to use gettid() instead of getpid(), but the better approach is just to rip out most of this mutex implementation, and have the game use SDL's mutexes directly.

We can do this by nopping out (replacing with the no-operation instruction 0x90) the bytes from 0x84E2E3A (the call to getpid()) through to 0x84E2E51 (the loc_84E2E51 label). We also need to nop out the equivalent code in the following function, which implements unlocking for the mutex. It lives from 0x84E2E7A (checking if the depth is 0), through to 0x84E2E85.

However, we have a problem: these offsets are memory addresses, not the offsets to that data in the civctp.dynamic binary. To make matters worse, if we've used patchelf above, it moves the contents of the binary around, so it might be different for everyone.

The best thing to do is to simply search and replace the code — after all, the actual data remains the same, according to this table:

With these changes, I'm yet to have a problem with Civilization: Call to Power on a number of modern systems. It's even possible to play (surprisingly well) on the Steam Deck (make sure you pick the Steam Linux Runtime as a compatibility tool).

Denouement: Making Camp

It's been a journey, but we've managed to get a game released in 1999 running pretty well on our modern systems.

To make things a bit easier, I've put together a script, civctp_fixup.sh which performs these steps for you, and creates a run_civctp.sh file to play the game. Note that you'll still need to provide your own libttf.so.2 and libsmpeg-0.4.so.0, as well as the game's music tracks (track01.dat and track02.mp3—track09.mp3). And you'll need a 32-bit x86 capable C compiler, patchelf, and perl installed for the script to work. Once run, it'll create the remaining files, and you can play Civilization: Call to Power by running ./run_civctp.sh.

Despite all of this, the game still isn't perfect. If we run it under a memory checker like Valgrind, we find a number of other issues (mismatched allocations / frees, invalid and uninitialised memory accesses, etc). If we play multiplayer (which works!), we might get a few (mostly harmless) error messages complaining that /sbin/ifconfig is not available (some systems still have it accessible, but it is deprecated in favour of ip). All this preloading, and patching, and finding ancient libraries is still kind-of horrible.

And wouldn't it be nice to get other Loki games working…? Several of them have the added complexities of OpenGL, and C++, and the unholy combination of the two. (The modern Mesa OpenGL stack uses C++ heavily, and will conflict very nastily with an old version of libstdc++.

I've got a version of libcivctp_wrapper which renames functions (so we can have the modern and new versions coincide), and implements a few more C++-y ones here. I've managed to get Alpha Centauri working with it, but it does get progressively more and more difficult.

Also, sdl12-compat didn't work with Civilization: Call to Power originally. A lot of things needed hacking and working around (enough to fill another entire article like this). Things like handling multithreaded rendering, the SDL 1.2 event thread, fixes to YUV overlays and CD audio support: there are a lot of fun things to deal with. You can see a list of most of them on the sdl12-compat issue tracker. A huge thanks to the inimitable Ryan C. Gordon for doing most of those fixes (and, you know, generally being awesome). And also to Sam Lantinga who not only invented SDL and still maintains it, but also worked on the original Loki Civilization: Call to Power port. And just for fun, the fact that Civilization: Call to Power works under sdl12-compat is mentioned in their recent GDC talk — neat!

And finally, in putting this article together, I wrote an extractor for the game's .zfs and .rim files, which might also be interesting to people who got to the end of this article.

Now, it's time for one more turn…!