davidgow.net: Fixing the Age of Empires Scrolling Bug

Background

The Age of Empires games form a very popular series of historical RTS games for the Windows platform. Unfortunately, games based on the Genie Engine suffer from a bug whereby the screen can get stuck scrolling in one direction, particualrly after switching away from the game using Alt+Tab. There's no way to then stop scrolling: pressing the arrow keys or moving the mouse to the edge of the screen can provide at best a temporary reprieve.

This bug was particularly evident when running under WINE, a Windows API implementation for platforms such as Linux. A bug report has long existed against WINE to fix the issue (which has happened), but there are a notable number of reports of it happening on Windows. Some of these link it to software bundled with the Zune media player, and others are clearly triggered by something else.

As the bug renders the game unplayable (at least until the game is restarted), and I played frequently on Linux, I took to investigating it, and eventually tracked the issue down. I've documented it in more detail on this page.

The Bug

The bug is caused by a misunderstanding of the the windows GetKeyboardState() function's return value. As the above MSDN link states, it returns — for each key — a byte whose least significant bit signifies that the key is "toggled" (à la Caps Lock), and whose most significant bit signifies that the key is pressed.

The specified possible values for the key state are therefore:

Bit:	7	6	5	4	3	2	1	0
Unpressed:	0	?	?	?	?	?	?	0
Pressed:	1	?	?	?	?	?	?	0
Toggled:	0	?	?	?	?	?	?	1
Toggled & Pressed:	1	?	?	?	?	?	?	1

The '?' bits are unspecified, and could be either 1 or 0. In practice these are almost always 0. The 'almost' in that sentence is the cause of the bug.

Age of Empires checks if a key is pressed by checking if the byte has a value greater than 1. This works when the undefined bits are all 0, but if any of the undefined bits are 1 then the game can assume the key is pressed, even when it isn't.

Of course, if the game thinks the arrow keys are pressed, then the screen will scroll uncontrollably, hence the bug.

The Fixes

There are several different ways to work around or fix the issue:

Disabling Scrolling with Key Rebindings

In Age of Empires II and later (including the remakes), the keys used for scrolling can be configured in the settings. If you try to assign a key that has already been used, it's previous function will be disabled (showing up as '???'). By repeatedly changing the keys used, it's possible to disable all of the scrolling keys in settings.

This has the downside that you can no-longer scroll with the keyboard at all — though scrolling using the mouse is still possible — but it does work, and can be done in the middle of a game if necessary.

Modifying `GetKeyboardState`

Given the problem only occurs if these undefined bits are 1, if GetKeyboardState() guaranteed they'd always be 0, the issue would never occur. Obviously, this required modifying to OS, or intercepting calls to the function.

WINE actually does this, as of version 5.9, thereby fixing the bug by default. The GetKeyboardState() function implementation now ANDs all values with 0x81, which effectively zeroes out those middle bits. See this patch by Markus Engel, and the WINE bug report for the issue.

For those people using Windows (or perhaps other versions of WINE), there are two open-source projects which intercept calls to GetKeyboardState() to fix this issue.

sftscrollbugfixer uses Microsoft's Detours library. It injects a custom DLL into the executable, and patches the GetKeyboardState() function (and the similar GetKeyState() function) to redirect to a new function, which calls the original and then ANDs the values with 0x81.

aoe_keystate_fix by udf similarly provides a new implementation of GetKeyboardState() and related functions, which AND the values by 0x80, so only the top bit is set, and all the others are zeroed. (Age of Empires doesn't care about the 'toggled' value in the lower bit anyway, so this works fine.) Instead of using the Detours library, these functions are compiled into a DLL called user33.dll, and the game executable is modified to load user33.dll instead of user32.dll, which is the system library which implements GetKeyboardState(). user33.dll also forwards all of the other functions the game needs from user32.dll to the real system library.

Patching the Game

The final way to fix the issue is to modify the game code itself so that it handles the result from GetKeyboardState() correctly. This typically involves reverse-engineering the game to find the calls to GetKeyboardState(), and then modifying the instructions which compare the results.

Unlike the other fixes, this one is very dependent on the exact version of the executable, as the code moves around and changes when new versions or games are released.

In practice, most of this code is very similar across the different games and versions. It (almost) always takes the form of a cmp instruction to compare the value with 1, followed by a jbe instruction which skips past the next block of code if the result was less than or equal to one.

For example (from Age of Empires 1.0c):

.text:005124DA:  80 7c 24 48 01          cmp    BYTE PTR [esp+0x48],0x1
.text:005124DF:  76 34                   jbe    loc_512515

One way of fixing the issue is to replace these instructions with a bitwise AND against 0x80 (and a jump if not zero instead of a jump if below or equal):

.text:005124DA:  80 64 24 48 80          and    BYTE PTR [esp+0x48],0x80
.text:005124DF:  74 34                   jz     loc_512515

The games each have several of these checks, all (or at least most) of which need to be patched to fix the bug completely. While the majority of these are compiled to very similar machine code instructions, there are a few variations.

The table below lists the patches reqiured for each version. By using a hex editor (such as XVI32 or Okteta), it's possible to go to the specified offset and replace the instructions necessary. There's also a download link for a pre-patched executable you can replace the original with.

For brevity, where an offset is listed, but no patch is specified unspecified, the changes to make match the above example, and take the form: 80 7C ?? ?? 01 76 → 80 64 ?? ?? 80 74

Version	Patches	Download
Age of Empires v1.0c	0x1124AD: 3C 01 76 → 24 80 74 0x1124DA 0x112515 0x112550: 80 7C 24 47 01 0F 86 1A 02 00 00 → 80 64 24 47 80 0F 84 1A 02 00 00 0x1125A0 0x1125B1 0x1125C0 0x1125D1	empires_1_0c_scrollfix.exe (1.5M)
Age of Empires: The Rise of Rome v1.0a	0x118ADD: 3C 01 76 → 24 80 74 0x118B0A 0x118B45 0x118B80: 80 7C 24 47 01 0F 86 1A 02 00 00 → 80 64 24 47 80 0F 84 1A 02 00 00 0x118BD0 0x118BE1 0x118BF0 0x118C01	empiresx_1_0a_scrollfix.exe (1.5M)
Age of Empires II: The Conquerors v1.0c	0x12B8C: 3C 01 76 → 24 80 74 0x12BBA 0x12BF5 0x12C30: 80 7C 24 47 01 0F 86 38 02 00 00 → 80 64 24 47 80 0F 84 38 02 00 00 0x12C80 0x12C91 0x12CA0 0x12CB1 0x31CCF: 80 3C 08 01 0F 97 C0 → 80 24 08 80 0F 95 C0 0x1B1C19: 3C 01 76 → 24 80 74 0x1B1C27: 80 7C 24 3F 01 0F 86 DB 00 00 00 → 80 64 24 3F 80 0F 84 DB 00 00 00	age2_x1_1_0c_scrollfix.exe (2.6M)
Star Wars Galactic Battlegrounds	See Star Wars Galactic Battlegrounds Patch
Age of Empires II HD (2013) v5.0	See this forum post.
Age of Empires II HD (2013) v5.8.3062235 Steam Build ID: 3062235	0x30B972: 80 BD 21 FF FF FF 01 76 05 → 80 A5 21 FF FF FF 80 74 05 0x30B980: 80 BD 23 FF FF FF 01 0F 86 C8 00 00 00 → 80 A5 23 FF FF FF 80 0F 84 C8 00 00 00 0x46BC02: 80 3C 01 01 77 12 → 80 24 01 80 75 12 0x484DE6: 80 BD 22 FF FF FF 01 8B D8 76 55 → 80 A5 22 FF FF FF 80 8B D8 74 55 0x484E46: 80 BD 24 FF FF FF 01 76 59 → 80 A5 24 FF FF FF 80 74 59 0x484EA8: 80 BD 21 FF FF FF 01 76 5C → 80 A5 21 FF FF FF 80 74 5C 0x484F13: 80 BD 23 FF FF FF 01 0F 86 D6 02 00 00 → 80 A5 23 FF FF FF 80 0F 84 D6 02 00 00 0x484FA6: B8 01 00 00 00 → B8 80 00 00 00 0x484FAB: 38 85 21 FF FF FF 76 05 → 20 85 21 FF FF FF 74 05 0x484FB8: 38 85 23 FF FF FF 0F 47 D8 → 20 85 23 FF FF FF 0F 95 C3 0x484FC1: 38 85 22 FF FF FF 76 05 → 20 85 22 FF FF FF 74 05 0x484FCE: 38 85 24 FF FF FF 0F 47 F0 → 20 85 24 FF FF FF 74 01 46	AoK_HD_3062235_scrollfix.exe (7.2M) Replaces `AoK HD.exe` or `Launcher.exe`
Age of Empires: Definitive Edition	Due to the game using anti-tampering and code obfuscation techniques, no patch is available.
Age of Empires II: Definitive Edition	Due to the game using anti-tampering and code obfuscation techniques, no patch is available.

For Age of Empires II HD (2013), the aoe2-scroll-bugfix project by kukkerman will automatically patch most versions of the executable. Instead of replacing the CMP instruction with an AND instruction, it simply compares with 0x7F, which achieves the same goal. Both a C++ and Python version are available.

In conclusion…

As you can see, there are lots of different ways to work around the issue. If you're using WINE (or Valve Software's Proton), then running version 5.9 or newer is probably the simplest way of fixing the issue. You should probably check the WineHQ AppDB pages (Age of Empires, Age of Empires II) or the Proton bugtracker entires (Age of Empires II HD (2013), Age of Empires: Definitive Edition, Age of Empires II: Definitive Edition) for the game and version you're playing to make sure you stay on top of this and other bugs.

If you're not running wine, you'll have to pick whichever method best suits you. The various tools all work (and there's a chance that some of them could be made to work with the newer Definitive Editions with a bit of work).

Do be warned that modifying the game — or injecting code into it at runtime — may trip anti-cheating systems, and could get you banned from online play. I've not heard any reports of this happening, but this is exactly the sort of thing anti-cheat systems are supposed to detect, so don't blame me if it all goes pear-shaped.

I have tried to contact some of the game's developers in the past in the hope they'd fix the bug in an update, but haven't had any response. If you work on Age of Empires, please fix this issue! :-)

Finally, it's worth noting that there may be other bugs which have a similar effect. Any form of stuck key bug, or having the mouse cursor off the edge of the screen (particualrly common with dual-screen setups) could look similar. Those are usually easier to fix though (mashing the keyboard and shaking the mouse wildly will probably do the trick).

With all that said, I hope something here works for you. Happy empire-building!
— David

Fixing the Age of Empires Scrolling Bug