Spotify for Windows contains code so awesome that OllyDbg can't look at it without crashing

Spotify for Windows contains code so awesome that OllyDbg can't look at it without crashing.

The protection exploits, among other things, a Borland library bug that apparently has gone undetected since 1991. Let's start at the beginning.

If you haven't seen it, Spotify is a music player similar to iTunes, except that it uses a massive distributed music library. It's ad-supported (banners + occasional radio ads), but comes with a nice party mode: If you're using it as a jukebox for your party, you can pay a token $1 to disable the ads for the day.

OllyDbg is a lovely Windows debugger written by Oleh Yuschuk.
For some reason, they trust Mac users

The Macintosh version of Spotify has no anti-debugger protection at all, which is extremely odd for a product like this. It even logs readable debug output directly to the console, so you can pinpoint e.g. the ad loading routines using nothing but dtrace:

sudo dtrace -n 'syscall::write*:entry /execname == "Spotify" && arg0 == 2/
{ trace(copyinstr(arg1)); ustack(); }'

This prints a stack trace every time something is written on stderr. Just by looking at the stack traces, you can get a reasonable overview of what is going on:
"Found audio ad"

libSystem`write+0xa
libSystem`__sfvwrite+0xac
libSystem`fwrite+0x74
Spotify`0x71db6
Spotify`0x720fe
Spotify`0x71e5a
Spotify`0x71a9b
Spotify`0x71b48
Spotify`0xb461b
Spotify`0xab0d2
Spotify`0xab1fc
Spotify`0xebf68
Spotify`0x64ee4
Spotify`0x65bf7
Spotify`0x669a3
Spotify`0x66e05
Spotify`0x13051
Spotify`0x44ba1
Spotify`0x4b48b
Spotify`0x7ea8e


"Found banner ad"

libSystem`write+0xa
libSystem`__sfvwrite+0xac
libSystem`fwrite+0x74
Spotify`0x71db6
Spotify`0x720fe
Spotify`0x71e5a
Spotify`0x71a9b
Spotify`0x71b48
Spotify`0xb461b
Spotify`0xab0d2
Spotify`0xab1fc
Spotify`0xedc6a
Spotify`0x125a0
Spotify`0x130fe
Spotify`0x44ba1
Spotify`0x4b48b
Spotify`0x7ea8e
Spotify`0x7e9ec
Spotify`0x6ea6


Some other message

libSystem`write+0xa
libSystem`__sfvwrite+0xac
libSystem`fwrite+0x74
Spotify`0x71db6
Spotify`0x720fe
Spotify`0x71e5a
Spotify`0x71a9b
Spotify`0x71b48
Spotify`0xb461b
Spotify`0x107fd5
Spotify`0xb4a79
Spotify`0xce3db
Spotify`0xce587

The blue part is common to every log entry. Presumably the entries in the 0x71000 range correspond to the printf family of functions, the function at 0xb461b is a custom log() function, and the red part is ad-related :)
On Windows

On Windows, it's an entirely different matter. The application is suitably paranoid, and merely starting a debugger on the same machine is enough to make it run for cover.

The first stage of the loader is a simple xor/add decryption loop.
First decryption loop

Being lazy, you could try stepping through it using OllyDbg. If you do, an interesting thing happens. As soon as you try to step through the jump to the newly-decrypted code, OllyDbg locks up. If you go back and try again, it turns out you don't actually have to run the decrypted code for it to crash; merely looking at it is enough. The application hasn't accessed anything outside of its own memory space yet, so it shouldn't be able to influence the debugger. Maybe some obscure opcode sequence is able to throw Olly's disassembler into an infinite loop?

If you suspect a bug in your debugger, the obvious thing to do is to debug it. Now, if you try loading a copy of Olly into Olly, then loading Spotify into the innermost debugger, something rather baffling happens: Both debuggers crash. In fact, you can crash a whole stack of debuggers at once.
The Medusa float

By having a disassembly window open while stepping through the decryption routine, you can narrow it down to a single float-point constant which apparently is impossible to display (this screenshot is from the patched version of OllyDbg):
"Dangerous" floating point constant (in Spotify)

The crash happens in this routine, which is supposed to convert an 80-bit floating point number (long double) into an unsigned 64-bit integer (long long):
Float-to-int conversion routine (in OllyDbg)

In pseudo-C, the routine does something like this:

// convert a float to an unsigned integer, given 0 <= float < 2^64

void float80_to_uint64(float80* in_ptr, uint64* out_ptr) {
if (float < 2^63) {
// the number won't overflow, so it's safe
// to use a signed conversion
float80_to_int64(in_ptr, (int64*) out_ptr);
} else {
// 80-bit floats have an explicit 1 in the mantissa, so we can just
// copy the raw bits if the exponent is exactly 63
*out_ptr = *(uint64*)in_ptr;
}
}

... where the float80_to_int64 subroutine is implemented using the FIST instruction, which raises a floating point exception if the number does not fit into a signed 64-bit integer.

The red comment, of course, is wrong. An 80-bit float has 64 bits of precision, so with an exponent of 63 it is possible to pass in a value of

2^63 - 0.5 = 111111111111111111111111111111111111111111111111111111111111111.1

... which, while intially less than 2^63, will round up to 2^63 when converted to an integer. That number is not representable as a signed 64-bit int, and the FPU throws an exception. The debugger dies, and since the bad number is still on the FPU stack, it is able to kill the next debugger as well.

To see it in action, try setting one of the FPU registers to 403D FFFFFFFF FFFFFFFF hex. As soon as the last character is typed, the debugger dies:
Try this at home

The faulty routine is actually part of the run-time libraries shipped with the Borland C++ compiler. To verify, you can compile the following program with bcc32 and watch it crash in printf().

The middle window shows the result as compiled with GCC; the right shows the buggy version: One bit of accuracy is lost in the conversion, and the last printf() crashed.
A workaround

Load OllyDbg in itself, and search (Ctrl-S) for the instruction sequence

fld [ra]
fistp [rb]

When you find this routine,
Original float-to-int routine

... replace it with something like
Patched float-to-int routine

The and will clear the least significant bit of the float before doing the conversion, narrowly avoiding the bad case.

To make room for the added code, the patch exploits the fact that the function is called with input and output to the same buffer, so the last 5 lines of the original are unnecessary.

http://www.steike.com/code/spotify-vs-ollydbg/