Compressed files

Compression is the process of encoding data in such a way that it consumes less storage capacity. This is done using special algorithms that identify and shrink down unnecessary and repeating data patterns using back-references. Some of Pokémon Ranger's files are compressed using the LZ10 algorithm, a derivative of LZSS that is used in many Game Boy Advance and Nintendo DS games.

In Pokémon Ranger, testing if a file is compressed with LZ10 is very simple. Whenever the game loads a file, it appends .cat to the file path and checks if that file exists. If it does, the file likely contains compressed data. Otherwise, it appends .dat to the path and tries to load that file instead. Additionally, the very first byte in the compressed data needs to be equal to 0x10. If these two conditions are met, the game will attempt to decompress the data.

The following Java code decompresses an LZ10-compressed array of bytes. getUInt8, getUInt24 and getSInt32 are functions to obtain integers of the respective bit lengths from a buffer.

public static byte[] decompress(byte[] in) {
    // Get decompressed size
    int outSize = getUInt24(in, 1);
    int posIn = 4;
    int posOut = 0;

    if (outSize == 0) {
        outSize = getSInt32(in, 4);
        posIn = 8;
    }

    byte[] out = new byte[outSize];

    // Decompress data
    while (posOut < outSize) {
        int flags = getUInt8(in, posIn++);

        for (int fi = 0 ; fi < 8 && posIn < in.length ; fi++) {
            if ((flags & (0x80 >> fi)) != 0) {
                int token = getUInt8(in, posIn++);
                int token2 = getUInt8(in, posIn++);
                int disp = ((token & 0xF) << 8) | token2;

                int lenCopy = (token >> 4) + 3;
                int offCopy = posOut - disp - 1;

                for (int i = 0 ; i < lenCopy ; i++)
                    out[posOut++] = out[offCopy + i];
            }
            else
                out[posOut++] = in[posIn++];
        }
    }

    return out;
}