Quantam's MoPaQ Specs Released

07-02-2005, 04:09 AM

Note that it's in "beta" (v0.9). I'm not calling it v1.0 until it has a chance to be peer-reviewed (which is why I'm posting it).

Also, I should note that this is strictly a format spec. It's not the frilly, plain-English-for-the-common-man style I intended to do with Inside MoPaQ, nor does it contain any of the extra info Inside MoPaQ would have had, such as how Storm works internally, how to program Storm and MPQAPI, etc.

You can download it as an attachment from http://forums.samods.org/index.php?showtopic=4688 (registration not required). I'm sticking it there for now because CC's FTP info has changed since I last used it, and it's been 2 days now and I still haven't gotten a reply from anybody who knows the info

07-03-2005, 10:42 PM

BlacKDicK

Nice work quantam, here is a small contrib.

Code:

unsigned long HashString(const char *lpszString, unsigned long dwHashType)
{
    assert(lpszString);
    assert(dwHashType <= MPQ_HASH_FILE_KEY);
    
    unsigned long  seed1 = 0x7FED7FED;
    unsigned long  seed2 = 0xEEEEEEEE;
    int    ch;

    while (*lpszString != 0)
    {
        ch = *pFileName++;
        if (ch >= 0x61 && ch <= 0x7A)
        {
           ch -= 0x20;
        }else{
           if (ch == 0x2F) ch = 0x5C;
        }
        seed1 = dwCryptTable[(dwHashType * 0x100) + ch] ^ (seed1 + seed2);
        seed2 = ch + seed1 + seed2 + (seed2 << 5) + 3;
    }
    return seed1;
}

There is a small bug on ladislav code: we need to convert "/" to "\" . My changes also avoid a call to toupper, speeding things a bit.

Second thing is simple: lines inside "(listfile)" are not separated with "0x0D0A". Check some starcraft maps listfiles, they just use 0x0D or 0x0A (donÂ´t remeber exactly).

Third thing: hashtableoffset and blocktableofsfet (MPQHEADER) are unsigned, meaning we can find those tables on "negative" offsets (based on the current mpqheader offset).

07-04-2005, 03:59 AM

Quantam

Quote:

Originally Posted by BlacKDicK

There is a small bug on ladislav code: we need to convert "/" to "\" .

That's not a bug, technically. The actual hash algorithm used in MPQs does indeed distinguish '/' and '\'. SStrHash (which is not the MPQ hash algorithm), on the other hand, treats the two as identical.

As for toupper, I just let the compiler do what it thinks is best. The optimal way to do that conversion is to use a lookup table, and to inline the toupper function. But I'm not that picky.

Quote:

Second thing is simple: lines inside "(listfile)" are not separated with "0x0D0A". Check some starcraft maps listfiles, they just use 0x0D or 0x0A (donÂ´t remeber exactly).

Ahh, I didn't know that. Everywhere I looked (the main game archives) used \r\n line breaks.

Quote:

Third thing: hashtableoffset and blocktableofsfet (MPQHEADER) are unsigned, meaning we can find those tables on "negative" offsets (based on the current mpqheader offset).

Yeah, probably. I actually didn't distinguish between signed and unsigned numbers in my spec because for most of the stuff I don't KNOW which it is. It definitely makes sense for them to be unsigned, but I can't trust Blizzard to always do the logical thing :P

07-05-2005, 09:08 AM	#4
Soar	I think ch &= ~0x20; is faster than ch -= 0x20;

07-09-2005, 01:30 AM	#5
Quantam	Posted v0.91. And by the way, a subtract and binary AND take the same number of cycles (0.5). I'll be really impressed if somebody figures out a way to do that case conversion without a conditional jump (and without a memory lookup, which adds on bus time).