HomeUser Control Panel (unavailable in archive)ForumsTutorialsArt GalleryResourcesMaps

dll'less MPQ library

06-21-2008, 01:39 PM#1
d07.RiV
I'm finally ready to present my [opensource] MPQ library that requires no additional dll's and is cross-platform (you just need to change file system functions to work for your platform).

The only things that are missing are compression/decompression functions other than zlib (I'm not sure inserting the huge pkzip, huffman and wave compression code is worth it because it isn't used in WC3, and bzip2 is easy to add but the bzip2 library requires a dll to work).
I haven't tested it with wow's MPQ archives but I suppose it would work if you add bzip2 library.

So, all you need is a few source files and the zlib library (http://zlib.net/ - build it for your system) which is a .lib and still doesn't require additional dll's).

If you ask why not just use sfmpq, you can customize this library to add your features that would be much harder to do with sfmpq (also you don't need to bother with dll imports).

Included are the library source files (rmpq.h rmpq.cpp filesys.cpp mpqcompress.h mpqcompress.cpp), test (test.cpp) and zlib compiled for Visual C++ 2008.
06-21-2008, 07:22 PM#2
Strilanc
war3map.imp in the map MPQs uses Huffman Encoding. Luckily most people don't need to read that one.

Your code needs comments. I count 2 in all of rmpq.cpp (a file with 2600 lines of code). Actually I'm surprised the line count is so high.
06-21-2008, 08:53 PM#3
BlacKDicK
Quote:
huffman and wave compression code is worth it because it isn't used in WC3

I wouldn´t be that sure....
06-21-2008, 09:30 PM#4
d07.RiV
Hmm all right I'm going to add that but I've never hit that problem
Oh war3map.imp uses huffman? Then I definetly have to change it since my AMHS injector adds an image file and adds a line to .imp file (it worked when I used it on a map with no imports).

And yea I don't usually comment code much because I'm not used to sharing code, if I have time I'll add some comments. Also I thought it's better if the library uses as little files as possible so I put most of the code in one file.
06-25-2008, 05:47 PM#5
d07.RiV
Are you sure war3map.imp uses huffman? I always get zlib when I save a map.
06-26-2008, 02:15 AM#6
Strilanc
I get "Can't decode Huffman" exception when extracting the file using my implementation, so either it's Huffman or my constants are wrong (mine only does DEFLATE but is only ~700 lines total, including comments).

Code:
mpq = new MPQArchive(path)
f = new MPQFileStream(mpq, "(listfile)")
'it doesn't get easier
06-27-2008, 02:25 AM#7
PandaMine
Im assuming that you would need to compile zlib for linux as well (if you want to compile it on linux that is)

Also is rlib.h part of zlib?

Otherwise it looks aweasome, gw.

Quote:
Originally Posted by Strilanc
war3map.imp in the map MPQs uses Huffman Encoding. Luckily most people don't need to read that one.

Your code needs comments. I count 2 in all of rmpq.cpp (a file with 2600 lines of code). Actually I'm surprised the line count is so high.

Agreed, that MPQ flush function is a bitch to read (its like 300 lines long)

There seems to be a lot of copy/paste code, think d07 didnt abstract it enough
06-27-2008, 04:26 AM#8
Vexorian
Quote:
Im assuming that you would need to compile zlib for linux as well (if you want to compile it on linux that is)
It would just need your favorite distro's zlib1g-dev package.

... Assuming it is gcc friendly, quite the source code in the zip certainly isn't, all too visual studio dependent so you wouldn't need some heavy porting...
06-27-2008, 05:14 AM#9
PandaMine
Oh yeah thats right, forgot about the zlib package
06-28-2008, 07:23 PM#10
d07.RiV
Sorry rlib.h was me forgetting to rename it properly, it should be rmpq.h (I was renaming it from mpqlib to rmpq because mpqlib was already taken, and I didn't test it on a compiler). I also changed a few things (e.g. added MPQLOADER that can store several archives and load files like WC3 would).
Also I think I need to change typedefs from uint32 to something else because you don't get any warning/error if you use wrong variable type, e.g. call MPQClose on a file instead of an archive.

Also what is visual dependant? Apart from the "configuration section" in rmpq.h (the typedef's) and filesys.cpp which uses visual-specific functions to allow 64-bit file offsets (you can always use normal functions if you don't want to support burning crusade archives, I don't play WoW so I don't even know if such large archives are used in client side; plus you would need bzip2 library).