Wc3 Map Optimizer 3.0b

It's true, the externalized strings cost quite some loading time.

Internalizing them is rather hard, that's what I found during the development of the widgetizer (due to special handling of " \ % characters and line breaks depending on the location the string is used in + there's some other crazy blizzard logic going on in several fields).
But it now works quite fine.
Also I've seen the WE generate redundant entries so I wonder if it's reusing entries at all. My guess is no.

@Yamaha: The program he's refering to is soon to be released.

Vex I am having some troubles.

Subject map file is about 1.5 megs. With all options checked but comments, it optomizes to 2 megs.

I was using Z-lib and the '65' compression setting.

I read the readme. I tried different combination of compression and z-lib, to no effort.

So I unchecked all options, except the compression, and it came out to 2 megs still.

No matter what I do.

I also noticed, no matter whats checked, it always deletes the list file; or I could give you more information. I am not experienced enough without the LF.

It has wavs, icons, maybe a model or two. If I open it in WINMPQ, and 'compact' the file, its back down to 1.5 meg.

Reccomendations?

Maps with sounds get bigger as I said in a previous post, people should stop importing sounds anyways.

I would fix the bug with sounds but I would first need to understand BlackDick's post...

Pitzermike: You can be completelly sure they never care about repeated stuff in the strings file.

---

New version, I really like the new feature, it is a method that will change the names of variables and functions in the map to the shortest possible values, it saves some kilobytes, makes the map faster, and as a secondary effect makes the life of unprotectors more difficult.

It is compatible with maps with ExecuteFunc based systems, and it is even able to remove unused global variables or functions that take arguments.

Of course, when you compact the map in WinMpq, it no longer uses the zlib compression

Can you tell us how to use this new feature? I'm not sure what it's asking for when I click the button.

If it's just asking for a common.j/blizzard.j file, then why not open up the war3x.mpq file and grab that one?

What about reading that file called readme.txt ?

And you can open mpqs with that button, you SHOULD open mpqs with that button, you should open war3patch.mpq

Here is some help:

First I have to mention that when adding files to MPQs the input buffer is split into separate "blocks" and those blocks are compressed individually. Keep in mind that we can compress each block with different compression types, thus we can have a file using 0x8 (PkWare) on it´s first block, 0x2 (Deflate) on the second block, 0x8 again on the third block and so on. A good MPQ library would try all allowed compression types on each block instead of using the same compression on all blocks, and that´s what I've planned for HeavyLocker altough I never finished it.

The second thing to mention is that waves use a "multi compression" scheme. Using 0x81 means we will first compress it with "Wave Stereo" and then recompress the outputt buffer with huffman, altough we could also use 0x2 (Deflate) or 0x8 (PkWare) here:

0x81 = Wave Stereo + Huffman
0x82 = Wave Stereo + Deflate
0x88 = Wave Stereo + PkWare

The idea is the same for "Wave Mono". 0x82 works nice, altough some testings show that the "standard" 0x81 works better. As I said above, it is up to the library check this and decide wich compression gets better results.

Here is the hard part: The Wave Mono or Wave Stereo compression are LOSSY, meaning we will LOSE information (sound quality) whenever using them. Try to compress a TXT file with wave compression and you will see what I mean, the result is garbage. We can only use Wave compression on the RAW PCM wave data of the inputt buffer. Thus, we some knowledge of the WAVE file format and also need to parse the input buffer and check some things BEFORE compressing it.

WAVE files have their file header and the "real' wave data is stored on a section called "data". ONLY THE "data" SECTION CONTENT IS OK to be compressed with Wave compression types: if we use it elsewhere, the result is garbage, as I stated above.

When we use non standard "block levels", , the 'data' section content may not start on the second block, corrupting the file.

I managed to make WAVE compression work when I relocated the "data" section content´s to the second "file block". The first block IS OK to be compressed using Deflete, PkWare.

Ho to do it? Read the WAVE file header, go trough the input buffer till you find the "data" section. Alloc a tmp buffer, copy the headers to the tmp buffer and adjust it, making sure the data section content will start at the second "block". Copy the data section to the tmp buffer and you´re OK to compress it using the wave "multi compression" scheme. I´ll post an example ASAP.

And here is the src (I used VC++ 7.1) for the padding thing. The code is not commented and ugly (pointer math) but it will help you.

This question may sound stupid now, but why does anyone want to use WAV files? Afaik you can also use MP3 files which can be compressed far more (I mean the sound data itself) and still maintain acceptable quality. Voices are still good at 32kbit/s. For effects you might have to go up to 64-96kbit/s, which is still very little. I see no reason to use WAVE files with their insane bitrates.

That´s true, MP3 get better results. altough old games (SC, D2) will not support them. For War3 we can even encode sounds as MP3, use a RIFF-WAVE header and save it as "*.wav". For old games, the ima adpcm compression still makes sense tough.

I didn't question the use of WAV in Blizzards legacy games. But we are talking about WC3, aren't we? But the idea with the MP3 encode is probably the best one. Tools like this one should do that. But in general users should use MP3 in the first place.

In the case of a tool encoding files to MP3 it might be difficult to select the target bitrate. As I previously explained, it depends on the type of sound how low you can go.

if wc3 supports mp3, does it support variable bitrate?
can someone test this?

It is incredible though, Blizzard used wavs in azeroth grand prix.

And if you want to replace unit sounds you will have to use waves.

---
Thanks a lot Blackdick.

I don't know what made Blizzard use WAVs in Azeroth Grand Prix. But I'd stay away from WAV whenever possible.

I know the case of the unit sounds. Blizzards failure to include a unit sound editor or the circumstance that WC3 doesn't load files from the UI folder when embedded into a map are to blame for this. But as Blackdick explained, you can still use MP3, by adding a WAVE header and changing the file extension. This is possibly the approach tools should take.

I think that making mp3 with wav headers is the best solution since their compression is much better.

Would have to investigate about formats unfortunally, arg more work

Updated with string optimization options, it can even remove editor suffixes