SFmpq fast version

This dll is based on SFmpq static 1.0.7.4 and has some speed
improvements. I changed it´s version to 1.0.7.6, so you can
distinguish them by checking the version.
Everything else works the same as the original 1.0.7.4.
USAGE: Just replace your SFmpq.dll with the one included with
this package.

1.0.7.6 Fixed some bugs. Also, since WC3Campaigns is now back on business
I´m uploading it again. I had to compile this thing again, coz I didn´t found any binary
working version on my PC. Here are my BenchMark results against the latest Game.dll (1.20b)
Athlon XP 2000+ 1658 MHz (12,5 * 133) L1=64KB L2=256KB
RAM = 2x 512 MB @ 2,5/3/4/7/2
HD: PATA Maxtor 120 GB 7200 rpm 2MB cache


And here is the original 1.0.7.4 results. As you can see, they look similar, except from the zlib-deflate compression, wich got pretty good improvements.

It sounds great, and it is also great you are back to this stuff, anyways I think the programming forum is a better place for it

I think it is amd specific, I tested it in my pentiu 4 , and rebuilding with Zlib was leaving an empty mpq

Thanks for the info. I´ll then develop separate versions. Hold on a bit and a P4 version should be coming along.

I think i fixed the errors. Here is a new beta version.
NOTES: I need someone with an INTEL cpu (P4) to test this library for me.
It is using a "fast" memcpy() routine wich may have some problems on
INTEL based CPUs. On my AMD Athlon XP it is working nice.
Please read the included "ReadMe.txt" and post some results. BTW, it is about 3 times faster than the original version. :o

EDIT:
[March,19,2005] New version: Fixed some bugs, reduced library size a bit, changed version to 1.0.7.5 to avoid misunderstandings, MpqBench now tests 3 compression types.

I will go home and test it

BTW, it seems I fixed the Zlib compression but messed up with the other compressions, damm I'm stupid lol.

Trying to locate that, I will probably have it fixed by tomorrow. Meanwhile, I´d appreciate some 1.18 Game.dll benchs.

I tested the benchmark today and I have to say that I get virtually the same results for both versions of SFMPQ. They were always in the same range for both versions of the library. I don't know whats wrong here.


System:
Athlon64 3000+ (754)
1024MB RAM (DDR400 Single-Channel)

Uploaded new version, fixed some things, improved MpqBench. See my post above for download.

Starcraftfreak, would you mind running the test again, with this new version?

Optimized is slower (~ 100ms in writing, 10ms in reading for the last test). I don't want to post the results now, since I'm running some other programs as well, which makes them worse. I made another run with the older version after posting the results and I scored like 900/126 after I closed all programs.

But what is wrong with that optimization? Could it be that the Athlon64 is not suited for those?
Maybe you should also try SSE/SSE2.

Edit:
I found out what was wrong with the "regular" sfmpq.dll. It wasn't the regular one. It was the one of your WinMPQ modification which supports buffer size modification for new files. It was dated somewhere 2004. It contains the string "Modified by BlacKDicK based on SFmpq_static version 1,0,7,4" in the file comments.
Oddly, when I wanted to create a new UMSWE build to fix a little bug in the current one, it would always corrupt the listfile. I double checked and messed around a lot until I came up with the conclusion that something has to be wrong. Then I overwrote my SFmpq.dll with the original one from the WinMPQ download. The result was that it wrote a correct listfile into the MPQ but was considerably slower. Then I knew why this dll was not faster than the one "from WinMPQ". I also wonder how this problem occured. The last version of UMSWE was built with it and just everything is fine. I was using this dll for 4 months now and I never experienced any problems until today. Now I put your latest optimized version into my WinMPQ directory and it works just fine

I wonder if you ever released a similiarly optimized version some time ago (must be back in fall 2004). Today I had problems building a new version of UMSWE with the script which builds the MPQ with WinMPQ for me. Oddly, I created a new UMSWE build last week and it works fine.

I'll probably post new benchmarks a bit later.

Yes I did, and I posted it here in this same topic. That version had some bugs when using the PkWare compression. Probably you only faced the bug when you first added files using this type of compression. Maybe the (listfile) worked before coz it got compressed with Zlib but now (dunno why) it was getting compressed with pkware.


This library can´t be slower than the original SFmpq. There are 3 separate things wich will help increase speed:
1) It is using zlib 1.22 , wich is faster.
2) I compiled the zlib using a special "Asm Version" wich can be found on the zlib package.
3) It is using the "fast" memcpy routine wich I got on the AMD dev site.

Even If we supose that this fast memcpy is not getting good improvements at all, there are still 2 other things helping the speed. Also, this library can have problems with Intel CPUs (I am not sure if fast memcpy is also using specific AMD code or just SSE) but on your K8 it should work flawlessly. I am pretty sure it ain´t using SSE2 coz my CPU (Athlon XP) does not support SSE2 and the library works just fine.

Btw, can you do a MpqBench using this new version and the regular SFmpq? I am curious about K8 performance coz I´ll probably buy one for me on july.

Thanks for the info. It seems that this version is working nice. If you find anything new, please let me know.

Here are my results (I actually did those some days ago):

I know, it's 1.8, the special WinMPQ ver, but I think it doesn't matter. It shows the difference quite clearly.

I doesn't look that impressive. But you have to take into account that that is the 3000+ on Socket 754 (2GHz, 512KB L2). Current Socket 939 models have dual-channel memory. Additionally I believe that it is possible to optimize that code even further for K8.

BTW, can you give me the specs of the machine you did the benches in the readme?

Sure, here we go:
Athlon XP 2000+ 1666Mhz (Thorton, 266 FSB, 256KB L2)
A7V266-E
1x Samsung 512MB DDR 400 (@ 266Mhz 2,2,2,5)
GF4MX400
2x PATA Samsung 120GB (7200 rpm, 2MB) @ MOBO RAID-0

Nothing unusual on it, the memory at low latency and the RAID-0 help general performance bit.

Thanks for the info. I will clean up the code a bit and if you´re interested, I can send it to you; maybe you can try some SSE2 or K8 stuff on it.

I got delphi 2005, when it is so easy to get new software as in this country, you don't need to know what you are exactly buying.

But anyways it allows to make .net apps, but the problem is that the current SFMPQ.dll is not too friendly with .net argument wise, you should try using some .net program and sfmpq header to see what I mean.

Since you are the only guy that seems to be updating it, could you try to get a .net compatible version?, no rush needed, just 2 or 3 years before a new game comes out, There is no need for an app to be .net at this time, but it will sooner or later get like that

.net assembler is much slower than 32bit original assembler codes, that's why I only install win32 part of d2005 on my pc