HomeUser Control Panel (unavailable in archive)ForumsTutorialsArt GalleryResourcesMaps

Benchmarking Operations

03-18-2008, 10:56 PM#1
Strilanc
I'm writing a system to assign an array index to arbitrary handles. Similar to what Cohadar's PUI ( http://www.wc3campaigns.net/showthread.php?t=96928 ) does, but with explicit creation and destruction and not only units.

I have it mostly finished. But now how do I measure how 'good' it is? Code runs 'instantly' in game time, making it difficult to get any real measurement in running time.

I'll include the system so anyone helping has a rough idea what the heck I'm trying to measure. I want to know the times on CreateHandleIndex, GetHandleIndex, and DestroyHandleIndex. Once I know how to measure time I can probably take care of the rest, like making sure I don't end up with collision free hash tables all the time.

I can get a rough operation count by assuming it translates to assembly... GetHandleIndex has 10-20 operations if there are no collisions, I think.

Collapse JASS:
///////////////////////////////////////////////////////
/// Strilanc's Handle Index Library
/// Last Updated: March 18, 2008
/// Version: Beta-2
///////////////////////////////////////////////////////
/// Description:
///  This library provides functions to create, destroy, and retrieve array indices
///  for handles (eg. units, locations, timers, etc). The library is easy to use
///  because it takes care of making sure the array indices don't collide, the same
///  handle doesn't have two associated indices, etc.
///
/// Explanation:
///  - The library is essentially a hash table with a very simple hash. A handle is
///  converted to a valid array index using H2I mod TABLE_SIZE. Collisions are
///  handled by probing.
///
/// Usage:
///  - Copy this trigger to your map
///  - Set the system settings just below this big comment block
///  - Create a handle index for any unit or whatever you want to work with using CreateHandleIndex
///  - Never create a handle index only if it doesn't exist; this interferes with other libraries
///  - Retrieve the handle index using GetHandleIndex
///  - Destroy the handle index when you're done with it using DestroyHandleIndex
///
/// Notes:
///  - Multiple libraries can use this library, even if they don't know about each other. This
///  works because when a handle index is created twice, it needs to be destroyed twice before
///  it is 'really' destroyed.
///  - Performance will drop if number of handles becomes large, due to probing.
///  - The constant HANDLE_INDEX_NOT_FOUND is returned when no handle index can be given.
///////////////////////////////////////////////////////
library HandleIndexLib initializer initHandleIndexLib
    globals
        //should the system show debug messages?
        private constant boolean SHOW_DEBUG_MESSAGES = true
    endglobals
    globals
        constant integer HANDLE_INDEX_NOT_FOUND = -1
        constant integer EMPTY_BUT_USED = -1
        private constant integer MAX_HANDLES = 6000
        private constant integer TABLE_SIZE = 8190

        private integer array tableValues
        private handle array tableHandles
        private integer array tableUsedCounts
        private integer array tableProbeCounts

        private integer numHandles = 0
        private integer numProbes = 0
    endglobals
    
    private function H2I takes handle h returns integer
        return h
        return 0
    endfunction

    ///Returns the array index for the handle h
    function GetHandleIndex takes handle h returns integer
        local integer i
        if h == null then
            return HANDLE_INDEX_NOT_FOUND
        endif
        
        //hash
        set i = H2I(h) //hope H2I(h) is positive
        set i = i - (i / TABLE_SIZE) * TABLE_SIZE
        
        //probe
        loop
            //check slot
            exitwhen tableUsedCounts[i] == 0 //end of probe-able area
            if tableHandles[i] == h then
                return i
            endif
            //next slot
            set i = i + 1
            if i >= TABLE_SIZE then
                set i = 0
            endif
        endloop        
        return HANDLE_INDEX_NOT_FOUND
    endfunction

    ///Allocates an array index for the handle h
    function CreateHandleIndex takes handle h returns integer
        local integer i
        local integer n
        if h == null then
            return HANDLE_INDEX_NOT_FOUND
        endif
        
        //Check if handle already exists
        set i = GetHandleIndex(h)
        if i != HANDLE_INDEX_NOT_FOUND then
            set tableUsedCounts[i] = tableUsedCounts[i] + 1
            return i
        elseif numHandles >= MAX_HANDLES then
            call DisplayTextToPlayer(GetLocalPlayer(), 0,0, "Map Error: Couldn't create a handle index. Too many handles indexed. (HandleIndexLib:CreateHandleIndex)")
            return HANDLE_INDEX_NOT_FOUND
        endif
        
        //Hash
        set i = H2I(h) //hope H2I(h) is positive
        set i = i - (i / TABLE_SIZE) * TABLE_SIZE

        //Probe
        set n = 0
        loop
            //check space
            exitwhen tableHandles[i] == null
            //next space
            set i = i + 1
            set n = n + 1
            if i >= TABLE_SIZE then
                set i = 0
            endif
        endloop
        
        //place the handle in the open space
        set tableHandles[i] = h
        set tableUsedCounts[i] = 1
        set tableValues[i] = 0
        set tableProbeCounts[i] = n
        set numProbes = numProbes + n
        set numHandles = numHandles + 1
        debug if SHOW_DEBUG_MESSAGES then
        debug     call BJDebugMsg("Handle Index Created: H(" + I2S(H2I(h)) + ") = " + I2S(i))
        debug     call BJDebugMsg("Handle Hash Table Status: " + I2S(numHandles) + " entries, " + R2S(numProbes*1./numHandles) + " probes per entry")
        debug endif
        return i
    endfunction

    ///Deallocates the array index for the handle h
    function DestroyHandleIndex takes handle h returns nothing
        local integer i = GetHandleIndex(h)
        if i == HANDLE_INDEX_NOT_FOUND then
            call DisplayTextToPlayer(GetLocalPlayer(), 0, 0, "Map Error: Destroyed non-existant handle index. (HandleIndexLib:DestroyHandleIndex)")
            return
        endif
        set tableUsedCounts[i] = tableUsedCounts[i] - 1
        
        //Don't actually destroy the handle if it has been created more times than it has been 'destroyed'
        if tableUsedCounts[i] > 0 then
            return
        endif
        
        //Destroy the handle
        set numHandles = numHandles - 1
        set numProbes = numProbes - tableProbeCounts[i]
        set tableUsedCounts[i] = EMPTY_BUT_USED
        set tableHandles[i] = null
        set tableValues[i] = 0
        set tableProbeCounts[i] = 0
        debug if SHOW_DEBUG_MESSAGES then
        debug     call BJDebugMsg("Handle Index Destroyed: H(" + I2S(H2I(h)) + ")")
        debug     call BJDebugMsg("Handle Hash Table Status: " + I2S(numHandles) + " entries")
        debug endif
        
        //Reset used flags when possible so probing doesn't go out of control
        set i = i + 1
        if i >= TABLE_SIZE then
            set i = 0
        endif
        if tableUsedCounts[i] != 0 then
            return //can't remove used flag now, it could break probing
        endif
        loop
            exitwhen tableHandles[i] != null
            set tableUsedCounts[i] = 0
            set i = i - 1
            if i < 0 then
                set i = TABLE_SIZE-1
            endif
            exitwhen tableUsedCounts[i] == 0
        endloop
    endfunction
        
    private function initHandleIndexLib takes nothing returns nothing
        debug if SHOW_DEBUG_MESSAGES then
        debug     call BJDebugMsg("Initialized HandleIndexLib with SHOW_DEBUG_MESSAGES")
        debug endif
    endfunction
endlibrary

I at least know it works, and that the hash table almost never collides. I've been integrating it into power towers, where I already have unit custom data monopolized for other purposes.
03-19-2008, 04:58 AM#2
Jazradel
Well:
Quote:
Originally Posted by SFilip;573367
A somewhat short version...
Get grimoire (or newgen), make sure japi and nativepack are loaded in your ongameload.lua, create a new map, import the custom common.j (found in the japi directory of newgen/grimoire) and use something like this:
Collapse JASS:
function dotest takes nothing returns nothing
    local integer n = 0
    local integer swatch = StopWatchCreate()
    loop
        exitwhen n == 1000
        // do whatever you want to test here
        set n = n + 1
    endloop
    call BJDebugMsg(R2S(StopWatchMark(swatch)*1000.))
    call StopWatchDestroy(swatch)
endfunction
Now use startwar3 or war3win in grim/newgen directory to start the game, load your map and you should get the output in µs (1/10^6 seconds).
Of course it's recommended that you run this a couple of times (I usually get the average out of 100 runs) for more precise results. Also note that this can't get you the exact time something takes to run, should be used mainly for comparing.
Quote:
Originally Posted by PipeDream
Measurements on my machine, an Athlon 64 4400+, 1stddev error bars are all at +-5us.
Calling empty native: 29us
Calling sin: 30us
Calling sqrt: 30us
Multiplying two numbers: 40us
Setting a global real: 20us
Calling empty JASS function: 18us

Or you can just run it for ~5000 trials and count (on a clock) how long warcraft III is frozen for.
03-19-2008, 11:13 AM#3
Strilanc
I actually did do the clock test before I posted, with 1 000 runs over 100 footmen and timed all of them to about ~3 seconds. But I figured there was no way that number would mean anything.

I didn't know grimoire included that kind of functionality. I already have newgen so I'll just need to follow the instructions carefully, as I've never really used grimoire for anything that wasn't done by default.
03-19-2008, 01:36 PM#4
Toadcop
Grimoire -> Japi (include it to ongameload) -> StopWatch natives...

import custom common.j into your map. (with new natives)
Collapse JASS:
//Measure wall clock elapsed time
native StopWatchCreate    takes nothing returns integer
native StopWatchMark    takes integer obj returns real
native StopWatchDestroy    takes integer obj returns nothing

use like this...

Collapse JASS:
 local integer i=0
 set i=StopWatchCreate()
... // do your stuff
 call echo(R2S(StopWatchMark(i)*1000)) // Display massage or something // *1000 - will increse the value so it will be "readable".
 call StopWatchDestroy(i)

you can also mark it 2 times and get the difference between 2 marks bu it makes not much difference.

yes i know it's bit to uninformative but i don't have a wish now to create a demo map ^^ // btw read some docs in grimoire folder.
03-20-2008, 05:10 PM#5
Strilanc
I tried doing this, and ran into some problems.

Whenever I try to test the benchmarking map, or even open it in warcraft 3 (NewGen Warcraft.exe), I get a fatal error. It says there's an access violation at "0x00000007", which is odd since it says that's the position of the instruction trying to access the memory.

Uncommenting the lines the readme tells me to uncomment (the readme tells me the wrong .lua files to edit, by the way) doesn't make a difference. The example japi map doesn't cause an error, it just doesn't work at all.
03-20-2008, 11:45 PM#6
Toadcop
hmmmm...
well "you do something wrong" sorry i can't tell you exactly what. =(