| 03-07-2011, 08:46 AM | #1 |
This version of Table is based on the philosophy that you can use one hashtable for your whole map. What it does is divide one hashtable into many different components, and each system in the map can have its own share of the hashtable. Taking advantage of parent keys and child keys to their fullest extent, the ability to have 1 hashtable for the whole map can now be realized. I came up with the idea for this project after using Vexorian's Table and hitting the limits a number of times. All of those limitations have been fulfilled by this project:
You can have 256 hashtables at a time, so with 2 ^ 31 - 1 Table instances at your disposal and (that's right, I said "and", not "or") roughly 2 ^ 18 TableArray instances with array size 8192 (JASS max array size) - not to mention the virtually-unlimited possibilities with the new HashTable struct - you will find yourself with more storage options than you know what to do with. JASS:library Table /* made by Bribe, special thanks to Vexorian & Nestharus, wc3c.net version. One map, one hashtable. Welcome to the new Table. This newest iteration of Table introduces the new HashTable struct. You can now instantiate HashTables which enables the use of large parent and large child keys, just like a standard hashtable. Previously, the user would have to instantiate a Table to do this on their own which - while doable - is something the user should not have to do if I can add it to this resource myself (especially if they are inexperienced). This version, featured exclusively on wc3c.net, allows full access of all new features while preserving the API of the original Table library. API ------------ struct Table | static method create takes nothing returns Table | create a new Table | | method destroy takes nothing returns nothing | destroy it | | method reset takes nothing returns nothing | reset all stored values inside of it | | method flush takes integer key returns nothing | remove the value at index "key" | | method operator []= takes integer key, $TYPE$ value returns nothing | assign "value" to index "key" | | method operator [] takes integer key returns $TYPE$ | load the value at index "key" | | method exists takes integer key returns boolean | whether or not the key was assigned | ---------------- struct TableArray | static method operator [] takes integer array_size returns TableArray | create a new array of Tables of size "array_size" | | method destroy takes nothing returns nothing | destroy it | | method flush takes nothing returns nothing | flush and destroy it | | method operator size takes nothing returns integer | returns the size of the TableArray | | method operator [] takes integer key returns Table | returns a Table accessible exclusively to index "key" */ globals private integer less = 0 //Index generation for TableArrays (below 0). private integer more = 8190 //Index generation for Tables. //Configure it if you use more than 8190 "key" variables and structs in your map (I set it way higher than it needs to be). private hashtable ht = InitHashtable() private key sizeK private key listK endglobals private struct dex extends array static method operator size takes nothing returns Table return sizeK endmethod static method operator list takes nothing returns Table return listK endmethod endstruct private struct handles extends array method exists takes integer key returns boolean return HaveSavedHandle(ht, this, key) endmethod method flush takes integer key returns nothing call RemoveSavedHandle(ht, this, key) endmethod endstruct private struct agents extends array method operator []= takes integer key, agent value returns nothing call SaveAgentHandle(ht, this, key, value) endmethod endstruct //! textmacro NEW_ARRAY_BASIC takes SUPER, FUNC, TYPE private struct $TYPE$s extends array method operator [] takes integer key returns $TYPE$ return Load$FUNC$(ht, this, key) endmethod method operator []= takes integer key, $TYPE$ value returns nothing call Save$FUNC$(ht, this, key, value) endmethod method exists takes integer key returns boolean return HaveSaved$SUPER$(ht, this, key) endmethod method flush takes integer key returns nothing call RemoveSaved$SUPER$(ht, this, key) endmethod endstruct private module $TYPE$m method operator $TYPE$ takes nothing returns $TYPE$s return this endmethod endmodule //! endtextmacro //! textmacro NEW_ARRAY takes FUNC, TYPE private struct $TYPE$s extends array method operator [] takes integer key returns $TYPE$ return Load$FUNC$Handle(ht, this, key) endmethod method operator []= takes integer key, $TYPE$ value returns nothing call Save$FUNC$Handle(ht, this, key, value) endmethod method exists takes integer key returns boolean return HaveSavedHandle(ht, this, key) endmethod method flush takes integer key returns nothing call RemoveSavedHandle(ht, this, key) endmethod endstruct private module $TYPE$m method operator $TYPE$ takes nothing returns $TYPE$s return this endmethod endmodule //! endtextmacro //Run these textmacros to include the entire hashtable API as wrappers. //Don't be intimidated by the number of macros - Vexorian's map optimizer is //supposed to kill functions which inline (all of these functions inline). //! runtextmacro NEW_ARRAY_BASIC("Real", "Real", "real") //! runtextmacro NEW_ARRAY_BASIC("Boolean", "Boolean", "boolean") //! runtextmacro NEW_ARRAY_BASIC("String", "Str", "string") //New textmacro to allow table.integer[] syntax for compatibility with textmacros that might desire it. //! runtextmacro NEW_ARRAY_BASIC("Integer", "Integer", "integer") //! runtextmacro NEW_ARRAY("Player", "player") //! runtextmacro NEW_ARRAY("Widget", "widget") //! runtextmacro NEW_ARRAY("Destructable", "destructable") //! runtextmacro NEW_ARRAY("Item", "item") //! runtextmacro NEW_ARRAY("Unit", "unit") //! runtextmacro NEW_ARRAY("Ability", "ability") //! runtextmacro NEW_ARRAY("Timer", "timer") //! runtextmacro NEW_ARRAY("Trigger", "trigger") //! runtextmacro NEW_ARRAY("TriggerCondition", "triggercondition") //! runtextmacro NEW_ARRAY("TriggerAction", "triggeraction") //! runtextmacro NEW_ARRAY("TriggerEvent", "event") //! runtextmacro NEW_ARRAY("Force", "force") //! runtextmacro NEW_ARRAY("Group", "group") //! runtextmacro NEW_ARRAY("Location", "location") //! runtextmacro NEW_ARRAY("Rect", "rect") //! runtextmacro NEW_ARRAY("BooleanExpr", "boolexpr") //! runtextmacro NEW_ARRAY("Sound", "sound") //! runtextmacro NEW_ARRAY("Effect", "effect") //! runtextmacro NEW_ARRAY("UnitPool", "unitpool") //! runtextmacro NEW_ARRAY("ItemPool", "itempool") //! runtextmacro NEW_ARRAY("Quest", "quest") //! runtextmacro NEW_ARRAY("QuestItem", "questitem") //! runtextmacro NEW_ARRAY("DefeatCondition", "defeatcondition") //! runtextmacro NEW_ARRAY("TimerDialog", "timerdialog") //! runtextmacro NEW_ARRAY("Leaderboard", "leaderboard") //! runtextmacro NEW_ARRAY("Multiboard", "multiboard") //! runtextmacro NEW_ARRAY("MultiboardItem", "multiboarditem") //! runtextmacro NEW_ARRAY("Trackable", "trackable") //! runtextmacro NEW_ARRAY("Dialog", "dialog") //! runtextmacro NEW_ARRAY("Button", "button") //! runtextmacro NEW_ARRAY("TextTag", "texttag") //! runtextmacro NEW_ARRAY("Lightning", "lightning") //! runtextmacro NEW_ARRAY("Image", "image") //! runtextmacro NEW_ARRAY("Ubersplat", "ubersplat") //! runtextmacro NEW_ARRAY("Region", "region") //! runtextmacro NEW_ARRAY("FogState", "fogstate") //! runtextmacro NEW_ARRAY("FogModifier", "fogmodifier") //! runtextmacro NEW_ARRAY("Hashtable", "hashtable") struct Table extends array // Implement modules for intuitive syntax (tb.handle; tb.unit; etc.) implement realm implement integerm implement booleanm implement stringm implement playerm implement widgetm implement destructablem implement itemm implement unitm implement abilitym implement timerm implement triggerm implement triggerconditionm implement triggeractionm implement eventm implement forcem implement groupm implement locationm implement rectm implement boolexprm implement soundm implement effectm implement unitpoolm implement itempoolm implement questm implement questitemm implement defeatconditionm implement timerdialogm implement leaderboardm implement multiboardm implement multiboarditemm implement trackablem implement dialogm implement buttonm implement texttagm implement lightningm implement imagem implement ubersplatm implement regionm implement fogstatem implement fogmodifierm implement hashtablem method operator handle takes nothing returns handles return this endmethod method operator agent takes nothing returns agents return this endmethod //set this = tb[GetSpellAbilityId()] method operator [] takes integer key returns integer return LoadInteger(ht, this, key) //return this.integer[key] endmethod //set tb[389034] = 8192 method operator []= takes integer key, integer value returns nothing call SaveInteger(ht, this, key, value) //set this.integer[key] = value endmethod //set b = tb.exists(2493223) method exists takes integer key returns boolean return HaveSavedInteger(ht, this, key) //return this.integer.exists(key) endmethod //call tb.flush(294080) method flush takes integer key returns nothing call RemoveSavedInteger(ht, this, key) //call this.integer.flush(key) endmethod //Remove all data from a Table instance method reset takes nothing returns nothing call FlushChildHashtable(ht, this) endmethod //local Table tb = Table.create() static method create takes nothing returns Table local Table this = dex.list[0] if this == 0 then set this = more + 1 set more = this else set dex.list[0] = dex.list[this] call dex.list.flush(this) //Clear hashed memory endif set dex.list[this] = -1 //for double-free protection return this endmethod // Removes all data from a Table instance and recycles its index. // // call tb.destroy() // method destroy takes nothing returns nothing if dex.list[this] != -1 then debug call BJDebugMsg("Table Error: Tried to double-free instance: " + I2S(this)) return endif call this.reset() set dex.list[this] = dex.list[0] set dex.list[0] = this endmethod //2D syntax from original Table - this uses 2 hashtable references now, but the same syntax. The old method was obsolete anyway. static method operator [] takes string id returns Table local integer index = StringHash(id) local Table t = Table(thistype.typeid)[index] if t == 0 then set t = Table.create() set Table(thistype.typeid)[index] = t endif return t endmethod static method flush2D takes string id returns nothing local integer index = StringHash(id) local Table t = Table(thistype.typeid)[index] if t != 0 then call t.destroy() call Table(thistype.typeid).flush(index) endif endmethod endstruct struct TableArray extends array //Returns a new TableArray to do your bidding. Simply use: // // local TableArray ta = TableArray[array_size] // static method operator [] takes integer array_size returns TableArray local Table tb = dex.size[array_size] //Get the unique recycle list for this array size local TableArray this = tb[0] //The last-destroyed TableArray that had this array size if array_size <= 0 then debug call BJDebugMsg("TypeError: Invalid specified TableArray size: " + I2S(array_size)) return 0 endif if this == 0 then set this = less - array_size set less = this else set tb[0] = tb[this] //Set the last destroyed to the last-last destroyed call tb.flush(this) //Clear hashed memory endif set dex.size[this] = array_size //This remembers the array size return this endmethod //Returns the size of the TableArray method operator size takes nothing returns integer return dex.size[this] endmethod //This magic method enables two-dimensional[array][syntax] for Tables, //similar to the two-dimensional utility provided by hashtables them- //selves. // //ta[integer a].unit[integer b] = unit u //ta[integer a][integer c] = integer d // method operator [] takes integer key returns Table local integer i = this.size if i == 0 then debug call BJDebugMsg("IndexError: Tried to get key from invalid TableArray instance: " + I2S(this)) return 0 elseif key < 0 or key >= i then debug call BJDebugMsg("IndexError: Tried to get key [" + I2S(key) + "] from outside TableArray bounds: " + I2S(i)) return 0 endif return this + key endmethod //Destroys a TableArray without flushing it; I assume you call .flush() //if you want it flushed too. This is a public method so that you don't //have to loop through all TableArray indices to flush them if you don't //need to (ie. if you were flushing all child-keys as you used them). // method destroy takes nothing returns nothing local Table tb = dex.size[this.size] if this.size == 0 then debug call BJDebugMsg("TypeError: Tried to destroy an invalid TableArray: " + I2S(this)) return endif if tb == 0 then //Create a Table to index recycled instances with their array size set tb = Table.create() set dex.size[this.size] = tb endif call dex.size.flush(this) //Clear the array size from hash memory set tb[this] = tb[0] set tb[0] = this endmethod private static Table tempTable private static integer tempEnd //Avoids hitting the op limit private static method clean takes nothing returns nothing local Table tb = .tempTable local integer end = tb + 0x1000 if end < .tempEnd then set .tempTable = end call ForForce(bj_FORCE_PLAYER[0], function thistype.clean) else set end = .tempEnd endif loop call tb.reset() set tb = tb + 1 exitwhen tb == end endloop endmethod //Flushes the TableArray and also destroys it. Doesn't get any more //similar to the FlushParentHashtable native than this. // method reset takes nothing returns nothing if this.size == 0 then debug call BJDebugMsg("TypeError: Tried to reset an invalid TableArray instance: " + I2S(this)) return endif set .tempTable = this set .tempEnd = this + this.size call ForForce(bj_FORCE_PLAYER[0], function thistype.clean) call this.destroy() endmethod endstruct //Need bigger or more random parent keys than TableArray provides? Here you go! struct HashTable extends array //Enables myHash[parentKey][childKey] syntax. //Basically, it creates a Table in the place of the parent key if //it didn't already get created earlier. method operator [] takes integer index returns Table local Table t = Table(this)[index] if t == 0 then set t = Table.create() set Table(this)[index] = t endif return t endmethod //You need to call this on each parent key that you used if you //intend to destroy the HashTable or simply no longer need that key. method flush takes integer index returns nothing local Table t = Table(this)[index] if t != 0 then call t.destroy() call Table(this).flush(index) endif endmethod method exists takes integer index returns boolean return Table(this).exists(index) endmethod method destroy takes nothing returns nothing call Table(this).destroy() endmethod static method create takes nothing returns thistype return Table.create() endmethod endstruct struct HandleTable extends array static method operator [] takes string index returns thistype return Table[index] endmethod static method flush2D takes string index returns nothing call Table.flush2D(index) endmethod method operator [] takes handle key returns integer return Table(this)[GetHandleId(key)] endmethod method operator []= takes handle key, integer value returns nothing set Table(this)[GetHandleId(key)] = value endmethod method flush takes handle key returns nothing call Table(this).flush(GetHandleId(key)) endmethod method exists takes handle key returns boolean return Table(this).exists(GetHandleId(key)) endmethod method reset takes nothing returns nothing call Table(this).reset() endmethod method destroy takes nothing returns nothing call Table(this).destroy() endmethod static method create takes nothing returns thistype return Table.create() endmethod endstruct struct StringTable extends array static method operator [] takes string index returns thistype return Table[index] endmethod static method flush2D takes string index returns nothing call Table.flush2D(index) endmethod method operator [] takes string key returns integer return Table(this)[StringHash(key)] endmethod method operator []= takes string key, integer value returns nothing set Table(this)[StringHash(key)] = value endmethod method flush takes string key returns nothing call Table(this).flush(StringHash(key)) endmethod method exists takes string key returns boolean return Table(this).exists(StringHash(key)) endmethod method reset takes nothing returns nothing call Table(this).flush() endmethod method destroy takes nothing returns nothing call Table(this).destroy() endmethod static method create takes nothing returns thistype return Table.create() endmethod endstruct endlibrary |
| 04-20-2011, 08:34 PM | #2 |
If you feel Vexorian's Table is an inadequate interface for using hashtables, you are welcome to write your own alternative library, but it should have a new name. I understand that you included all the old Table's functionality to justify keeping the name, but that is not enough. Even if your library is backwards compatible, it is not interchangeable - any library that uses the new functionality will not be able to work with the old Table, or any other "new Table" that may come up if we encourage such behaviour. Therefore, as long as it has the same name, this library can not be approved. |
| 04-25-2011, 10:23 PM | #3 |
Bribe's getting married, so I think he'll be out for a couple of weeks to a month. So no, he's not ignoring you ; p |
| 05-04-2011, 12:22 AM | #4 |
Ok, so Bribe updated this to Array w/o anyone noticing apparently ;o. wc3c gets a special edition, lol So I guess this can be approved now by your reasoning, and then I can update the Base thread and we can move on, eventually getting Encoder on to wc3c. Hurray Goal: http://www.hiveworkshop.com/forums/s...-2-2-a-189883/ |
| 05-23-2011, 11:55 AM | #5 |
|
| 05-23-2011, 03:05 PM | #6 |
Thank you for the review. I have made most of those changes and upgraded this to version 2.0.0.0 - includes fewer variables, removes some limitations and consolidates the methods as suggested by Nestharus (using one method of handle.remove and handle.contains instead of using a unique remove and contains method for each handle type). I don't like keeping safety outside of debug-mode - the reason being is that a skilled user who manages his script carefully should be rewarded with an efficient script, while someone who isn't sure how it works should run it in debug mode until he/she gets it right. |
| 05-24-2011, 01:45 PM | #7 | |
Quote:
|
| 05-24-2011, 02:27 PM | #8 |
Updated with the flavor "SAFETY" for non-speed freaks, enabled by default. |
| 05-28-2011, 10:53 PM | #9 |
The more I think about this resource, the less use I see for it. Table was originally created to be a wrapper for gamecache which had many issues: using it natively was ugly as you had to convert keys to strings, initializing it required some arcane tricks, there was the 256 limit and so on. Then Blizzard gave us hashtables and Table lost most of its usefulness. Sure, it still had nicer syntax but the difference was no longer as significant as compared to gamecache and many people didn't find it worth the extra library requirement. If it hadn't turned out that hashtables also have a 256 limit, very few people would still be using Table. You boast two main advantages over Table: being able to store any type and improved 2D syntax. Very few libraries need one of these two features. Those few that do could easily use native hashtables without having to worry about the 256 limit and like I mentioned above, the 256 limit is just about the only reason most people still use Table in lieu of native hashtables. In fact, in the case of 2D functionality I find using a native hashtable superior to Array. With Array, you have two options: either the code will be slower than using a native hashtable, or there will be no safety and bugs in other resources can cause yours to malfunction. The situation is made worse by the fact that you have no control over which of these two cases occurs as that depends on how the people who use your library set their Array SAFETY boolean. I find the slightly uglier syntax of native hashtables a small price to pay for avoiding this can of worms. The ability to store all types doesn't have these issues, it is in fact equivalent to using native hashtables, only with nicer syntax. However, as already stated, I do not consider the nice array-like syntax alone to be a sufficient reason for approving this resource. |
| 05-29-2011, 07:53 AM | #10 | ||
Quote:
A valid point. And if you notice, one of the first things I mentioned was a philosophy of staying as far away from the limit as possible. Quote:
I disagree. AutoIndex, for example, has an option "AutoDataFastMode" because grim001 hashes his 3-D arrays like "index*8190+typeid". With a DualArray, he could use "DualArrayInstance + index" which would speed up the operation. 3-D syntax is quite useful for projectile systems to index the target to the missile and vice-versa. I have come up with many reasons to use so many dimensions in my career and the 3-D option opens up a lot of doors. For example, I built a Retro library that indexes unit properties like health, coordinates, facing, for all units on the map over a period of time, which is a mess of dimensions to say the least. Being able to store more than just integers has also been requested by many people in the past, and Anachron wrote a custom version of Table just to unlock such syntax for his full-screen inventory system. |
| 05-29-2011, 01:03 PM | #11 | |||
Quote:
Quote:
Quote:
|
| 05-29-2011, 02:08 PM | #12 |
So you make a rule that says 100 libraries have to use this to be valid. A rather unfair rule - should I brute-force into existence 100 different libraries to prove a point? If that's the kind of basis you are going on, I won't waste my time trying. Add to this the optimizer is supposed to kill inlined functions and constants, so this would have no more bloat in the end than Table would. I gather from your comments you disagree with the philosophy "the further we can get from the limit, the better". However, that philosophy is something I treasure - it couples nicely with the philosophy "doing more with less". Less RAM consumption, fewer handles in the stack, positive things altogether. |
| 05-29-2011, 04:21 PM | #13 | |||
Quote:
Quote:
Quote:
Array also extends the nice syntax to a few more cases than Table does, but that is not nearly a sufficient reason to replace it in our resource section. |
| 05-29-2011, 08:30 PM | #14 |
I really don't see a need to store anything else than a struct index in a hashtable. Everything else should be stored within the struct itself, and you know that array look-ups are ~2 times faster than hashtable look-ups. Yes, some systems might need to store more than that, but in such case, using a normal hashtable would be better in order to simply eliminate the extra requirement (and if you're storing lots of stuff in a hashtable, you're better off storing it inside a struct anyway). Sure, the interface of yours is very pleasant, and if I'm asked by a person with little knowledge in (v)Jass / Zinc (or programming in general), I'd actually recommend yours over Vexorian's, but I still don't think that a replacement is needed anytime. You may argue that the 2D-syntax Vexorian's Table provides is bad because of StringHash(string) being involved, but let me remind you that the user can manually do things like Table(-(GetHandleId(unit))[key] = 0x7FFFFFFF and/or any other combination of parent- and child-keys. Also, please refrain from doing implicit integer-to-struct and struct-to-integer type conversion. Changing stuff such as JASS:method operator $TYPE$ takes nothing returns $TYPE$s return this endmethod JASS:method operator $TYPE$ takes nothing returns $TYPE$s return $TYPE$s(this) endmethod and if this == end then to if integer(this) == end then is just so much better and more readable. Well, even after I said all this rubbish, I don't think that this should be graveyarded. It is coded nicely, and some people might want to use it. And I'm aware that nobody was asking for my opinion here, but I couldn't resist. |
| 05-30-2011, 08:03 AM | #15 | ||
Quote:
The speed is much faster than Table's gamecache-inspired 2-D array syntax, and both have their own safety precautions. You have to prefix all of the 2-D array-type operations in Table with the library name, and note StringHash is case-insensitive so you can have a conflict if two libraries share the same name but with different capitalization. You overestimate the safety issue with DualArray as the person building the map has control over the SAFETY option, not the people who wrote the libraries that require it. If there are libraries present in the map that are so poorly-coded they require SAFETY to be on perhaps a simple recommendation to users that SAFETY should be on. Realistically, if the system needs such safety built-in, it probably has bigger problems than just that. It has the exact same level of protection (without SAFETY on) as one of vJass' 2-D arrays, and no one was complaining that those were unsafe. As well, you can't destroy 2-D arrays in the original Table. DualArrays are 100% dynamic meaning you can pop an entire DualArray sized 0x2000 per struct instance and for as many libraries as you want, destroy when done, even freeing the slot should it need to be recycled, while Table is limited to a mere parent key. Arguably, you could do "SCOPE_PREFIX" + I2S(structInstance) but the speed cost and string leakage here is devestating. Quote:
That kind of typecasting would generate a data sharing conflict easily resulting in a lot of things being accidentally overwritten or erased. |
