WeWarlock / JassPreprocessor - Beta Release

10-03-2006, 06:24 PM

Latest version: WeWarlock Beta 0.7.0 (The source of WeWarlock can be checked out with Subversion from https://svn.sourceforge.net/svnroot/jass/WeWarlock)

What is it?
WeWarlock is first and foremost a JassPreprocessor. It allows us to extend the current syntax and possibilities on Jass, creating systems and syntax that would otherwise be impossible, such as //! include, static keyword, SmartAttach/Stack/Arrays and Callables.

Second it is also a very practical tool. It allows things such as Rawcode Renaming (as found in the Widgetizer), automatically adding a directory of files, checking the syntax with PJASS and more.

Right now it is in beta stage which means that the program has been tested for several weeks in a complex actual project. In other words, I can both claim it is somewhat stable while at the same time being sure that there are still bugs to be found.

The largest amount of work left for this tool is actually documentation. Writing documentation for a project that is still being heavily developed is difficult, so this post will only give an overview and basic description of all possible features.

Running the Program
When you run the program you will first want to select the input file. This can be a .j file if you only want to process a script, but you are probably mostly interested in processing an entire .w3x map archive.

When you open an input .w3x map you will see that the fields "Output File", "Source Dir" and "Data Dir" is automatically filled out. Output File will be set to the input file + "_compiled.w3x", the Source Dir will be the directory of the input file + "/source" and Data Dir "/files"

Now don't worry about what /source and /files directories are used for, they will be explained while we go through each of the options.

When you are satisfied with the options press the red compile button and watch it do it's thing. The compilation options is stored in the ORIGINAL input .w3x so that you only have to set your favorite options once or twice. (If you edit the input file in the World Editor, you will notice it has an imported file named WeWarlock.yaml - this is the options file. You can remove it if you like.) If the "Debug Build" is checked, a (*) will be added to the name of the map so you can tell the compiled/original apart within War3.

Use Data Dir
This causes the entire directory structure in /files to be imported, with path, into the compiled map archive. This is often more convenient than using WE's import manager.
If you have:
files\Units\someunit.mdl
files\Units\SuperNinja\SuperNinja.mdl
files\Units\SuperNinja\SuperNinja.blp
files\ReplaceableTextures\CommandButtons\SomeButton.blp

It will be imported into the output map as:
Units\someunit.mdl
Units\SuperNinja\SuperNinja.mdl
Units\SuperNinja\SuperNinja.blp
ReplaceableTextures\CommandButtons\SomeButton.blp

Parse Includes
Very useful options that recursively parses all //! include commands. You may for example put the following in the custom script section of your map:
//! include MySpell.j It searches the /source folder and all the content of 'MySpell.j' will be inserted at that point. If MySpell.j has a globals/endglobals section, the globals will be merged, so each file can easily declare its own globals.

A more advanced example would be, in your custom script to //! include main.j and in "main.j" you can have the following:

JASS:

//! include SystemUtility.j
//! include ProjectileSystem.j
//! include UltraLich_Spells.j
//! include MegaKobold_Spells.j

Then 'ProjectileSystem.j' can use functions from SystemUtility.j and both the spell files could use anything above them. This is very useful for structuring large projects.

Check Output Syntax
This causes PJASS to validate the syntax of the final war3map.j script. DO NOT TURN THIS OFF EVER!

Sometimes you will get a syntax error. Scroll down compilation text output to see which lines there were errors. Press "View output script" to view the last script that was validated. (If you prefer to use your own editor, just open pjass/validate.j yourself).

Rawcode Renamer
This causes all objects in the object editor with names ending in "::rwcd" to have their rawcode changed to 'rwcd' and removing "::rwcd" from their name. 'rwcd' can of course be any rawcode.

Replace Parser
Parses //! ReplaceLocal commands.
In the following segment, MAX is replaced with 100.

JASS:

//! ReplaceLocal MAX 100
function DoSomething takes integer i returns boolean
    if i < MAX then
        return true
    endif
    return false
endfunction

//! ReplaceLocal affects ALL instances of the given text in the file it is found. So a ReplaceLocal command in "SomeFile.j" will only affect code in that file. It is generally smart to use unique names for the replace, so that you avoid replacing text that shouldn't be replaced.

Include Jlib
This will be grayed out most of the time since several other options depends on it. If it is not grayed out, you can enable it to cause the functions in "library/Jlib.j" to be available in your map.

Include VectorLib
Checking it makes the functions in "library/VectorLib.j" to be available in your map. In the future there will be a better support for external libraries than this.

SmartAttach
One of the major features of WeWarlock is the SmartAttach system. Just check the box (and optionally set the max number of ids you can have for each attach type, but the defaults should be good.)

Writing a comprehensive manual for this system will take me some time, and there are some tiny things I still have to fix, but at least most of the current functionality can be found through THIS POST. Read it. THOROUGHLY!

JassPreprocessor

Oboy. This one has a LOT of features. I will write down some of them.

Static
The 'static' keywords allow you to easily define a local that is initialized to a value on map init. Perfect for some situations where you would be building and destroying the same value all the time. In the following example it is used to store a boolexpression.

JASS:

function ExcitingSpell takes unit source, unit victim, real damage returns nothing
    static boolexpr target_options = And(TargetEnemy, TargetIsNot)
    
    call CS_EnumUnitsInAOELoc(possible_targets, victim_pos, area, target_options)
    set target_options = null

Notice that you dont destroy it at the end of the function, but you do have to null it.

Event Commands
Registering functions to run when certain events happen is very useful. The Event Commands makes this easier and more effective than ever before.

Commands:
//! initfunc (no args) - Add a call to the function in the maps init function.
This one is VERY useful and simple.

JASS:

function SomeFunction takes nothing returns nothing
    //! initfunc
    //This function now runs on map init
endfunction

//! BeginAbility rawcode - When a unit starts casting ability, but before the actual effect happens.
//! BeginEffect rawcode - When the effect happens.
//! StopAbility rawcode - If the unit stops casting, this will be triggered. Also triggers after FinishAbility.
//! FinishAbility rawcode - Triggers when the unit has finished casting completely AND the casting animation is finished.

The above commands are used as follows:

JASS:

function ABIL_Fireball takes nothing returns nothing
    //! BeginEffect A003
    //This function is now run when the ability 'A003' begins it's effect.
endfunction

//! ObjectOrderEvent order_integer_id
//! PointOrderEvent order_integer_id
//! InstantOrderEvent order_integer_id

These trigger when ObjectOrder, PointOrder or InstantOrders are issued.

JASS:

function ABIL_DrainSpellChange takes nothing returns nothing
    //The following orders are when enabling or disabling 'bloodlust'
    //! InstantOrderEvent 852102
    //! InstantOrderEvent 852103
    //This function now runs when bloodlust (or an ability based on it) is right clicked.
endfunction

You may always add a "_cond" function to automatically evaluate a condition:

JASS:

function ABIL_DrainSpellChange_cond takes nothing returns nothing
    //Will cause the above function to never run.
    return false
endfunction

Callables
Yet another very powerful feature. Allows calling of functions by variable, "downward" calling, optional Fastcall/Threadedcall as well as passing arguments to and receiving returnvalues from the called functions.

Syntax:

JASS:

local integer a_function = Func("SomeFunction")
//Call the function below
//It is required to specify what the function returns, in this case nothing
call Call(nothing a_function)

function SomeFunction takes nothing returns nothing
    //do something
endfunction

//Define a much more interesting callable
callable AwesomeFunction takes unit u, real speed returns location
    return Location(GetUnitX(u) + speed, GetUnitY(u) + speed)
endfunction

function MoveUnit takes integer move_func, unit whichUnit, real speed returns nothing
    local location loc
    //Notice how it is vital to define the types the function return (argument 1)
    //as well as the type of each argument.
    set loc = Call(location move_func, unit whichUnit, real speed)
    call SetUnitPositionLoc(whichUnit, loc)
    call RemoveLocation(loc)
    set loc = null
endfunction

//To call the above function with AwesomeFunction:
call MoveUnit(Func("AwesomeFunction"), someUnit, 100.0)

Additional PreProcessor functions that I have yet to document are:
ExtData (I want to rework it)

Want to see WeWarlock in action? Try out either the simple Callable test or the much more complex Lightning Storm spell that heavily uses SmartAttach and SmartArrays and more. Remember to try to compile these for yourself, and to read the source in the source folder.

Thats it! Try out SmartAttach today! Come with suggestions on useful //! commands or other interesting features.

Changes:

Quote:

Version 0.5:
New features:
SmartAttach now has Write/Read as GameCache equivalents of Attach/Get. These are 60% faster than normal GameCache usage, completely dynamic, and do not take a domain argument.

JASS:

//Xxx can be an SA_Unit/Item/Timer/Id and Yyy can be any type i.e Unit, Lightning etc.
call XxxWriteYyy(whichXxx, key, value)
call XxxReadYyy(whichXxx, key)

Also, to make SmartAttach a complete GameCache suite, the following functions are available for each type 'Yyy'. These are normal GC speed, but allows you to attach to any handle.

JASS:

call HandleWriteYyy(handle, key, value)
call HandleReadYyy(handle, key)

Fixed a bug with the quoted string parser that would break some strings.
Fixed a bug introduced in previous commit that caused i.e BeginEffect and similar commands to fail (among other things).

Quote:

Version 0.4b:

WeWarlock has commandline support (pass -h for a list of options).

Improved: The progress bar now correctly progress within SmartAttach/Array/Stack compiling, and the entire program display refreshes a bit more reliable during compilation.

Bugfixes: Now correctly skips all archive stuff when compiling .j files.

Changes: SmartAttach Unit/Item/Timer/IdRelease now takes no arguments, and a Unit/Item/Timer/IdReleaseEx has been added that takes 3 boolean arguments (clean local, global and gc)

This means those that were using the old SA_Unit/Item/Timer/IdRelease functions need to either remove the boolean arguments from those calls, or use the Ex version of the functions (which requires 3 boolean args)

Quote:

Version 0.6:
New features:
WeWarlock no longer has a GUI. To compile a map do "WarlockCompiler.exe <yourmapname>". It will use the source/ and files/ folder where your map is located and outputs "yourmapname_compiled".

Consequently the download is much smaller and the program takes 1 second less to run for those that are in a hurry.

//! include support has been dropped.

Added support for //! require and //! section. You can //! require script scriptname (without the .j), //! require library libraryname (looks in WeWarlock/Library instead of the source/ folder and //! require preprocessor SmartAttach. The complete list of preprocessors that can be required are: "SmartAttach", "CommonLibrary", "CommandParser", "Callable", "RawcodeRenamer", "Callback", "ExtData" and "Static". The //! ReplaceLocal command is always processed.

I suggest putting a //! require script main in your map and then put further requirements into source/main.j - Any script-file that does //! require script some_other_script can call all functions from that other script.

//! section is vexorians library command that lets you define multiple scripts in one file. You might for example do

JASS:

//! section MyFuncs
function MyFunc takes nothing returns nothing
endfunction
//! endsection

And further scripts could then //! require MyFunc
Also a small improvement to static to make sure it initializes the values of all statics at the end of the main function, allowing other utility functions such as my (undocumented) //! filterfunc to finish their own initialization. This also means that code that runs on map init using //! initfunc will always get a null-value in their statics. This is ok by me as statics are not of any use in one-time init functions.

Update:

Quote:

Version 0.6.2:
New features:
WeWarlock can now parse comma seperated datasources directly into gamecache. It will parse all files in the /source dir with the .csv extension. An example file could be "UnitStuff.csv":

Code:

"Key",   "Name",      "hp_base", "hp_growth", "mp_base", "mp_growth"
"types", "string",    "real",    "real",      "real",    "real"
"HH00",  "Kremo",     550,       50,          0,         0
"HH01",  "Itoneraxu", 350,       30,          350,       75
"HH02",  "Harmbong",  700,       80,          0,         0
"HH03",  "Nuup",      450,       40,          40,        20

You can then access this by doing

JASS:

//! require preprocessor GcData
call GcDataString('HH00', "name")
call GcDataReal(GetUnitTypeId(someUnit), "hp_growth")

Also fixed TWO major bugs that both, independently, made XxxRead/WriteYyy malfunction for SmartAttach. UnitReadInt, TimerWriteReal etc has now been tested and works as intented.

0.6.2b:
Hopefully fixed the replaceparser to also work on the main-script (and not just included scripts)

Update:

Quote:

Version 0.6.3:
New features:
Added functionality to GcData that allows it to replace ExtData entirely. It now also parses .txt as well as .csv files into one unified source of data that can be accessed both as GcDataReal(rawcode, key) and GcDataRealList(rawcode, key, level).

To create a multi-level entry in the .csv files or .txt files seperate each value with a ;

Example .txt file format:

Code:

A001: //Some ability rawcode
real damage = 10;50;100;50040
string spell_effect = "some//model.mdl"

You can then do

JASS:

//! require preprocessor GcData
set damage = GcDataRealList('A001', "damage", 2) //Returns 100 (using 0-indexed counting)
//If you have multiple spells all with a "damage" key defined, you can do
set damage = GcDataRealList(GetSpellAbilityId(), "damage", GetUnitAbilityLevel(GetSpellAbility()) - 1) //Remember to subtract 1 from GetUnitAbilityLevel() since it uses 1-based indexing.

//You can also do the following to make sure you stay within the correct
//number of levels defined for any given value
set levels = GcDataListLength('A001', "damage")

Removed ExtData

SmartAttach.j:

SA_UnitEx and SA_ItemEx returns the SA-id if the Unit/Item has been registered as a SA_Unit/Item or 0 if it hasn't.

10-03-2006, 08:44 PM

Meanie

I can asure you that this stuff is GREAT, i have been using this for about two weeks and it is super easy to learn and the best thing is that i know that whatever i do will be super fast and superior to everything else

.

Well the "con" is that it requires some time to master the compiling and you can't compile the map when it is being played in War3 or when it is selected in the map browser. But after i got a fancy keyboard with macro function it all went super fast and easy. Yesterday i found some great stuff made by PitzerMike that allowed me to play the map in windowed mode instantly...even better! ...Propaganda...

10-03-2006, 08:52 PM

karukef

Yeah about that: If you try to Compile a map for the first time while that map is open in WE, you will get an error. This is because WeWarlock tries to insert the compilation options in the original map but if WE is open it can't do that. So the first time you compile a map (or when you change options), make sure the map is not opened in WE.

Same goes for War3, you cant compile a map while it is being selected inside War3. But I guess many mappers already know this details.

10-04-2006, 04:37 AM	#4
Vexorian	I would have used $ $ instead of ' ' for replace local inside strings but oh well. I think I am still downloading the dependencies and I started this afternoon, well the biggest download so far was tortoise SVN. Edit: the callable's sample syntax says return location , is it a typo? Edit: Finally got able to run the source, time to mess around...

10-04-2006, 12:42 PM

karukef

Quote:

Originally Posted by Vexorian

Edit: the callable's sample syntax says return location , is it a typo?

Yes, a typo. It should be returns location, obviously.

10-04-2006, 10:01 PM	#6
PitzerMike	It's great, I'll write more when I have internet again :).

10-05-2006, 02:01 PM

karukef

Update:

Quote:

10-06-2006, 06:47 PM

karukef

Update:

Quote:

JASS:

//Xxx can be an SA_Unit/Item/Timer/Id and Yyy can be any type i.e Unit, Lightning etc.
call XxxWriteYyy(whichXxx, key, value)
call XxxReadYyy(whichXxx, key)

Also, to make SmartAttach a complete GameCache suite, the following functions are available for each type 'Yyy'. These are normal GC speed, but allows you to attach to any handle.

JASS:

call HandleWriteYyy(handle, key, value)
call HandleReadYyy(handle, key)

Fixed a bug with the quoted string parser that would break some strings.
Fixed a bug introduced in previous commit that caused i.e BeginEffect and similar commands to fail (among other things).

EDIT: Another minor update: Both examples have been updated to be bug free and compilable with newest WeWarlock.

10-19-2006, 08:00 PM

karukef

Update:

Quote:

JASS:

//! section MyFuncs
function MyFunc takes nothing returns nothing
endfunction
//! endsection

10-20-2006, 05:32 PM	#10
Iead	Can you update the demo abilities, they dont' work with your latest compiler.

10-20-2006, 07:39 PM	#11
karukef	What the hell, someone actually uses this? Ok, I'll update them for your pleasure. (I'll edit this post when I'm done)

10-21-2006, 05:45 AM	#12
Vexorian	I think it shoud remove the preprocessors or remove the ! after // once they are evaluated . //! require preprocessor Static does not seem to be working

10-25-2006, 07:59 PM

#13

SFilip

A great idea and a very good tool...yet I seem to be doing something wrong here.
I copied this code from your main post

JASS:

//! ReplaceLocal MAX 100
function DoSomething takes integer i returns boolean
    if i < MAX then
        return true
    endif
    return false
endfunction

intro a new file and ran WarlockCompiler on it.
It seems that it simply ignored the ReplaceLocal line since the error I got was

Code:

Error:validate.j:5: Undeclared variable: MAX

Any ideas?

10-26-2006, 05:56 AM

#14

karukef

I am currently burned out and can not as of now work on WEWarlock. I'm taking a week or so break, after which I will adress any issues. The current release (0.6) is somewhat unstable since I made so many changes without having the proper time and energy to fix it.

I apologize for the inconvenience and hope I can make up for it shortly.

Other than making the current version work better, the only feature I currently plan on implementing is Templates, after which I can hopefully write proper documentation and avoid further drastic changes.

SFilip: As for why //! ReplaceLocal failed, I am guessing the Require Parser forgets to parse Replaces in the original file. If you put the code that requires ReplaceLocal in an external file and //! require script yourfile.j I think the replace will work within that file.

Vexorian: A quick glance on the code does not show me a reason why requiring static should not work, so I'll have to look at it later. And if you think WeWarlock should invalidate the //! commands it parses, I would like to hear the reasons so that I know in which cases you need this. Only for //! require or for all parsed //! commands? Be more specific.

10-27-2006, 04:25 AM	#15
Vexorian	In the strange, case wewarlock is ran on an already compiled map, it generates code like smart attach again since it didn't unvalidate the preprocessor commands, so pjass spawns parse errors, this stopped being an issue for me cause I just avoided to compile an already compiled map.