preJASS - the JASS preprocessor

I'm currently writing a small perlscript (which I've named preJASS (might be subject to change)) for preprocessing JASS-scripts.
Thanks to Vidstige for inspiring me to write this ;-)

The current feature-list is:
* IF/ELSEIF/ELSE - clauses (limited expressions ATM)
* INCLUDE - includes another file and grabs the globals, putting them on top of the preprocessed file.
* DEFINE - defines a global constant
* IFDEF/IFNDEF - checks for the definition of a constant
* DIE - aborts preprocessing with an error
* WARN - show the user a warning

Now, I need some input on what kind of expressions you want in IFs.
Currently, the only four types of expressions that I support are string compares ('a' == 'b', "a" == "b" and 'a' == "b"), case-insensitive string compares (same as above, only using keyword NOCASE), numeric compares (1 == 2) and stand-alone numerals (#if 0 or #if (anything but 0), 0 is false).
All #defines are evaluated before conditionals, so #define A 0, #if A == 0 will be true.
I think I should add #ifdef & #ifndef, agreed? (So you can just do #define A, instead of #define A 1, #if A == 1).

Also, what other features do you want implemented?

After talking to Technetium, I've decided to do the thing in C ;-)
It might also compile on both Win32 *and* Linux (the latter one is only useful for things like web-applications that parse scripts or automated parsing ;))
I'll release a binary and source as soon as I have anything worth releasing ;)

Perhaps if you had looked around on the forums, you would've noticed there are 2 other preprocessors and 2 parsers already. If you really want to help, maybe you should help improve those, instead of writing a new tool.

Cuncurrent development may actually lead to better results... ;-)

While a Precompiler written in C is likely faster than the Perl equivalent, you have a much harder time developing for some features as you do not have such nice things as Eval or regular expressions (unless you use some external module which is probably not as handy as the Perl equivalent).
If you want to do it in C, do yourself a favor and use a Lexer and Parser generator (like Flex and Bison).

Currently, I only see the Eval-part as a challenge, but a preprocessor does not need the functionality of perl in the preprocessor directives.
Does Bison / Flex produce OS-independent code?
I'll try using the formal syntax definition (EBNF) at the jass.sf.net-page :-)

-- Edit
Also, does a preprocessor have any desire to parse any part of the language?
(Except, of course, globals / endglobals in headers)

--Edit 2
Since Bison does not support EBNF, only BNF, I'm currently looking into finding scripts for automatic conversion. If not, I'll have to do it on my own, by hand :(


I dont know the purpose of the parsers, but I assume they are non-related to the functions of a preprocessor?
Also, the two other preprocessors are in Java and perl, mine is in C. This makes a huge difference, in program requirements, program size and program efficiency.

I dont know the purpose of the parsers, but I assume they are non-related to the functions of a preprocessor?
Also, the two other preprocessors are in Java and perl, mine is in C. This makes a huge difference, in program requirements, program size and program efficiency.

The choise of language is probably not of much importance to the end user, as long as the program does what it is supposed and doesn't take forever. The question though is what functionality really is needed. The normal functionality of a pre-processor is ofcourse needed (like #ifdef, #define, #include and such), but the there is also a need for extended #includes of data in tables. Another thing that is also nice is to add constructs like traditional for-loops, which isn't the task for a normal pre-processor. A third thing that would be good would be an optimizer, which evealuated stuff like 6*9 (which, of course, is 42). Maybe it would be a good idea to create separate tools for these task, and specify how does should work together?

By the way, feel free to borrow code from the _ejass_ pre-compiler-thingy. ;-)

For one, it does matter a bit to the end-user in regard of speed. Also, the post about language was because I'm making a product in C, I won't be joining the Java or perl one (especially not the Java one, since I dont know Java ;))

I agree that they should be different tools, atleast the optimizer and the preprocessor. What other tools can be decided later on :)
They can in turn be implemented by an IDE or a GUI. (like MSVS and their commandline tools (cl.exe etc))

The pre-processor I wrote is in JavaScript. Don't even bother saying you don't know the language: it's so easy you can learn just by looking at it. It's the same JavaScript used in HTML (actually, this is JScript, the MS ri-off of JavaScript). It's fast (Blizzard.j in a blink), small (5 KB), only 200 lines, it's open source (doh, it's a script), and you don't need a parser (like perl) to use it.

The one in perl has the advantage that it includes conditional includes for tables. I haven't tried it yet, but I'm sure it's just as fast.

Now tell me again why we need a third preprocessor? If anything, I'd like to see a tool that converts Java/C like code to JASS code, instead of another #define/#ifdef preprocessor.

If you re-read my latest post above, you will see that I propose 3 kinds of tools, one of which is pretty much the tool you seem to want. I won't go into a discussion about which language is best for writing such things but it would be a good idea if it's non-commercial and works on Windows, Mac and *nix though (WarCraft is actually runnable under Linux).

If there are someone else actually interested in discussing and defining these different tools, how they shall co-operate and what format they shall output, then I might take an interest in it, otherwise probably not. Are there anyone else?

*nix? How, emulation? Emulation of windows is generally !good :/
What I make will be runnable on Win32 and *nix, I have no way of compiling or testing it for Mac. OSX is *nix, isn't it? If so, it should be a breeze :-)

Vidstige that's a pretty good idea :)

I agree totally, Vidstige.
For a list of programs, my suggestions are:

• preprocessor -
  does non-syntax related stuff like #ifs, #ifdefs and #defines. (macros?)
• enhanced syntax processor -
  adds programming shortcuts, like x++ (to set x = x + 1) and similar.
• optimizer -
  does common optimization-stuff, both memory and speed-wise.
• packer -
  like a protecter, but with an unpacker too. (only to reduce size for bnet)

The preprocessor and the enhanced syntax processor could / should be combined, if they are you should be able to disable the one or the other (to reduce preprocessing time).

-- Edit:
Regarding output, all tools should output to .j (except packer), and possibly have support to incorporate it into a(n existing) .w3?-file. Input should be like .ej or similar, packer should take and return .w3? ;)

WarCraft is runnable under Linux using WineX from http://www.transgaming.com/ . The sad things are though that you have to pay a three month prenumeration fee to be allowed to download it. I haven't yet been able to start the WE either. :-/

Your list of tools looks good daxxar. I would suggest though that the tools should be availible as separate tools. This would mean that if someone writes a really mean preprocessor, then several different syntax extenders could be used with this. A good idea when defining what the preprocessor shall do is probably to look at what the normal C-preprocessor does, and what the version of the eJass-preprocessor that AMAI uses does.

For the syntax extender, it is pretty easy to implement things as traditional for, while, ++, += ...

It would probably also be a good idea if we consider how a normal compilation process works. What we are trying to is taking different source-files, creating raw Jass code of them and stuff them into a .w3 file...

How about splitting this discussion, and make a separate discussion about the pre-processor and another about the syntax enhancer, since these tool probably would benefit from being kept separatet in functionality.
------------------------------
Edit: I have actually managed to start the WE under linux. It is a bit shaky, but I am able to import my AI scripts into the maps at least.