Modding system discussion

5.2 version released 16 Dec 2011. Obsolete? :-/

AVS why Lua? maybe it should be Tcl/Tk or even LoLcode? (joke)

(calmly, and seriously)
Python is much better, much richer, and has good integration with C/C++ code which work both sides (C/C++ can call Python Scripts, and Standalone Python Scripts can use libraries not in python, which are mostly in C/C++, and we may enable the feature for our “internal” scripts). It’s jack-of-all-trades. Currently you cannot have anything better than C/C++ & Py (well i lie there’s Lisp there, but it’s somewhat harder to learn).

Lua is most popular from game specific script languages (why eg. Daedalus should be more popular?), but Python is most popular from all script languages (well if you not count real noobies, Tcl-like syntax is easiest to learn and easiest to parse and many game-specific/project-specific languages may have similair syntax)

and most important Python is really powerfull language - you can do anything possible in C/C++ in python (well almost), and if it’s not possible directly you can write library for it in C/C++ itself.
(great breath)

Is this just an accident?

Lua has complete design, much easier to learn (than python). The “power” of Python is not required for gamedev.

Well, Lua was popular at some point and now many developers simply follow that way, as some players got used to it.

Still, we can be innovative and choose better solutions. The same way as JSON won with XML.

Python isn’t “game specific” script languager
It’s “general use” script language with abilities onboard for which you frequently would need compiled language if you use other than python script language (another script languages are frequently much poorer and are not capable of many things python can do)

Can you provide an example: what python can do and lua can`t?


Instead of arguing which language is better, let’s select a feature (f.e. some WoG artifact) and design scripted implementation and engine interaction for it. I suggest Gate Key.

Great! And every modder will have to find 32-bit Win, 64-bit Win and several Linux systems to make his mod available to everyone. Not to mention nice segfaults caused by mods (usually error in script won’t crash whole game).
This is a nice feature but will have too many issues without any significant gain.

Networking? Python have nice standard library much bigger than Lua. It may remove any need to use native dll\so plugins. Won’t make notable difference for WoG objects but may prove useful later.

Of course lua has no networking in stdlib (but 3rd party libs are present). Lua designed for embedding scripting (Python is designed for complete enduser applications), therefore it lightweight (only one shared/static lib). If we are planning in future to rewrite entire engine in scripted language (with minimal native code for speed sensitive parts) Python is great, but for game-objects lightweight engine is better.

I prefer One Powerfull Spartan (Python) than hundred Castrated Obedient and pain-resistant Wariors (Lua, LOLcode etc.).
Especialy that for as huge project as VCMI. For VCMI even the nice language Tcl/Tk isn’t enough powerful, although it’s really lightweight (the syntax is made for most lightweight interpreting).

@DLL/SO
well Python should be enough, but we may enable is as option. most of binary plugins will only use internal VCMI API so porting is just recompiling (well if people handle LE/BE properly), and we can make it use same API as python, we can even do Python-Support as one standard binary plugin (it would be just a wrapper from VCMI C++ API, to Python API), and doing it this way or just giving powerful API for binary plugins may enable other modders to add support to another languages (eg. Lua, Lolcode, anything) independently of our python-support. And for limiting problems that SOME plugins would need exotic external to plugin binaries, we can make standard library of tools published with VCMI, and perhaps made even an API wrapper for them (or just put it with VCMI and let modders use their default API). (Exotic libraries are only picked when standard tools doesn’t satisfy and picker don’t know popular non-standard tools, and know one exotic which match his needs, also if we really fear incompability we can make license for binary API so people using them is forced to license Open Source, so plugins may be forked and/or recompiled for another machines)

I like the Warcraft 3 or Savage 2 system.

If you a ressource is inside a map or inside a special folder it just overwrites the standard ressource. In Savage 2 its working by the name:

files inside resource0.zip (standard resource) are overwritten by resource1.zip are overwritten resource2.zip are overwritten by the resources in a map (which is a zip archive too).

This makes it very easy to add resources and remove them. And make maps special without external mods :slight_smile:

suggestion: triple index.
Let give each object Unique Number on list of objects of similair type (dfferent towns, different units, different heroes, etc), but not give only one number but as much as 3.

Global Object ID - unique to whole game.
Mod Object ID - unique to mod. 0 will be first object from that mod each one next.
Patch Object ID - there can be patches for other mod and the list would contain only new elements, and will contain every new Object defined in the patch.

Accumulating of IDs - mods will have order, Global Object List list is concatened together ModObjectLists respecting that order, ModObjectList starts from items from unpatched mod, then content from all patches to this mod in order. Object ID is just an Index fro ObjectList, which may be treated as an array. It’s forbidden to changing size of the lists outside of data preload, however if mod author may reserve additional IDs to be fullfilled in runtime, and the reserve number is a constant, so same number count will be used even if situation don’t need as much.

4 Local Object ID - unique to mod, original SoD and all mods that current mod depends on. (circular dependency forbidden of course)

Mod config (all new|altered objects) should be written as if there only this mod and all it dependencies. Mod Scripts also use this ID by default other id`s can be obtained via API.
This will give more compatibility with existing mods.

majaczek, and what about patches to patches? :stuck_out_tongue:

suggestion: no indexes
I don’t see any reason to expose internal numeric ID’s to modders if they can use human-readable strings as ID’s. And why should modders bother themselves with questions "What id have that creature I added? How many ID’s should I reserve for runtime?"
Compare this: WoG.creatures(1)
and this: WoG.creatures(“supreme archangel”)
Performance loss from string comparison will be neglible in scripting and second version is much more readable.
To avoid name conflicts we can use prefixes\namespaces. Patching\extending other mods can be done in this way as well.

Simple accessing creatures by string name is of course more convenient, but what about accessing multiple ones? Imagine you want to get all WoG tier 8 creatures and not the others. Simple for loop could do the trick, but it’s not likely with string array only. So I’d leave ID just for when you need it.

Give access to dictionary {“name”: , “name2”: } and use it with “for” loop.
Either way numeric ID are fine as long as they’re not the main way to access game data.

Patches to patches would be formally patches to the same mod as first patch, and it will sort with priority. About string indexes: you can do :

int mon; mon=monsters.mod[thismod].findID("two-headed phoenix"); somefunction(mon);

numeral indexes are useful, and we can always proxy it to string IDs if each object has name field. my approach allows to cycle all monster from mod (including patches) or cycle all monsters from entire game without knowing what is inside. SO:

int mon; for(mon=0;mon<monsters[findmod("funky town")].count;mon++) {monsters.mod(findmod"funky town")[mon].givebonus(permanent,bonus.mod[thismod]("can sing"));}

and it works even after adding some patch to mod which adds “bluesman” to “bluesrockman” upgrade and “bluesrockman” itself. (this one can be also achieved by looping through all monsters from faction, but more handy would be example of tiny mod which doubles buycost for all creatures in the game through script, or the mod which removes specific bonus from all mods in the game or just replaces it with another)

EDIT: and accessing by numbers (array is sorted, and the ID variable are 8bytes or less) is much faster, an if we want any other way of accessing we can always make a dictionary with any key type and integer value type.

Why do you insist on making things so complicated? It should be so much easier:

monID = creatures("two-headed phoenix")
giveCreatureBonus(monID, Bonus)

OR (overloading)

giveCreatureBonus("two-headed-phoenix",Bonus)

OR

monID = OurMod.creatures(index_of_phoenix);

OR even

monID = OurMod.creatures("two-heade phoenix")

If somehow we allow more creatures with same string id - but this would make no sense I believe.

Also, can’t see why “mod” could be a part of “monsters”.

mod isn’t part of monsters. “monsters” is extended list of monsters, where each extended list has one aray for game, arays for mods, and arays of patches. perhaps “frommod” would be better name. all object list would inherit from extended list class.

another way of speaking: bonuses, monsters, artifacts, castles handlers are hiearchic set which each has one array with all elements of the type from game, array of arrays which first index is from mod and second is ID in list for that mod, and same way 3d array of objects from patches. of course to not double memory each array may be just array of (inteligent) pointers to real data.

the idea is internally to have fast numeric access, which will be exposed for things like making loops or just caching IDs for faster than string access, and in the same time has wrappers which gives you numeric ID to a variable by giving any of key (key may be string but not limited to). note that normal pointers (in C) are homogenous to arrays so perhaps the real table will be the one from whole game and other arrays will be pointers shifted from that array (so whenever access you use you will get access to same object and only one reference is done).

EDIT: two diiferent town mods may want just to add a pirate… if we don’t have unique ID how do we know the pirate is from one of other? for a programmist is easier to access through arrays/numbers, and that should be done internally, and we have no reason to not expose it. Of course we will provde a wrapper so modders still can access it in high-level way, and same time allow optimising his script for master modder. and it allows many things which would be hard/impossible to do when we will base on only-string indexes.

Fast numeric access? In scripting language? In most cases number (part of string) still must be converted into number(in memory). And scripting languages are heavily optimized for string comparisons anyway.
Numeric ID aren’t safe - what if mod with 10 objects will remove object #2? Should we shift all remaining IDs and broke all code that uses objects 3-10 by number? Or modder will have to check object validness every time?

Good point. To keep everything simple we can do it in such way:

  1. Accessing objects from this mod or from its dependencies - via string ID
creature = mod.creatures"pirate"]
creature.giveBonus(someBonus)

Note that “creature” should be instance of class CCreature (or something like that), “mod.creatures” is map\dictionary which have only creatures added by this mod or its dependencies.
2) Accessing all creatures available in game - neither we can use strings here (conflicts between mods) nor numbers - we don’t know load order of mods and how many objects were added by them.

for creature in engine.creatures:
    if creature.level == 8:
        doSomething()

Here “engine.creatures” is an array\list

ID discussed here are class indentifiers, not instance identifiers.

  1. I see no reason why objects IDs would be removed at run time. Once you launch the game with set of mods, they are necessarily here. At most you can append new at run time, at the end of the list.
  2. Numeric IDs of ALL game objects are distributed by Mod Handler at map launch. We can only use queries to get numeric IDs for convencience. In example above, there is a function that converts string ID into numeric ID.

Numeric IDs are imortant as you don’t know which mods will be used at run time. Imagine the script which generates random army of tier 7 creatures. You can use a function (or even simple iteration at full range) to get an array of numeric IDs of all tier 7 creatures. Strings won’t be of any use in this situation.