Modding system discussion

@Ivan@scripts_online
i was first (see wiki discussion) but of course totally agree

@Ivan

If use string IDs we should be of format.
Capital case with “_” and w|o spaces, camel case etc. Or engine should handle most of all cases (“supreme archangel”=“supreme_archangel”=“SupremeArchangel” …)

Thank you for your contribution, this discussion makes more sense than I expected :smiley:

  1. Lua is popular, but already obsolete. It doesn’t seem to be developed much further.
  2. Python has more advanced features, like explicit class and metaclass support, which may be useful for modding.
  3. Python comes with tons of documentation, examples and rich library in case you may need to write something ambitious.

Lua implementation in H5, for example, was just a set of fixed functions… and you couldn’t go beyond them with your own code.

I agree with that. Json files may be easier to scan and manage in first place.

That makes sense in general, but there’s an issue: client-side scripts (GUI) which result in change of game state, such as garrison drag-and-drop. This must be done in a smart way and I can’t help much with network programming.

Well, I believe some definictions can be conditional or generated at run time - for example, in a middle of a map you may want to create new artifact depending on player’s choice. Or multiply certain type of creature, each with new abilities.

Every mod resides in its own folder under [game_dir]/mods/… Uninstalling shouldn’t be a problem. Original files remain unchanged.

For AI it may be OK. Client/Server/Lib executables shouldn’t be replaced by mods. This would be the false direction. How to support several mods instantly then?

No problem. You can use both point 1(resource override) and point 3(changes to trunk).

AFAIK python can be connected with C with just a macro in the class/function definition. So calling a C++ function from Python shouldn’t be a problem. The big problem is that we have to define places in VCMI code where we are likely to expect an addition to the normal behaviour(invoke predefined python methods from cpp). This can be a major task. Don’t know if data type conversions will cause trouble. It would be nice especially for this discussion if someone could look how VCMI CPP/Python Integration could be done. Which steps are necessary exactly to call a function from cpp and the other way round? Where are performance bottlenecks? Which other risks are there? How could a creature ability be added? What is easily possible, what takes more work?
Developing mods in Python, than directly in c++ is more complicated as there is one additional layer anyway. Python as a map scripting language seems to be quite fine, but don’t know if it makes sense to introduce gameplay changes which affects lib, client, server and ai.

This makes sense. We should define a limited set of method declarations which can be overriden by Python mods dynamically. That methods have to be invoked from VCMI Core at specific positions.

About defining objects in python:
RMG & map editor shouldn`t have dependency on python engine. Every object in mod should be accessible w|o executing scripts.

  • Give to scripts access to CCallback - will allow client-side scripts to safely modify game state just like in our C++ gui code.
  • For interaction between client and server parts of mod something like this should work:
  1. on client convert python data to c++ string (in scripting system wrapper)
  2. client sends message (modID + string) to server
  3. convert c++ string back to python data on server
  4. pass received data to python script on server

I know that there is Boost.Python which may work for us. Don’t know anything about this lib though.

Asking modders to use C++ is even worse - it almost requires some computer science knowledge. Writing every ability in C++ by VCMI devs isn 't an option too.
Using python for gameplay mods shouldn’t be that difficult - we already have clearly defined events (net packs). All we need is way to pass this events to scripts.
For example some kind of “Random encounters” mod will have script attached to “Hero moved” game event and will be triggered by engine each time a hero moves.

I vote YES for python.
I vote for Dynamic Library Plugins also (similair way as AIs are done in engine) with dynamic loading (you may enable/disable module)

“Things Integrated In Client/Server Executable” in my sentence was the same as your “Include major mod mods in trunk” but worded different (and also allows for branches/forks). Who for the hell would be patching binary code if we have the source open for everybody :stuck_out_tongue: ?

@JSON
i think there should be configuration files as for map editor independence from python, but i think there could be a few types of configuration files

  1. those loaded at mod loading (name it “autoloaded settings”)
  2. those loaded on demand by python script function (or API call for binary plugins), call them “dynamic settings”
  3. those loaded on demand to patch any of (1) (2), call them “settings patches”

since all config can be autloaded and demand-loaded there is no reason to not define objects directly from python code without use of JSON files (but those objects would be unavailible in map editor until we have placeholder objects in JSON configuration which will be changed in runtime)

You’re right! I’ve said above in point 2 that we should use Python only as a map scripting language, but I think for specific purposes we should provide modders a easy way to realize the most common needs. The interface and the framework should be documented, clean and stable. With those Python extensions you should be possible to do e.g. such tasks(common tasks for 90% of modders):

  • Add/modify creature abilities
  • Add/modify secondary skills
  • Add/modify (special) artifacts

In order that this works well we should implement standard creature abilities in python too. I think this would make development more straightforward.

If someone wants to do greater modifications e.g. to increase the size of the battlefield, introduce a new skill system, town conversion,… we should not blow our code that such things would be possible in python. As this would be a major task and combined with much probability. What if the modder wants then to have feature XY to be moddable? Then we’ll come to point 3, where an experienced modder could implement that features what he wants directly in the source code. Therefore as said in my first post the changes should be of high quality, stable, separated from OH3 logic and optionally.

The problem is that development of these abilities was NOT straightforward. Each of them needs an access to callback for checks and triggers or condictions placed in the very different places of code. Adding remaining spells will also require brand new pieces of code and reorganization. These mechanics are simply very complex and hard to plan in advance.

On the other hand, Nix’s ability from HOTA is one-liner in current C++ code.

Ahh, so how’s the development of siege going? If you had a look at that code, you’d see how complex it is and how many things are fixed or hard to figure out at all. It can’t be done “from outside” as the code inside is a real puzzle on its own.

In order that we can provide ability modding we have to re-arrange a lot of source code. But perhaps it may be possible. Even for spells. My initial thought was that it is by far too much complicated to do such gameplay mods(spells,abilities NOT radical changes!). Not only for us developers, but for modders too. They have to override several methods to get this working. That’s why I’ve thought that python would be used best as a map scripting language only where you could do simple but more advanced things compared to what OH3 offers. I don’t want to step to deep into technical details, as I just wanted to talk about the rough picture of the modding system. So HOW gameplay mods could be realized from a technical point of view, needs much further investigation. BUT adding new abilities may be more probable as it’s more centered and bordered than town conversion or such things.

:mrgreen: So true! Siege and battle interface code have to be re-factored anyway. Even a perfectly modular written code wouldn’t allow easily a greater battlefield without any changes to source code in that case.

5.2 version released 16 Dec 2011. Obsolete? :-/

AVS why Lua? maybe it should be Tcl/Tk or even LoLcode? (joke)

(calmly, and seriously)
Python is much better, much richer, and has good integration with C/C++ code which work both sides (C/C++ can call Python Scripts, and Standalone Python Scripts can use libraries not in python, which are mostly in C/C++, and we may enable the feature for our “internal” scripts). It’s jack-of-all-trades. Currently you cannot have anything better than C/C++ & Py (well i lie there’s Lisp there, but it’s somewhat harder to learn).

Lua is most popular from game specific script languages (why eg. Daedalus should be more popular?), but Python is most popular from all script languages (well if you not count real noobies, Tcl-like syntax is easiest to learn and easiest to parse and many game-specific/project-specific languages may have similair syntax)

and most important Python is really powerfull language - you can do anything possible in C/C++ in python (well almost), and if it’s not possible directly you can write library for it in C/C++ itself.
(great breath)

Is this just an accident?

Lua has complete design, much easier to learn (than python). The “power” of Python is not required for gamedev.

Well, Lua was popular at some point and now many developers simply follow that way, as some players got used to it.

Still, we can be innovative and choose better solutions. The same way as JSON won with XML.

Python isn’t “game specific” script languager
It’s “general use” script language with abilities onboard for which you frequently would need compiled language if you use other than python script language (another script languages are frequently much poorer and are not capable of many things python can do)

Can you provide an example: what python can do and lua can`t?


Instead of arguing which language is better, let’s select a feature (f.e. some WoG artifact) and design scripted implementation and engine interaction for it. I suggest Gate Key.

Great! And every modder will have to find 32-bit Win, 64-bit Win and several Linux systems to make his mod available to everyone. Not to mention nice segfaults caused by mods (usually error in script won’t crash whole game).
This is a nice feature but will have too many issues without any significant gain.

Networking? Python have nice standard library much bigger than Lua. It may remove any need to use native dll\so plugins. Won’t make notable difference for WoG objects but may prove useful later.

Of course lua has no networking in stdlib (but 3rd party libs are present). Lua designed for embedding scripting (Python is designed for complete enduser applications), therefore it lightweight (only one shared/static lib). If we are planning in future to rewrite entire engine in scripted language (with minimal native code for speed sensitive parts) Python is great, but for game-objects lightweight engine is better.

I prefer One Powerfull Spartan (Python) than hundred Castrated Obedient and pain-resistant Wariors (Lua, LOLcode etc.).
Especialy that for as huge project as VCMI. For VCMI even the nice language Tcl/Tk isn’t enough powerful, although it’s really lightweight (the syntax is made for most lightweight interpreting).

@DLL/SO
well Python should be enough, but we may enable is as option. most of binary plugins will only use internal VCMI API so porting is just recompiling (well if people handle LE/BE properly), and we can make it use same API as python, we can even do Python-Support as one standard binary plugin (it would be just a wrapper from VCMI C++ API, to Python API), and doing it this way or just giving powerful API for binary plugins may enable other modders to add support to another languages (eg. Lua, Lolcode, anything) independently of our python-support. And for limiting problems that SOME plugins would need exotic external to plugin binaries, we can make standard library of tools published with VCMI, and perhaps made even an API wrapper for them (or just put it with VCMI and let modders use their default API). (Exotic libraries are only picked when standard tools doesn’t satisfy and picker don’t know popular non-standard tools, and know one exotic which match his needs, also if we really fear incompability we can make license for binary API so people using them is forced to license Open Source, so plugins may be forked and/or recompiled for another machines)