Proposal for creature config

[Ivan]

Check this: dl.dropbox.com/u/22372764/vcmi/hotapackage.zip
Creatures from hota + incomplete config for vcmi. All issues I know about are noted in readme but config may have other issues as well like missing or incorrectly named fields.

The creature config needs to be named mod.json and placed in Mods\modname folder.

It looks that it is time for some “mod config” format. Specifying all creatures\heroes\artifacts\etc in one file will make it too big.
I think we should use something like this:
A:

{
    "creatures" :  "file1.json", "file2.json"], //list of files that actually store creature configuration
    "artifacts" :  "artifacts.json" ], // same for artifacts
    "filesystem" : { <same as current format> }
    <any extra fields like mod name>
}

B:

{
    "content" :  "file1.json", "file2.json"], //list of files that store *all* content (e.g. creatures and artifacts may be stored in one file)
    "filesystem" : { <same as current format> }
    <any extra fields like mod name>
}

[Warmonger]

You’re right, I just wanted to make mod handling as simple as possible to just see if they work at all.

Big thanks for the package Will try to load it when I have some time later.

[AVS]

1. move “filesystem” to separate file(s) too
2. variant B is better, let engine detect config file content by content itself, not by filename.

[Ivan]

AVS,

1. Why? I’d rather keep all this data in one file. Filesystem even for complex mods won’t be large - 10-20 lines top and may be further decreased with some auto-detection.

2. It won’t be “detection”. In “B” file with content will look like this:

{
    "creatures" : {...},
    "artifacts" : {...}
}

Engine will merge all files in one structure and send all parts from it to creature\art\etc handlers.

With “A” data in each file won’t need “creatures” or “artifacts” nodes - they are already specified in main file.
As result json structure in “B” will be one level bigger.

[AVS]

AVS,

1. Why? I’d rather keep all this data in one file. Filesystem even for complex mods won’t be large - 10-20 lines top and may be further decreased with some auto-detection.

2. It won’t be “detection”. In “B” file with content will look like this:

{
    "creatures" : {...},
    "artifacts" : {...}
}

Engine will merge all files in one structure and send all parts from it to creature\art\etc handlers.

With “A” data in each file won’t need “creatures” or “artifacts” nodes - they are already specified in main file.
As result json structure in “B” will be one level bigger.

1. Just some kind of standardization.
2. “B” is more flexible. F.e. main file scheme is independent of available JSON “formats”. But I agreed that flexibility costs smth.

[Warmonger]

After r2892, I tried to load mods with this code:

	auto resourceLoader = CResourceHandler::get();
	auto iterator = resourceLoader->getIterator(](const ResourceID & ident) ->  bool
	{
		std::string name = ident.getName();

		return ident.getType() == EResType::TEXT
		    && std::count(name.begin(), name.end(), '/') == 3
		    && boost::algorithm::starts_with(name, "ALL/MODS/")
		    && boost::algorithm::ends_with(name, "MOD"); //all mods have "mod.json" name - does it make sense?
	});

	std::set<std::string> foundMods;
	while (iterator.hasNext())
	{
		foundMods.insert(iterator->getName());
		++iterator;
	}

But seemingly it does not work, even if it follows similiar code fromCResourceHandler::loadModsFilesystems().

Of course I renamed creatures.json to mod.json for now, but it returns 0 results. Any tips?

[Ivan]

Filesystem with ALL/MODS/whatever entries exists only during FS loading (initalLoader).
To get all available files check CAnimation::init():

auto & configList = CResourceHandler::get()->getResourcesWithName(
	                      ResourceID("CONFIG/MOD", EResType::TEXT));

BOOST_FOREACH(auto & entry, configList)
{
	auto stream = entry.getLoader()->load(entry.getResourceName());
	std::unique_ptr<ui8]> textData(new ui8[stream->getSize()]);
	stream->read(textData.get(), stream->getSize());

	const JsonNode config((char*)textData.get(), stream->getSize());

	<any manipulations with config JsonNode>
}

Note that any files in root mod directory (“Mods/modname/”) are not available after loading - you’ll need to move them to config directory for now (and add appropriate entry to filesystem).

All of this should be fixed at some point - currently mod support from filesystem is quite basic.

EDIT:

1. Warmonger, in your last commit you’ve used “” as path sepatator for #include - please replace with “/”

2. What do we need from filesystem for mods?

• complete tree with all loaded mods
• tree for each mod which consists from entries from this mod + dependencies + base game.
• tree to get data only from specific mod shouldn’t be needed - previous entry should suffice.

Anything I missed?

[Warmonger]

Well, what I want is:

• Put mods into Mods\ folder
• Find all these mods (mod.json files)
• Load them with ModHandler

I do believe it’s pretty simple to understand and should be easy to do

[Ivan]

Yeah I got your idea.
As I said - right now files in “mod root” directory are unavailable for engine.

I’ll try to fix this soon but right now something similar to CAnimation::init() is the only way to get files from mods. Should be more than enough for creature test.

What I want to do:

1. Filesystem loader scans Mods directory and sends list of mods to ModHandler.
2. ModHandler marks enabled\disabled mods, reorder them to load dependencies first and sends resulting list back to filesystem.
3. Filesystem loads received mods and generates mod-specific filesystems (or some other way to get mod-specific data)

[Warmonger]

I’m not sure about that, I’d like to pre-load ALL the mods in firts place and then only enable / disable them at run time. So list of used mods will be stored internally in game and filesystem doesn’t have much to do later.

There are alreayd some structures to store all creatures and artifacts in Modhandler, just they’re unused.

[Ivan]

If all mods are loaded into filesystem then what to do with files from disabled mods? Especially when they override files from H3 or even files from another mods.

Disabling\enabling mods in runtime should be easy. In this case loading process will be:
During loading:

1. Scan FS (same)
2. send list of available mods (maybe including main config file) to modhandler
   After this anytime when (re-)configuring is needed:
3. ModHanders sends list of enabled mods
4. FS Loader loads selected mods into main FS (same)

[Warmonger]

I can understand that there can be only one filesystem at once, but for new creatures that’s not an issue. We can load an arbitrary number of them to ModHandler and then pas only some to CreatureHandler, so only enabled creatures will appear in random stacks.

Still, before we attempt to disable or replace anything, first we should add new content to game.

[Ivan]

Ah. By “preloading” you also mean loading creatures configs into mod handler? Loading them should be OK but parsing may be a bad idea - what if this this creature is incomplete and based on some parent mods?

Anyway files with creatures are also part of filesystem, content (like icons) is also part of filesystem. This needs to be handled somehow… Can be tricky - in most cases prefix to URI (SPRITES or DATA) is inserted in bitmap\sound\def handlers. Mod-specific prefixes (“MODS/NAME/SPRITES/SOMEANIM”) won’t work with them.

So far best solution I found is to “tell” FS list of mods where file should be looked for with possibility to set default search list (=list of enabled mods).

Any other possibilities I found are even more tricky to implement.

1. Why? I’d rather keep all this data in one file. Filesystem even for complex mods won’t be large - 10-20 lines top and may be further decreased with some auto-detection.

1. Just some kind of standardization.

Kinda missed that. Apart from list of files with content there will be such data like list of required mods, name, description, etc. So storing FS description here won’t help consistency anyway.

[Warmonger]

you’ll need to move them to config directory for now (and add appropriate entry to filesystem).

Could you be more specific? I tried this

		"DATA/" :
		
			{"type" : "file", "path" : "ALL/Data/H3ab_bmp.lod"},
			{"type" : "file", "path" : "ALL/Data/H3bitmap.lod"},
			{"type" : "dir",  "path" : "ALL/Data"},
			{"type" : "dir",  "path" : "ALL/Config/Mods"}
		],

But failed. Not sure what the filesystem is really looking for, if anything.

EDIT: It worked after I hardcoded entire path

>getResourcesWithName (ResourceID("CONFIG/MODS/HOTA/MOD", EResType::TEXT)); 

Not really practical, but at least can do some tests now.

[AVS]

I suspect that access method for resources (all from H3/wog + all graphics) and VCMI configs (and most likely scripts except ERM) have to be different.

1. resource is defined by name and type but not (relative)path. And loaded from path defined in filesystem config
2. config cannot be accessed the same way, because different mods may have config files with same name.
3. main config + filesystem config must be handled separately anyway.

[Ivan]

In your case something like this should work:

"CONFIG/" :  <- this is directory where all files will be placed after initialization
          
             {"type" : "dir", "path" : "ALL/MODS/HOTA/CONFIG"} <- this is path to directory, relative to H3 root directory. Prefix "ALL" is needed for user-specific directories on Linux.
         ]

This should make files in mods/hota/config directory available via “CONFIG/filename” URI from game.
In case of multiple mods with such config only file from last mod (alphabetically) will be available. To get all files check CAnimation::init();

2. config cannot be accessed the same way, because different mods may have config files with same name.

True. This is main problem with URI’s right now. But this is true for graphical content as well.
Let’s imagine mod with 1 creature. What is the most logical name for creature icon? Icon.png? And half of mods will have such name…

[AVS]

Let’s imagine mod with 1 creature. What is the most logical name for creature icon? Icon.png? And half of mods will have such name…

Right now nearly all mods are in fact targeted to era (including HotA in future versions). Era has same issue with filenames. So requiring unique resource names wont be surprise for modders. OTOH this is an issue anyway.

[Warmonger]

In your case something like this should work:

It didn’t, so i restored last working configuration in r2896.

Have a look at new mod & filesystem.
config.rar (4.09 MB)

[Ivan]

Reuploaded archive, same link: dl.dropbox.com/u/22372764/vcmi/hotapackage.zip
Just unpack it into root directory.

Main config is available as

ResourceID("CONFIG/mod", EResType::TEXT) <- in this case extension not needed
ResourceID("CONFIG/mod.json") <- type will be determined from extension

[Warmonger]

Thanks, now it works in Mods folder.

But you included old mod.json file in package, it can’t even be parsed correctly. I also added some info (creature upgrades) in new version.