Support for new formats not present in H3

Since need for more common formats had appeared already (look 800x640 thread) this is my thoughts about this:

[size=117]Image formats[/size]
SVN version can already load from any format supported by SDL_Image.

[size=117]Animation formats[/size]
Currently only def which is pain to work with. Possible formats:

-** Animated file** (like gif) can be somewhat difficult to add due to lack of animation support from SDL_Image.

  • Directory with multiple images is most easy to add but this will remove possibility to sort different files since folders will be used exclusively for animation.

  • Frame number in filename - form animation from files file#1.png, file#2.png… - doesn’t differ from previous much but do not require specific directory structure, name separator can be changed - not sure if such names are allowed on Win.

  • Sprite sheets working with it is more complicated but will result in one file with better compression and loading time. Config file is almost mandatory however.

For me the optimal solution is adding frame number to file name - no need for extra work, no limitation to file system.

To finish image formats support fastest solution will be to ignore file extension so HeroScr4.pcx and HeroScr4.png will be considered as same file but as result HoTraits.png will override HoTraits.txt from lod. Quite rare situation I suppose.
More correct solution for image loading is to check every possible extension which will be much slower however.

+1 for the frame number in filenames.

Instead of checking all possible extensions when you need the image, you could cache all needed images upon loading the map (scaled to reduce memory usage on small resolutions) for later usage.

Or avoid duplicate filenames by adding any letter to all used image files. Example:

i_file#1.png
i_file#2.png
i_file#3.png
i_HeroScr4.png
HeroScr4.pcx

Adding frame number to filename seems to be a good idea. The # character is not forbidden on Windows. I have just two suggestion:

  1. we should use two numbers for specifying the position of frame in animations - some animations have groups of frames (like creature animations) that should be supported too. So I suggest to use file#0.png for animations with one group of images and file#0#3.png for multi-group animations (the number of group would be the first number)
  2. we should force people to use just one (popular) file format for images for two reasons:
    • There will be no ambiguity when loading a file which has duplicates with different extensions.
    • Creature animations in battles are very memory-consuming. They must be kept in memory in a compressed form, with at least RLE compression.
      If this behavior seems to be very undesired, we could just emit a warning about such ambiguity and transform heavy animations into RLE-compressed versions.

Well… according to that logic, everyone should use .def :stuck_out_tongue:


Another thing (besides groups of frames) are .MSK/.MSG files that describe corresponding .DEF file. They’re needed only for adventure map objects sprites.

I think it’s acceptable.

I believe we can assume that graphics will have different names than config files. Besides, we can check extension for calls coming from CLodHandler::getTextFile and ignore it in BitmapHandler::loadBitmap.


I once thought about adding as another animation format 7z archive with graphics + txt dscribing them. Well, that was before I downloaded and looked into LZMA SDK (I wanted 7z compression for savegames anyway) :stuck_out_tongue:


Anyway, firstly we need to replace CDefHandler’s and CDefEssential’s with more general structure that contains only the info actually needed by VCMI. [The first class is bloated with def format details, the second - awkward.]
Then we should be able to easily add new formats of animations, transparently for the rest of engine.

We could simply prioritize .def over .something or vice versa but introducing a decent ordering of arbitrary formats is not that simple and understandable.

Simultaneously we could introduce a resource manager that would prevent multiple loading of the same file and provide a more abstract way of obtaining resources (uniform for files from lods and outside of it - it still seems to be a problem in case of campaigns).

To make clear how VCMI load graphics.
At starting VCMI creates file list:

  1. Creating entry for every file in *.lod
  2. Scan specific directory for files and replace entries with same name as in lod. But since HeroScr4.pcx and HeroScr4.png are different names they will got separate entries.
    After this on opening hero window game will ask for entry "HeroScr4.pcx"
    So ignoring extension is fastest way to get this work done.

@Tow dragon
1)You are right. Forgot about that thing.
2)Png is probably best one - any possible bits per pixel including indexed, can have compression (deflate). However there is some stuff for single images that uses other formats: non-H3 pcx, probably bmp too. Forcing one format may be too much. “Please, use %our favorite format name%” with optimizations only for this format should be enough.

Since all available anim groups are not required in battles loading only some of them can help a bit even without any compression.

Forgot about that. What they are used for? Some masks? Extracted several such files and .msk looks identical to .msg. Have not found any mentioning of “.msg” in the code too - only msk.

Do we need something except SDL_Surface for every frame and anim groups?

I had cases where such thing will be nice to have. Partial support for this is present in SDL already - SDL_Surface::refCount so counting how many requests we have got for each image is not needed.
For me it should be something like this:
graphics - for loading\unloading resources, the only part acessible from GUI code.
graphics should use BitmapHandler and some kind of AnimationHandler for loading single image \ multi-frame animation. A way to load one image or one group from anim is needed as well.

IIRC they are used to store sizes of advmap objects (width & height) + masks that tell where on object’s graphics is shadow/full transparency.

I think we don’t.

@resource manager

I’m not sure how and if we can use SDL_Surface::refCount. If we can use that member freely, that’d be nice but I’m not sure how would it work.

I’ve used refcount to load single image from def an it works fine. It is used when freeing surface - usage example is here at the end of page:
sdl.beuc.net/sdl.wiki/SDL_Surface
Code for image loader can be something like this

if (s) // already loaded
     s.refcount++;
else
     Load(s);//Load new

For unloading it just free surface without any usage checks (will be done by SDL) and NULL-ify if this was last ref needed.

Not too much at once.

PNG compression won’t be any help, if we load PNG to an SDL_Surface. It’ll save only the HDD space, not RAM. And even loading all “needed” animations for all creatures in battle would be too much. I don’t remember values now but we once calculated with Tow Dragon how much memory that would need and the numbers were terrifying.

We need groups only for creature animations and creature animations can be hold in a SDL_Surfaces.
In the rest of graphics we don’t care about groups of frames, there even single vector of surfaces would be enough. (So we need frames and their order for most of animations. For creatures -> compressed frames, order, groups.)

AFAIR .MSK and .MSG are always the same. I don’t know what was the purpose of duplicating them like that.
MSK is used for getting width/height (int H3 tiles) of graphics and optimizations in displaying objects (it tells which tiles contain graphic of objects and which tiles require alpha-blending). MSK also affects games logic - they’re used to determine whether special battlefield ground (like holy ground) covers given tile.

20 8-bit images with 450x450 size for 15 creatures = 60 Mb. That’s a lot of memory indeed.

We can load PNG file (or any other format) itself to RAM without loading as SDL_Surface and unpack only when needed (SDL_Image has IMG_Load_RW for this).
Def has some kind of compression as well so this should work for it too. However such method may not be fast enough.

Anim groups are also needed for hero\boat animation so only difference between creatures anim and normal anim should be presence of compression.

I already removed extensions check for files, MSK\MSG can be replaced by file#msk to fit animation naming style.

Sometimes we have more than 15 creatures (14 normal creatures, + 6 war machines + 3 turrets + cloned / divided stacks + summoned elementals… game should work fine with even 25 - 30 creatures on the battlefield).

Just uploaded CAnimation on SVN. For now it is used only for buttons.
Class already can load image partially if needed or store them in compressed state which should be enough for now. Any propositions for future tweaking?
Will write a way to display creature animations with this class when I’ll get some more free time.

I’ve just checked it and creature animations seem way better ordered, or is it just my imagination?

@Ivan

Nice work, we needed such class for a long time. I think all CDefHandlers should be replaced with your new class (and probably creature animations too).

Your class obviously lacks some things needed for displaying creature animations (but not only) - such as “get next frame”, “get next frame from group” (that is tracking of currently displayed frame and automatic incrementation) and printing frame into given surface at given position.

@Waromnger

I think it’s just your imagination, as far as I can see Ivan didn’t touch creature animations in that commit :).

New class is intended to replace DefHandler and be used not only as animated images but for any set of pictures. This is why it don’t have getNextFrame or something like this.
Rewriting existing CreatureAnimation to use new class will be a nicer solution for me. Will try to do this on next weekend. Nice opportunity to check how compression works - haven’t tested it a lot myself.

So there should be a superclass that has those methods. And printing to any given surface is a must-have.

There’s an issue. Try starting campaign eg. “Long live the queen”. Graphics of bonuses are not present. They’re loaded in CBonusSelection::updateBonusSelection - pretty much happens there, something must be colliding with recent changes.

Otherwise great work :slight_smile:

Sorry. Missed that part during testing somehow. Should be fixed now.