Please check my concept of new mods repository:
wiki.vcmi.eu/index.php?title=Mods_Repository
Keep in mind I just registered organization for to make more clear how it’s should work, but nothing else is done. Feel free to leave any ideas here.
Some implementation idea. Advantage of having repositories on GitHub would be ability to verify mods integrity with Travis, e.g:[ul]]Verify repository JSON integrity./:m]
]Verify JSON integrity of mod itself since Travis can clone mod from git./:m]
]Depend on what mod is changing we can run several vcmi tests too. At least make sure it’s won’t crash on loading./:m][/ul]
Your proposals sound good enough. And yeah, few persons should be choosen to ensure intergity of all vcmi mods (for cross testing to find all possible conflicts for example and alterative mods of given mod).
Also content folder should be in .zip form - it’s easier to update mods in this form when - for example - new mod version will be refractored.
And launcher should have few additional features. For me, better funcionality to “conflict” command should be implemented.
I not quite get what you mean, but it’s not good idea to store archives in Git and let’s say even WoG would need to be unpacked properly even if this would require additional overrides support in engine.
GitHub have 1GB size limit (include history) per repository which is sufficient for almost every H3 mod, but we’re better not to abuse it without a reason. Then launcher will always download mods as ZIPs so for end-user it’s doesn’t matter how exactly assets stored in Git.
Launcher really need a lot of work, but so far I not sure there anyone want to work on it.
From wiki.vcmi.eu/index.php?title=Modding_guidelines
All content of your mod should go into Content directory, e.g. Mods/myMod/Content/. In future it will be possible to replace this directory with single .zip archive. HotA mod is build like this.
If we going to store mods in Git then we would not compress anything in it and will need to actually unpack everything from mods like WoG. Compression would be handled by github when mod is downloaded.
Hi guys,
I’m following your projects for a long time and I just stumbled upon this thread. I am a developer at faforever.com (Supreme Commander Forged Alliance community).
We have a comparable issue with mod repository versioning. As a solution I developed a software called BiReUS. It creates binary patches using the bsdiff4 algorithm and is also capable of handling zip-files (extract, compare, generate patch --> extract, apply and zip again). It is developed in Python 3.x. The server part works like any unix cli tool generating the patches after you copied a new version into the repository folder. The files need to be serverd by an http server.
The client has a git-like interface but can also be embedded as a library. The client compares checksums before and after patching to ensure nothing went wrong.
We are not using it in our main client yet, because it did not made the transition to Python 3, so there is no productive experience yet, but I tested it succesfully on our cores repository (around 50 versions, patches between 200kb to 40mb depending on the changes between).
You can find it in my git repository under github.com/Brutus5000/BiReUS , there is also a basic Qt-based launcher application and a Java client library
If you are interested, but have additional requirements, I am sure we can find a solution for that as well.
Cheers
Brutus5000
Welcome. Thanks for the idea, but I don’t think we trying to solve similar problems since mods for FA can be large since they include heavy models and a lot of textures. So there can be actual reason to use binary diff.
In case of H3 largest mods are towns like HotA that take up to 50MB and they rarely change binary assets so we really can store them in git uncompressed. If GitHub limit wouldn’t be enough we can simply move some of them to GitLab since they allow 10GB repositories. And for the same reason download size isn’t issue for us too: mods just too small.
I think you should take a look at IPFS protocol:
ipfs.io/
youtube.com/watch?v=HUVmypx9HGI
It provides all the tools you will ever need for a good mod repository management
tl;dr:
IPFS = HTTPS + SFS + GIT + BITTORRENT + DHT
I like IPFS as concept, but that would mean anyone who want to install mods would need IPFS installed and this is no go. 
IPFS is not a software. IPFS is a protocol (same way as HTTP is protocol).
There are multilple implementations of this protocol in different languages.
Implementations are modular so they can be used standalone or as a part of any software.
So i see no reason why not use it in VCMI.
List of existing VCMI implemetations
Please note that currently only Javascript and Go implementations are mark as stable.
Terminology makes little difference because we still have to ship implementation that have to work on Windows, Linux, MacOS and Android.
Integrating it would require someone who actually know what he’s doing and my own experience in Go limited to writing easy API web server and some file management tools. So reason is super simple: we have no developer who have time to handle task like that.
And even if we would use something like IPFS for content delivery we still need to use git for contribution management and tracking of changes.
Here is git extension that adds full IPFS support for your command line git
github.com/larsks/git-remote-ipfs
I think the simplest solution is to recreate new Mod manager as a standalone web app written in entirely in JavaScript (HTML5 + CSS for GUI).
JavaScript should parse VCMI json files natively (as long as they are truly valid JSON) and IPFS has stable JavaScript implementation.
Tell me if such a idea appeals to you.
If yes, i can squeeze some of my free time to create some proof of concept app.
Problem is that people who going to develop mods or manage repository shouldn’t be required to use it. I see teaching them how to use Git and doable since there GUI tools and most of mod changes can be even made using GitHub website, but anything else is just way too hard.
Also plan is to not provide anyone access to our servers / keys to anyone at all so it’s should be possible to automate all interactions will IPFS.
I like IPFS as idea. Problem is that I want implementation where mod creators (and maintainers) as well as players would never have to even think how mods are distributed and should never ever have issues with it’s not working.
If you have time to explain how do you see it’s going to work I’ll appreciate if you could jump on our Slack:
slack.vcmi.eu/
another question, how to add other soils such as grass, snow, etc.?
how to create new ones, like: cobweb, gold, bricks, and others