CMake build system

Colleague of your has just commited a stub for a cmake build system :slight_smile: Lucky you have smart colleagues :stuck_out_tongue:

I’m using debian stable 6.0, boost 1.49 and gcc 4.6.3 - both compiled from source. Other libs are from the stable repository. Everything is compiling successfully but I’ve NOT tried to run client with all required files. Without required game and configuration files it looks promising:

[email protected]:~/projects/vcmi$ build/client/vcmi_client 
Starting... 
Creating console and logfile: 0
Failed to open file 
Last system error was : No such file or directory
Loading settings: 0
VCMI 0.89b (client)
Note: SDL suggests to use 32 bpp instead of24 bpp 
New screen flags: 0
	Initializing screen: 60
Cannot open ./Data/VIDEO.VID: failed opening file: No such file or directory
Cannot open ./Data/H3ab_ahd.vid: failed opening file: No such file or directory
	Initializing video: 0
	Initializing minors: 0
Cannot open ./Data/Heroes3.snd: failed opening file: No such file or directory
Cannot open ./Data/H3ab_ahd.snd: failed opening file: No such file or directory
	Initializing sound: 0
Initializing screen and sound handling: 0
Cannot open ./Data/H3sprite.lod
Cannot open ./Data/H3bitmap.lod
Cannot open ./Data/H3ab_bmp.lod
Loading .lod files: 0
Cannot find file: ZELP
Fatal error. Missing game file: ZELP.TXT. Aborting!
Killing console... done!

[email protected]:~/projects/vcmi$ build/server/vcmi_server 
Port 3030 will be used.
Cannot open ./Data/H3sprite.lod
Cannot open ./Data/H3bitmap.lod
Cannot open ./Data/H3ab_bmp.lod
Loading .lod files: 0
Cannot find file: ZELP
Fatal error. Missing game file: ZELP.TXT. Aborting!

Locally I’ve edited file GameConstants.h and hardcoded paths:

#ifdef _WIN32
		const std::string DATA_DIR = ".";
		const std::string BIN_DIR = ".";
		const std::string LIB_DIR = ".";
		const std::string SERVER_NAME = "VCMI_server.exe";
		const std::string LIB_EXT = "dll";
		const std::string PATH_SEPARATOR = "\";
	#else
		//#ifndef M_DATA_DIR
		//#error M_DATA_DIR undefined.
		//#else
		//const std::string DATA_DIR = M_DATA_DIR;
		//#endif
		//#ifndef M_BIN_DIR
		//#error M_BIN_DIR undefined.
		//#else
		//const std::string BIN_DIR = M_BIN_DIR;
		//#endif
		//#ifndef M_LIB_DIR
		//#error M_LIB_DIR undefined.
		//#else
		//const std::string LIB_DIR = M_LIB_DIR;
		//#endif
		const std::string DATA_DIR = ".";
                const std::string BIN_DIR = ".";
                const std::string LIB_DIR = ".";
		const std::string SERVER_NAME = "vcmiserver";
		const std::string LIB_EXT = "so";
		const std::string PATH_SEPARATOR = "/";
	#endif

It needs to be addressed with some more elegant solution.

Locally I’ve also edited file CVideoHandler.cpp and replaced the line 850 with the content from before Ivan’s patch:

#ifdef WITH_AVCODEC_DECODE_VIDEO2
                                avcodec_decode_video2(codecContext, frame, &frameFinished, &packet);
#else
                                avcodec_decode_video(codecContext, frame, &frameFinished,
                                                                         packet.data, packet.size);
#endif

I guess this is because debian stable got older version of the library, which doesn’t contain function avcodec_decode_video2().

CMake files are using default mechanisms for finding all libraries except ffmpeg, which is using a file “cmake_modules/FindFFMPEG_swscale.cmake”. This file may require some tweaks on other *nix systems.

With CMake compilation is like the following:

  mkdir build
  cd build
  cmake ../
  make

For multi-core processors I suggest using -j option to speedup building process eg. make -j 4. This option can be also be set in KDevelop for the project.

Lots of warnings is generated because I’ve turned on warnings by default. To disable them you can edit CMakeFiles.txt in the project root directory:

if(CMAKE_COMPILER_IS_GNUCXX)
    set(CMAKE_CXX_FLAGS "-std=c++0x -Wall -Wclobbered -Wempty-body -Wignored-qualifiers -Wmissing-field-initializers -Wsign-compare -Wtype-limits -Wuninitialized")
endif()

Looks good. Nice to finally have cmake!

Shouldn’t CMake set this variables one way or another?
“Elegant solution” should involve searching for data on startup and set all *_PATH vars correctly:

  1. check for any command line parameters
  2. check current directory
  3. check system location defined at compile time (/usr/share/vcmi or /usr/local/share/vcmi)

Probably good time to finally implement this.

Argh. There is just too much API changes in ffmpeg. I think all changes to video handler are just to make it work with version xx.yy…

Autotools configure script should set this define if you have this function somewhere. Probably better to replace this with versions check (like the rest of VideoHandler). I’ll try to find when they got this function.

Regarding ffmpeg compile issue - my bad I think. Should be fine in trunk now (please check on Debian).

Just ran cmake build:
Compilation:

  • Missing libraries. There are two libraries (librt and libdl) which should be only included on some platforms.
  • Clang. It does not work - all flags are set only if compiler is gcc. Maybe you should change this condition into “if not MSVC”? All other compilers usually support gcc options.
  • DATA_DIR defines. Check link to my CMakeLists. Not sure if there is any real need for BIN and LIB dirs so removing them completely may be better idea. Will look into it.
  • There should be some way to disable debug for “release” builds.
  • Warnings settings. I’ve copy-pasted them from configure script instead. This gives me almost warnings-free build.

My working CMakeLists.txt: pastebin.com/7B29Ni6g

Resulting binaries:
Not sure if this is a bug or feature but “make install” does not works (no “install” rule). So I can’t tell if they work or not for sure.
Naming format is a bit different (may cause issues). With autotools it looks like this:

/bin:
vcmiclient vcmiserver (no underscore, server name is hardcoded in GameConstants.h)

/lib/vcmi/:
libvcmi.so (no _lib suffix, should be safe I suppose)

/lib/vcmi/AI/:
libStupidAI.so (capital "S", must be changed)

Status update:
CMake is completely functional at least for development on non-Win (ffmpeg is mandatory right now), may need some tweaks for correct installation. Will implement this along with CPack support.

I also switched Linux readme to use CMake instead of autotools.

If there won’t be issues with CMake [size=117]I will completely remove autotools from trunk before next release[/size]

CPack support is now in trunk :slight_smile:
Some Windows-specific stuff is still missing (e.g. no dll’s in package) and obviously not tested. Can’t help much here.

On my side everything works (including Debian packages), resulting package also includes all data files (if they’re present)
To create package on Linux:

cmake PATH_TO_SOURCE # generate makefiles
make -j2             # compile (cpack can compile too but in one thread - too slow)
cpack                # create package (default for Linux is tar.gz, for Win - zip)

On Win + VS this may be quite different.

  • You can’t use makefiles without “make” (part of mingw\cygwin) - instead cmake is used to create VS project files
  • As result I don’t think that you can create binaries via cmake, only via VS
  • This leads to inability to use cpack - it may not know where to find binaries

I assume that I’m missing something - there has to be way to use CPack with VS.

Finally I got VCMI 0.9 built on Fedora with CMake! There occured some problems, but now it should be fine. I’ll post tomorrow what things I’ve changed to got a successful RPM package. I’ve commited to both tag 0.9 and trunk, because with the tag 0.9 it was not really possible to build a VCMI package.(not with CMake, not with gnu autotools because of moved icons,…)

…but 0.9 tag was not intented to build anything new, just to have unmodified release version for reference -.-

Do I have to use CMake to build VCMI (by makefile or by generating VS project) to be able to use CPack? Or would it be possible to just provide it with a location of binaries to copy or generate package without them (and manually put them later)?

That was buildsystem change that hopefully doesn’t affect the VCMI itself.

I assume yes.

From what I see this is how packaging should work:

  • CMake generates VS project files
  • VS compiles binaries
  • CPack creates package

It may be possible to place binaries where CPack expects them but this would diminish advantages of one-click packaging.

Tags are not just references for releases. They should be feature freeze, this is true. If a tag contains A-Bugs or cannot be built on some systems it should be ok to commit to tag and trunk in those cases. If you develop software commercially and not only there(e.g. firefox, linux os) then after a major release you may provide mostly bugfixes for a certain time. This requires committing to both repos. If the RPM .spec file is located in SVN it is required to update it always afterwards the tag has been created. (version update)

@Ivan:
I wanted to compile VCMI with GNU autotools, but the RPM packaging process couldn’t locate the icon for the vcmiclient.desktop file. Before it was added to /usr/share/icons/… automatically. Never mind I’ve chosen to build VCMI with the upcoming CMake system. Some points had to be done:
1.

FIND_PATH( FFMPEG_INCLUDE_DIR NAMES libavcodec/avcodec.h ffmpeg/avcodec.h 
 PATHS /usr/include /usr/local/include /usr/include/ffmpeg

FFMPEG headers weren’t found. They’re located in /usr/include/ffmpeg by default in Fedora. So I had to add this directory, this should be ok.

include(GNUInstallDirs) 
set(LIB_DIR "${CMAKE_INSTALL_LIBDIR}/vcmi" CACHE STRING "Where to install main library") 

Libdir is named /usr/lib64 on 64-bit linux OSes except on Debian/Ubuntu. The cmake variable CMAKE_INSTALL_LIBDIR handles this problem.

# remain full RPATH when installing (needed to find shared libraries) 
 	SET(CMAKE_INSTALL_RPATH "${CMAKE_INSTALL_PREFIX}/${CMAKE_INSTALL_LIBDIR}/vcmi") 
 	SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)

The rpath is needed to locate the shared library libvcmi. It is set after compiling to the build dirs. If you install the binaries with make install then the rpath will be erased by default. There are complicated rules how libs are found in linux. Most are user/machine dependent so this is not a good solution. I’ve investigated your debian binaries they hold those RPATHs. Do you call the make install routine or not? If not how do you change the RPATH to /usr/lib/vcmi? Nevertheless the problem should be fixed with above instructions.

At last I want to know why there is a core data package which is shipped separately? (some part of /mods/vcmi is in SVN…) Are there file size limits or anything else for SVN?

PS: CMake is great! Powerful, platform independent, fast, easy to use. I wonder why we haven’t had this earlier:)

Most of this is automated with debhelper but at some point this does results in make install + fakeroot.
This page describes how CMake handles RPATH (in case if you haven’t found it)
cmake.org/Wiki/CMake_RPATH_handling

No SVN limits I know about. Guys at Wesnoth store whole their content under VCS (svn and git). Probably one of old habits that started long time ago even before I joined this project.
There are may be some legal issues - at least some of them are edited graphics from H3. I can upload all of it to SVN but I’d rather wait for Tow opinion regarding this.

I tried and I just can’t get CMake to find my Boost installation. I tried setting BOOST_LIBRARYDIR, BOOST_ROOT, Boost_DIR, BOOST_INCLUDE_DIR — some of them and all of them… and it still fails. Whatever I assign to Boost_DIR it gets reset to “Boost_DIR-NOTFOUND” when trying to configure.

The error message is:

CMake Error at C:/Program Files (x86)/CMake 2.8/share/cmake-2.8/Modules/FindBoost.cmake:1191 (message):
  Unable to find the requested Boost libraries.

  Boost version: 1.51.0

  Boost include path: C:/C++/boost_1_51_0

  The following Boost libraries could not be found:

          boost_program_options
          boost_filesystem
          boost_system
          boost_thread

  No Boost libraries were found.  You may need to set BOOST_LIBRARYDIR to the
  directory containing Boost libraries or BOOST_ROOT to the location of
  Boost.
Call Stack (most recent call first):
  CMakeLists.txt:18 (find_package)

Any ideas?
Both BOOST_ROOT and BOOST_LIBRARYDIR are already set.

Once we decided on not uploading gfx resources to SVN. Reason were:
a) graphics that are derivative work could be potentially an object of copyright claims
b) graphics can be versioned but that uses much space (when compared to code). Once we had a repository on Assembla and there was a space limit, now it is not an issue.
c) we wanted to separate the “engine” stuff from the “game resources” stuff. Obviously we failed with all that configs but nevertheless…

I fixed the Boost issue by setting Boost_USE_STATIC_LIBS. CMake looks for libraries only when it is set.

Other issues when trying to build generated VS project:

  • VS generated solution does not have dependencies between project set (so that lib is built first and so on), leads to fails
  • very slow build, compiling only one file at the time (lacks /MP option)
  • no precompiled headers, that also slows things down
  • lib fails to build, need to define VCMI_DLL macro
  • lib fails to build, need to add /bigobj option

More info on progress tomorrow. :stuck_out_tongue:

Ok, this is true. But don’t know if we provide those graphics separately is better regarding copyright issues. We should provide “legal” graphics before vcmi 1.0 as this graphics package is mandatory and VCMI can’t be played without them.

But it makes things more complicated when installing :slight_smile: I noticed it when I wanted to extract the core package into the /usr/share/vcmi dir. It requires the command cp -p, mv command is not enough. With the -p argument it preserves the ownership of the files. This results to this bug: bugs.vcmi.eu/view.php?id=1142 (which shouldn’t happen I know…)

Not entirely true. You can play VCMI with no high-res, no creature window (+commanders), no stack queue. But most of those are enabled by default…

It should be possible to make this package optional but I don’t think that this is needed right now…

I don’t think that it’s even needed to make this optional. It’s ok for now to provide a separate core tar.gz archive file as long as legal issues aren’t fixed. (just wanted to know why there is a core archive file…)

Can someone explain to me how storing graphics files in archive on sever but not in version control system helps to avoid any copyright claims? Isn’t it just a way how you store this files?

stopiccot, it does not. The right direction would be to get rid of derivative graphics, not put graphics in SVN – especially since everything that is put into SVN is supposed to stay there forever.

But how we should get rid of AdvMap1024x768.pcx, … needed for running in high resolutions?
It is absolutely normal to have repository for project, not repository for project sources. Game art is also updated from time to time so it could also benifit from version history just like code files. And there is no problem to remove something from version control once it is not needed anymore.

We should generate such graphics algorithmically, at least some of them. Something more like what HDMod does.

By staying forever I didn’t mean staying in newest revision but staying in old revisions. Even though one could find a method to edit it, I would like to not have to do so or think someone else has to do so.