Compilation with Code::Blocks and MinGW

International board Development
#21

For MSVC it is about 100 MB. For some projects it requires passing a special option for encreasing memory limit.

But still, 300 MB should not be an issue. We have like gigabytes of memory available, even more with 64-bit. Is this a memory limit problem or some else compiler issue?

Yes, I can post them if you like.
Still, is there anything wrong with the old ones? (apart from their age) The MSVC libraries pack contains binaries both for 32-bit and 64-bit.

oO

Is there any particular reason why you want to have a 64-bit build of VCMI?
I once succeeded in builting 64-bit VCMI with Visual but it didn’t seem to work any faster. It just introduces more issues (incompatible with H3 video player DLLs, not portable on 32-bit systems, and so on).

Basically the whole point of PCH is gathering stable includes — that is things that rarely change, like STL, Boost or other libraries. Trying to strip PCH from rarely used stable headers is against its purpose.

As for the build times there are basically 3 constituents:

  1. processing includes (ton of text that gets after preprocessing the file)
  2. processing the actual file contents
  3. handling templates

Precompiled headers successfully solve issue #1. I can have tons of Boost and Qt headers included and still be able to compile file very fast — provided that source code is really small. However, as the file grows, factors #2 and #3 become more significant.
The #3 is probably worst. Every compile unit instantiates all used templates, so we end up compiling definitions of containers, smart pointers, etc dozens and dozens of times in every compilation unit.
I am very curious if that could be worked around by blocking template instantiations globally (extern template) and instantiating them explicitly in a single compilation unit.
Obviously, to have any sense and being scalable, such instantiations would have to be automatically generated. *

Btw, are there any particular reasons why CMake doesn’t use PCH?

There is one thing that puzzles me.
You say that you have precompiled headers turned off. This means that effectively you are compiling the contents of PCH for every compilation unit. [This is a burden, the question is how significant it is when compared to other building time factors.] Still, you manage to compile the whole project without problems in 2:30.

How is that you can compile a file that has PCH (by incldues) + file contents without issue but MinGW has trouble with the PCH itself?
Is this the MinGW-specific issue or sth related to the build system?*

#22

with old ones VCMI fails on assertion. (line 517 CMusicHandler.cpp). (It seems to work only with old *.lib but new dlls from recent dev build)

And server will fail to assemple too, only MSVC can compile such things using secret-undocumented extension to PE\COFF format. (this is not a joke)

There is no reason to use 64 bit for me except try to to support as much platforms as possible.

I disabled PCH and commited that change as I got compiler crash. Now I have new PC and tried to use PCH again, even 64 bit gcc with tons of available RAM crashes. So I think this is “size” issue not related to the amount of available memory, some internal limitations maybe.

#23

In my case it a bit more than 2,5 min but fast enough (8 core ADM FX with 16GB RAM). (ld is much slower while linking PE compared to ELF)

#24

Perhaps no one got the time to do it. =)

I added PCH compilation for CMake. It took me 1-2 hours to fix all linkage errors and compilation errors. The thing which makes it convenient and easy to use is the CMake module cotire. For example it doesn’t use PCH for targets with less than 3 compilation files like BattleAI or StupidAI.

PCH is disabled by default and can be enabled with ENABLE_PCH=ON. We can enable it by default when no one has any problems (not tested on GCC or MinGW).

On my system compilation time for a full rebuild (including generating PCH the first time) was reduced from 2:30 minutes to 1:40 minutes, that is 33%.

#25

Just tested PCH with gcc - seems to be OK, compile time went down by ~30% as well.

I think it is OK to enable cotire by default - at least for various gcc versions we have nicely automated daily builds on launchpad so if anything goes wrong I’ll be spammed with e-mails from launchpad:

code.launchpad.net/~vcmi/+recipe/vcmi-trunk

BTW - it is just me or peak memory usage became lower as well?

#26

Huh, strange.
The libraries pack that is currently online (built on Aug 2011 download.vcmi.eu/msvc-pack.7z ) should work well with all compilers both in 32 and 64-bit configuration and not trigger the assertion. The old libraries pack that is mentioned in the comment is the one that was used before that.
Do you get the assertion error only with 64 bit binaries from the package or both?

Thanks! I am sure that this effort will pay off.
I am glad to know that PCH actually works well with GCC/Clang (at least on Linux).

#27

Assertion with 32 bit, 64 bit crashes inside SDL, currently only 32 bit with new dlls works.

#28

Impossible. All the releases up till 0.94 were send out with the DLLs from the package I have linked. If the SDL_mixer from the package were faulty, it would have failed assertion for last few years.
What are the exact modification dates of the files you refer to as “old dll” and “new dll”?
The Aug 2011 SDL_mixer.dll should not fail on that assertion. The Jan 2014 SDL_mixer.dll should behave exactly the same. Everything older (like 2006 build from older packages) will fail.

I am going to upload the new libraries pack anyway (it’ll take a day or two, since I have it on another machine + need to finally sort out some dependencies issues) but the issue you describe sounds inexplicable to me.

#29

I have uploaded a new libraries pack, that I have used to build last development releases:

  • these are C libraries, so they should work across all popular compilers (including MSVC and MinGW). Both 32 and 64-bit binaries are provided. I have not tested 64-bit binaries.
  • pack was compiled with Visual 2013, so libraries depends on MSVC runtime (they need msvcp120.dll and msvcr120.dll — you can copy them from last dev package)
  • I believe the libraries does not require anything else. (if they fail to run due to a missing dependency, please report this)
  • you can extract additional DLLs (like libpng15-15.dll) that will be used as plugins by the library from pack. *

Link: download.vcmi.eu/msvc-pack.2014.1.7z*

#30

Thanks for updated libs.

Problem with assertion solved, but it was different problem - “redundant” SDL mixer plugin smpeg.

BTW. Found the bug report about gcc crash with large PCH gcc.gnu.org/bugzilla/show_bug.cgi?id=56926.

#31

Thanks for sharing this discovery. I wasn’t aware that SDL_mixer prefers using SMPEG plugin even when it was compiled with libmad.

Nice found. Still, it seems there is nothing we can do about it.

#32

32bit gcc + debug simbols = out of memory compiling registertypes.cpp o_O (again about 64 bit)

#33

registertypes.cpp compiles at the beginning, right? I had the same issue on 4%, vcmi’s compilation ate all memory.

#34

Even 64 bit gcc 492 just crushes with precompiled header enabled …

#35

Any tips for build vcmi with code::blocks? :blush:

#36

Are any particular problems or you are asking for complete guide?

#37

If you can give me a complete guide, i’ll be happy, but if you say what libraries are needed, or something like that it can help me too.

#38

Libraries: zlib, boost, ffmpeg, SDL2, SDL2_image, SDL2_mixer, SDL2_ttf
(also QT5 needed for launcher but there is no C::B project for launcher yet)

#39

Hmm I can’t build it -_-

C:\Users\Vampir\AppData\Local\Temp\ccoA7Llf.s: Assembler messages:
C:\Users\Vampir\AppData\Local\Temp\ccoA7Llf.s: Fatal error: can't write .objs\client\Client.o: File too big
c:/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/4.7.2/../../../../x86_64-w64-mingw32/bin/as.exe: .objs\client\Client.o: too many sections (61905)
C:\Users\Vampir\AppData\Local\Temp\ccoA7Llf.s: Fatal error: can't close .objs\client\Client.o: File too big
Process terminated with status 1 (0 minute(s), 43 second(s))
0 error(s), 6 warning(s) (0 minute(s), 43 second(s))
#40

Set “Settings-Compiler-Other settings-Compiler logging” to Full command line and post here several messages from Build log.