Trunk discussion / complains

I feel the same as “worse” default AI always cause misunderstanding.
Though just wonder if there any particular reason why that wasn’t done yet.

Hi. I found ugly data race in AI code which gives me almost 100% crash in a few days of game.

Thread A:

==6362==    by 0x31DB6BBA: SectorMap::retreiveTile(int3 const&) (VCAI.cpp:3451)
==6362==    by 0x31DB55EF: SectorMap::firstTileToGet(HeroPtr, int3 const&) (VCAI.cpp:3191)
==6362==    by 0x31DA52A4: VCAI::isGoodForVisit(CGObjectInstance const*, HeroPtr, SectorMap&) (VCAI.cpp:1364)
==6362==    by 0x31DA592F: VCAI::getPossibleDestinations(HeroPtr) (VCAI.cpp:1395)
==6362==    by 0x31DA65C7: VCAI::wander(HeroPtr) (VCAI.cpp:1461)
==6362==    by 0x31DAFC74: VCAI::performTypicalActions() (VCAI.cpp:2454)
==6362==    by 0x31DA1431: VCAI::makeTurnInternal() (VCAI.cpp:829)
==6362==    by 0x31DA0941: VCAI::makeTurn() (VCAI.cpp:741)

Thread B:

==6362==    by 0x31DCD90B: std::map<HeroPtr, SectorMap, std::less<HeroPtr>, std::allocator<std::pair<HeroPtr const, SectorMap> > >::clear() (stl_map.h:810)
==6362==    by 0x31D99ACD: VCAI::newObject(CGObjectInstance const*) (VCAI.cpp:388)
==6362==    by 0xACEF70: NewObject::applyCl(CClient*) (NetPacksClient.cpp:927)
==6362==    by 0x91443A: CApplyOnCL<NewObject>::applyOnClAfter(CClient*, void*) const (Client.cpp:78)
==6362==    by 0x83AF5F: CClient::handlePack(CPack*) (Client.cpp:654)

Thread A obtains SectorMap for hero and attempts to use it
Thread B receives NetPack and cleans SectorMap’s used by Thread A.

Result - access to deallocated memory in Thread A. Can somebody more familiar with AI fix this or at least explain how this should work?

UPD: I could try to use shared_ptr here so used SectorMap won’t be deallocated on clear() but I’m not sure if hero movement during net pack handling is correct behaviour. Missing locks?
UPD2: shared_ptr seems to be working OK ( as in “no crash”) but I’m still not sure that this is the “right” way.
UPD3: Found another (probably old one) crash in AI, will commit fix soon.

And now I have edless spam from AI “Another allied hero stands in our way”, “object is not visible” or even “Hero Aenain tries to visit himself”.

They shouldn’t be in different threads to begin with. In fact I don’t know how threads in AI were supposed to work or be synchronized.

However, a question: how hero movement can cause creation of new object?

And yet current design allows AI to use multiple thread:

  • AI thread
  • Event handling thread
  • threads created by requestActionASAP

Perhaps this was caused by new hero recruitment? It can’t be tavern (map object) - this bug happens a bit too often for that. Perhaps AI switched to movement AFTER performing action that created new object? E.g. recruit hero and then try to move already existing hero, without waiting for reply from server.

Can’t say for sure - bug was very random and hard to track. I only was able to detect its source via analyzer. Which runs VCMI at 1% speed due to CPU emulation :frowning:

The first two are required if AI can receive messages from server. The last one was added by Tow just to overcome limitations of receiver thread.

However, AI was not supposed to be interrupted in a middle of perfomed action. Especially when it’s the only active player and server is passive. Probably the only solution is to keep AI action under lock and process messages after AI is done.

So i’m really sorry, but I somehow manager to fuck up with serialization. I tested everything in my branches or at least I was sure that I’d tested everything and backward compatibility is okay. Unfortunately problem has come from the side I would never expect…

For some reason saves created with current develop code fail to load, trying to find out why… :frowning:

Most likely issue caused by that branch:
github.com/vcmi/vcmi/pull/144/files
Sadly I’m not 100% sure. :neutral_face:

So it’s end up problem is related to “destinationTeleportPos” serialization.
For some reason int3 don’t serialize / unserialize properly within VCAI and I don’t know why.

Fortunately this isn’t critical at all and won’t cause any issues because destinationTeleportPos can only be set when destinationTeleport (ObjectInstanceID) is set. It’s unlikely that anyone save game during AI turn and even in that case only problem is uninitialized value.

Now all save/loading should be okay.

I tried to compile VCMI after longer break and there are some significant issues with CTypeList:

Error	2	error C2440: 'return' : cannot convert from 'CGTownInstance *' to 'CGObjectInstance *'	d:\vcmi\vcmi\lib\mapobjects\CommonConstructors.h	44
Error	20	error LNK1120: 6 unresolved externals	D:\VCMI\vcmi\AI\VCAI.dll
Error	13	error LNK1120: 7 unresolved externals	D:\VCMI\vcmi\VCMI_server.exe
Error	29	error LNK1120: 8 unresolved externals	D:\VCMI\vcmi\VCMI_client.exe
Error	9	error LNK2001: unresolved external symbol "__declspec(dllimport) private: class std::shared_ptr<struct CTypeList::TypeDescriptor> __thiscall CTypeList::registerType(class type_info const *)" (__imp_?registerType@CTypeList@@AAE?AV?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@PBVtype_info@@@Z)	D:\VCMI\vcmi\server\CGameHandler.obj
Error	16	error LNK2001: unresolved external symbol "__declspec(dllimport) private: class std::shared_ptr<struct CTypeList::TypeDescriptor> __thiscall CTypeList::registerType(class type_info const *)" (__imp_?registerType@CTypeList@@AAE?AV?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@PBVtype_info@@@Z)	D:\VCMI\vcmi\AI\VCAI\VCAI.obj
Error	23	error LNK2001: unresolved external symbol "__declspec(dllimport) private: class std::shared_ptr<struct CTypeList::TypeDescriptor> __thiscall CTypeList::registerType(class type_info const *)" (__imp_?registerType@CTypeList@@AAE?AV?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@PBVtype_info@@@Z)	D:\VCMI\vcmi\client\Client.obj
Error	7	error LNK2001: unresolved external symbol "__declspec(dllimport) private: class std::vector<class std::shared_ptr<struct CTypeList::TypeDescriptor>,class std::allocator<class std::shared_ptr<struct CTypeList::TypeDescriptor> > > __thiscall CTypeList::castSequence(class type_info const *,class type_info const *)const " (__imp_?castSequence@CTypeList@@ABE?AV?$vector@V?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@V?$allocator@V?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@@2@@std@@PBVtype_info@@0@Z)	D:\VCMI\vcmi\server\CGameHandler.obj
Error	14	error LNK2001: unresolved external symbol "__declspec(dllimport) private: class std::vector<class std::shared_ptr<struct CTypeList::TypeDescriptor>,class std::allocator<class std::shared_ptr<struct CTypeList::TypeDescriptor> > > __thiscall CTypeList::castSequence(class type_info const *,class type_info const *)const " (__imp_?castSequence@CTypeList@@ABE?AV?$vector@V?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@V?$allocator@V?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@@2@@std@@PBVtype_info@@0@Z)	D:\VCMI\vcmi\AI\VCAI\VCAI.obj
Error	21	error LNK2001: unresolved external symbol "__declspec(dllimport) private: class std::vector<class std::shared_ptr<struct CTypeList::TypeDescriptor>,class std::allocator<class std::shared_ptr<struct CTypeList::TypeDescriptor> > > __thiscall CTypeList::castSequence(class type_info const *,class type_info const *)const " (__imp_?castSequence@CTypeList@@ABE?AV?$vector@V?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@V?$allocator@V?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@@2@@std@@PBVtype_info@@0@Z)	D:\VCMI\vcmi\client\Client.obj
Error	11	error LNK2001: unresolved external symbol "__declspec(dllimport) private: virtual void __thiscall CGDwelling::blockingDialogAnswered(class CGHeroInstance const *,unsigned int)const " (__imp_?blockingDialogAnswered@CGDwelling@@EBEXPBVCGHeroInstance@@I@Z)	D:\VCMI\vcmi\server\CVCMIServer.obj
Error	17	error LNK2001: unresolved external symbol "__declspec(dllimport) private: virtual void __thiscall CGDwelling::blockingDialogAnswered(class CGHeroInstance const *,unsigned int)const " (__imp_?blockingDialogAnswered@CGDwelling@@EBEXPBVCGHeroInstance@@I@Z)	D:\VCMI\vcmi\AI\VCAI\VCAI.obj
Error	24	error LNK2001: unresolved external symbol "__declspec(dllimport) private: virtual void __thiscall CGDwelling::blockingDialogAnswered(class CGHeroInstance const *,unsigned int)const " (__imp_?blockingDialogAnswered@CGDwelling@@EBEXPBVCGHeroInstance@@I@Z)	D:\VCMI\vcmi\client\Client.obj
Error	27	error LNK2001: unresolved external symbol "__declspec(dllimport) public: bool __thiscall TerrainTile::hasFavorableWinds(void)const " (__imp_?hasFavorableWinds@TerrainTile@@QBE_NXZ)	D:\VCMI\vcmi\client\mapHandler.obj
Error	12	error LNK2001: unresolved external symbol "__declspec(dllimport) public: class boost::any __thiscall CTypeList::castShared(class boost::any,class type_info const *,class type_info const *)const " (__imp_?castShared@CTypeList@@QBE?AVany@boost@@V23@PBVtype_info@@1@Z)	D:\VCMI\vcmi\server\CVCMIServer.obj
Error	18	error LNK2001: unresolved external symbol "__declspec(dllimport) public: class boost::any __thiscall CTypeList::castShared(class boost::any,class type_info const *,class type_info const *)const " (__imp_?castShared@CTypeList@@QBE?AVany@boost@@V23@PBVtype_info@@1@Z)	D:\VCMI\vcmi\AI\VCAI\VCAI.obj
Error	25	error LNK2001: unresolved external symbol "__declspec(dllimport) public: class boost::any __thiscall CTypeList::castShared(class boost::any,class type_info const *,class type_info const *)const " (__imp_?castShared@CTypeList@@QBE?AVany@boost@@V23@PBVtype_info@@1@Z)	D:\VCMI\vcmi\client\Client.obj
Error	28	error LNK2001: unresolved external symbol "__declspec(dllimport) public: class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __thiscall CGameInfoCallback::getTavernRumor(class CGObjectInstance const *)const " (__imp_?getTavernRumor@CGameInfoCallback@@QBE?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@PBVCGObjectInstance@@@Z)	D:\VCMI\vcmi\client\GUIClasses.obj
Error	8	error LNK2001: unresolved external symbol "__declspec(dllimport) public: unsigned short __thiscall CTypeList::getTypeID(class type_info const *,bool)const " (__imp_?getTypeID@CTypeList@@QBEGPBVtype_info@@_N@Z)	D:\VCMI\vcmi\server\CGameHandler.obj
Error	15	error LNK2001: unresolved external symbol "__declspec(dllimport) public: unsigned short __thiscall CTypeList::getTypeID(class type_info const *,bool)const " (__imp_?getTypeID@CTypeList@@QBEGPBVtype_info@@_N@Z)	D:\VCMI\vcmi\AI\VCAI\VCAI.obj
Error	22	error LNK2001: unresolved external symbol "__declspec(dllimport) public: unsigned short __thiscall CTypeList::getTypeID(class type_info const *,bool)const " (__imp_?getTypeID@CTypeList@@QBEGPBVtype_info@@_N@Z)	D:\VCMI\vcmi\client\Client.obj
Error	10	error LNK2001: unresolved external symbol "__declspec(dllimport) public: void * __thiscall CTypeList::castRaw(void *,class type_info const *,class type_info const *)const " (__imp_?castRaw@CTypeList@@QBEPAXPAXPBVtype_info@@1@Z)	D:\VCMI\vcmi\server\CGameHandler.obj
Error	19	error LNK2001: unresolved external symbol "__declspec(dllimport) public: void * __thiscall CTypeList::castRaw(void *,class type_info const *,class type_info const *)const " (__imp_?castRaw@CTypeList@@QBEPAXPAXPBVtype_info@@1@Z)	D:\VCMI\vcmi\AI\VCAI\VCAI.obj
Error	26	error LNK2001: unresolved external symbol "__declspec(dllimport) public: void * __thiscall CTypeList::castRaw(void *,class type_info const *,class type_info const *)const " (__imp_?castRaw@CTypeList@@QBEPAXPAXPBVtype_info@@1@Z)	D:\VCMI\vcmi\client\Client.obj
Error	6	error LNK2001: unresolved external symbol "__declspec(dllimport) public: void __thiscall CPrivilagedInfoCallback::getTilesInRange(class std::unordered_set<class int3,struct ShashInt3,struct std::equal_to<class int3>,class std::allocator<class int3> > &,class int3,int,class boost::optional<class PlayerColor>,int,bool)const " (__imp_?getTilesInRange@CPrivilagedInfoCallback@@QBEXAAV?$unordered_set@Vint3@@UShashInt3@@U?$equal_to@Vint3@@@std@@V?$allocator@Vint3@@@4@@std@@Vint3@@HV?$optional@VPlayerColor@@@boost@@H_N@Z)	D:\VCMI\vcmi\server\CGameHandler.obj

Probably missing include(s).

cannot convert from ‘CGTownInstance *’ to ‘CGObjectInstance *’ = missing includes (need both complete types)

Error    9    error LNK2001: unresolved external symbol "__declspec(dllimport) private: class std::shared_ptr<struct CTypeList::TypeDescriptor> __thiscall CTypeList::registerType(class type_info const *)" (__imp_?registerType@CTypeList@@AAE?AV?$shared_ptr@UTypeDescriptor@CTypeList@@@std@@PBVtype_info@@@Z)    D:\VCMI\vcmi\server\CGameHandler.obj 

This one (and similar) is weird - please try full rebuild.

Once I fixed 1st error, everything linked correctly. Thanks.

Please do not use constexpr. My MSVS 2013 doesn’t support it, besides in current examples it stands just for simple const keyword.

fixed

I can’t link Filestream.cpp. The line

fill_fopen64_filefunc(&MinizipFilefunc);

Seem to use function from other project minizip.

However, when I add MINIZIP_EXPORT to fill_fopen64_filefunc definition, I get crash at launch - no such procedure found in minizip.dll.

How is that supposed to be linked together?

Look like fill_fopen64_filefunc should have also “extern” modifier. (No idea how it work in my case.)

Didn’t work :frowning:

Also, why do we use 64-bit (?) function in first place when there’s alternative? VCMI is 32-bit application.

64bit zip is nothing about 64bit OS. Actually minizip will use “64bit” functions internally anyway.

Then do hardcore :slight_smile: Just copy ioapi functions from ioapi.c to FileStream.cpp. FileIO will be rewritten in MapFormat branch anyway.

Warmonger, lets make develop branch history linear. Please do not merge upstream develop to your local develop use pull fast-forward only or rebase local changes.

What about merging develop into branches?