AI Improvement

Lets discuss current AI state and how to improve it.

What problems current AI has as for me:

  • Not sure if it is problem but it buy no army nor it build dwellings in towns until it has Capitol. It might be ok for a big map but in case of small one player rashes to AI towns quickly and see no resistance.
    ** As for me sometimes it might be better to develop army and capture gold mines but might be good to do vise versa and by army in map dwellings.
  • Explore activity is somewhat secondary. If AI has nothing else to do it explores. Exploring also captures some nearby objects. It is not bad but a little bit surprising.
  • AI does not capture neutral towns sometimes. I feel like it is important?
  • No defence/self saving instincts.

Regarding to FuzzyLite, my oppinion. In general it is interesting thing. It requires to understand fuzzy logic theory so. Also as far as I see fuzzy for visittile now can only analyze

  • a distance to an object. The distance is calculated using CPathfinderHelper::getMovementCost(g.hero.h, g.tile). Does it give correct value? As far as I understood it gives cost of movement to a single tile but not all path cost.
  • army strength ratio
  • can distinguish if it is a town or not. This might be extended I think by adding reward value to all object types.
  • And missionImportance which is priority of hero locked goal.

Can we understand easily what will happen if we change something in fuzzy? Probably unit tests can help here. For a newcomer it looks very complex

Now about my rework at github.com/nullkiller/vcmi. First of all it was trial. It helped to understand some VCAI parts.

  • I wanted some simple hierarchical prioritizing schema. Probably I can combine it with Fuzzy and let it decide priority of CaptureObject tasks.
  • Separation of elementar and abstract goals. Goals analyze and issue tasks. Tasks do exact actions to achieve goals
  • Reduce VCAI state - if it is clever enough it can reconstruct state. But only if it is clever enough.
  • Improve town capture and make it more explicit

Now I suggest to search for problems in current AI. Some obviously bad behavior. Give me save file and I will investigate as soon as I have time and report what actually happens inside. This way we can understand where are our problems and how to fix them. I am ok to abandon my rework and start it from scratch.

Got a plenty of bugs from @Povelitel related to current VCAI. Will investigate them and publish results or fixes. Also I will publish as a separate request a few fixes to VCAI including dead end exploration cancellation.

Also a few questions:

  1. Guys, what do you think about exceptions in goals lie goalFulfilled and goalFulfilmentException? Should we avoid them or is it ok? Does it affect performance?
  2. How about my idea of Task/Goal separation? Do you like it or we better move on with elementar goals?
  3. There is a set of methods in VCAI like VCAI::wander. Should we move this logic to goals?

can distinguish if it is a town or not. This might be extended I think by adding reward value to all object types.

That was added by Dydzio recently. Yes, it shoudl be extended to all objects, as mentioned here: https://wiki.vcmi.eu/TODO_list#Adventure_AI

AI does not capture neutral towns sometimes. I feel like it is important?

… in order to improve town capture :stuck_out_tongue:

Not sure if it is problem but it buy no army nor it build dwellings in towns until it has Capitol. It might be ok for a big map but in case of small one player rashes to AI towns quickly and see no resistance.

This is (temporary) workaround as we lack proper build -> gather resources reasoning. Until this is implemented, this condition shouldn’t change.

How about my idea of Task/Goal separation?

I would like to see you explain how this part is supposed to work. In details. Because no, it’s not self-explanatory.

There is a set of methods in VCAI like VCAI::wander. Should we move this logic to goals?

Wander is what AI does when it has no specific goals left.

Goals only analyze gamestate and produce tasks. Tasks do actual things. Goals never do anything, only analyze. For now if build decides to build something nobody can stop it.
Prioritization is a different question. Now fuzzy handles only visit tile. All the rest priorities are hardcoded, right? I can use fuzzy in my captureobjects the same way. Related to my prioritization, goals form a tree. When a goal produces a task it assigns some priority 0…1 The parent goal also assigns priorities to child goals. When task pops up to the parent goal the priority is divided by 10 and the child goal’s priority is added. If the child goal has priority 0.7 and its task has 0.5 the final priority is 0.75 All the tasks produced by the child goal will be in range 0.7…0.8. Also I think to add scaling so that range can be any within 0…1 All the goals in the tree produce tasks and finally only one is taken and executed. Goals can have ranges overlapped so AI can capture resource if it is close and explore if all resources are far. Tasks and goals should be selfexecuting. This will reduce responsibility of vcai class. The parent goal sees how child ones will interfere and can control them.

Was it considered to move wandering to a goal. If not I am interesting why? What is goal in the current AI? Why build is not a subgoal of win? Does conquer touch neutral towns and mines?

Yes, it was the point of wander not being a goal is that it is never assigned to a hero or prioritized. So whatever goal AI has, it will decompose it before wandering.

Does conquer touch neutral towns and mines?

Yes it does. This is quite clear.

auto conquerable = [](const CGObjectInstance * obj) -> bool
	{
		if(cb->getPlayerRelations(ai->playerID, obj->tempOwner) == PlayerRelations::ENEMIES)
		{
			switch(obj->ID.num)
			{
			case Obj::TOWN:
			case Obj::HERO:
			case Obj::CREATURE_GENERATOR1:
			case Obj::MINE: //TODO: check ai->knownSubterraneanGates
				return true;
			}
		}
		return false;
	};

(Neutral player is also considered enemy here.)

Why build is not a subgoal of win?

Build is not a subgoal of win. Win is decomposed based solely on game rules. Game rules don’t tell that you win if you just build, unless there’s special win condition like build Grail.
The proper solution would be to make build a subgoal of Gather Army or Gather Resource, for instance, as this is according to game rules. Still, there are some buildings that don’t give you creatures or resources, and it’s nice to build them anyway - but not as a subgoal to Win.

Goals only analyze gamestate and produce tasks. Tasks do actual things. Goals never do anything, only analyze. For now if build decides to build something nobody can stop it.

This makes perfect sense to me, however how do you handle circular dependency? Gather army -> capture dwelling -> gather army or explore -> move to guarded tile -> gather army -> explore?

When a goal produces a task it assigns some priority 0…1 The parent goal also assigns priorities to child goals. When task pops up to the parent goal the priority is divided by 10 and the child goal’s priority is added. If the child goal has priority 0.7 and its task has 0.5 the final priority is 0.75 All the tasks produced by the child goal will be in range 0.7…0.8. Also I think to add scaling so that range can be any within 0…1

This is pretty interesting, however feels very arbitrary. Where do these values come from?

Now fuzzy handles only visit tile

Yes, however 99% of AI actions actually end up in visiting tiles. I would like to employ fuzzy logic more, but it may be tricky to evaluate completely different types of goals, such as build.

All the rest priorities are hardcoded, right?

As above, we would like to avoid hardcoded values whenever possible.

Why gather army should invoke explore? Explore is very important as for me, and it should be on top of goal tree. So circular dependency is more like a sign of invalid decomposition as for me. The first one is more interesting. Maybe capturing dwellings should not be a subgoal of gather army. We might introduce another goal on top of them which will capture dwellings, build dwellings and gather army.

I got this idea because it is a little bit hard to deal with fuzzy. Sometimes prioritization is quite obvious as for me. I want towns to be considered at a first place. Task priority can be calculated by fuzzy but ranges might be hardcoded. My first idea was to get all possible tasks and only afterwards do global prioritization. Might be a solution also. I feel like it will change anyway. We need to better understand our needs. Task/Goal separation might use existing prioritization. The main idea of it is separate analysis and action.

When wandering takes place in ideal AI gameplay? I watched 1 week of pro game, Povelitel gave me a youtube channel, and all he does is explore, find gold and fight for gold. He completely refuses to develop economics in town and still have enough gold to by 6 heroes or even more. It does not look like aimless wandering. Also he managed to gather enough sulfur and build leeches.

This depends on map. “find gold and fight for gold” works only if there is gold (including creature banks) on map.

In my rework I use number of towns + 1. And this sometimes seams too much. Also AI should have 7500 gold to be able to hire a hero. If there is not much to just take we do not need many heroes. If there are a lot interesting visitable objects we can hire more heroes. Maybe fuzzy can be used here with parameters of amount of gold, amount of visitable objects per hero, amount of towns per hero.

Why gather army should invoke explore?

Because if we don’t have any objects which give army, we need to find new ones.

Maybe capturing dwellings should not be a subgoal of gather army

Dwellings can provide army, can’t deny that. If AI finds itelf in a situation where there are no towns or allied heroes left, it must use dwellings. This is objective game rule, not your own preference.

I want towns to be considered at a first place

And why is that? What if there are no towns available?

Also AI should have 7500 gold to be able to hire a hero

An why not 7777 or 6666 gold?

Look, you replace all AI rules by your own ideas from the top of your head. It is none of “artifical intelligence”, it is your own set of rules which will at the very best play as good as you do in simple and predictable scenarios, but in reality it will underperform.
Can you finish every map following this rules exactly? If not, then you can’t expect AI doing so.

Should it actually finish maps? As for me it should just give enough resistance to keep human player interested in game. Is it possible to build what you want using fuzzy? I am not going to use neural networks. I am not going to build AI which is stronger than human or even equal to human. I just want enough resistance.

Capturing dwellings and visiting dwellings are different things. You do not need to explore in order to visit dwelling. At the same time you might need to find and capture new dwellings. The same way you might need to build dwellings. But it might be a decision of a little bit higher level. As for me there are two times of need army: urgent and regular. Regular is a desire to be more strong so when you analyze what you want to capture next you decide that you want to capture dwelling. Urgent - I need army to complete some task. It might be even like army request: can I get this amount of army right now? Btw can you explore in some special way to find dwelling?
What AI needs: to win. How AI can win?

  • AI needs army
  • AI needs to find enemy
  • AI needs to destroy enemy

How to destroy enemy?

  • kill heroes
  • capture towns

How to get army?

  • get resources
  • buy army

Where?

  • town
  • on map dwellings

and so on.

But do we really think this way?
I need to explore to reveal map and plan my actions more effectively
I need to explore to be prepared if enemy appears
I need to buy a lot heroes to gather things fast

Where is this in game rules? It is our experience. Neural network also needs experience. So decomposition should not follow game rules. It should follow experience, our experience. This is my point of view and I will follow it.

1 Like

In VCMI AI must be able to act as normal player by design. This is used at least in SXX`s test maps suite.

  1. Fulfil own win condition
  2. Fulfil other players loose condition (not yet implemented)

In some custom maps AI is trapped and can’t win at all. So it shouldn’t freeze in first place, but also should perform some sensible actions it can do.

@Warmonger, you said that capitol is just a workaround before we have good town development and resource managrment system. What do you mean under resource management?

It is me who implemented “capitol first”, because AI was heavily lacking money before so I went “full secure” in this regard. I had in in mind original H3 AI here which cheats by receiving 2000 more gold per day to function normally, so I used getting stable gold income as a base to make AI able to operate properly. In my opinion Warmonger’s idea about rushy “tournament style” AI is outside reality. It works because players exploit H3 AI algorithms by proper AI baiting, and I do not see why VCMI should follow that Battle AI design. It would work only if we set dumb AI to neutral units and different algorithm to AI player (it is like that at the moment but I do not see it as the only right way to make AI in VCMI worth anything).