AI Improvement

#33

There isn’t anything better than Wiki entry. The idea is in fact pretty simple.

The big picture I’m talking about is:

  • AI must be able handle any situation: singleplayer, multiplayer, campaign or random map, empty map or whatever extreme scenario one could imagine. Keep in mind that mapmakers come up with weird things.
  • Player goals in this game unfortunatelly create circular dependencies. This can’t be avoided. On the other hand, AI shouldn’t get stuck in infinite loops (even across multiple turns).
  • AI should be able to realize long-term goals without skipping quick opportunities. This is a big problem, for instance you may want AI to save and gather money for Capitol, but on the other hand you don’t want it to stall if it can’t afford Capitol at all.
  • AI can’t follow simple if-else condition, as this will be easily noticed and exploited by human opponent. Also, this will explore only certain conditions picked by AI designer but ignore the others. This is why fuzzy logic is used, as it can process multiple conditions simultaneously.

This is why AI design is very difficult. Now, you have TODO list on Wiki for what else AI should do to be complete in terms of gameplay.

#34

Ok, for now I stop work in my branch.

Danger hitmap will be good for detecting danger for towns and heroes. It will be good for implementing survival instinct. I really like this idea and thought about it myself. Also I think we can precalculate it for a few days to see danger early enough.

Regarding build in my branch I searched for each town the next building for economics and next dwelling. 6 - 7 day I also considered citadel/castle. I tried to enforce balance so I can afford all army grows. I use army = income/3 to have some money for building. What if I use balance army = income and some resource reservations? We can use balance state as a parameter to fuzzy. Kind of armyCost/income*7. If it is high we need more economics.

Can it be useful to have more army grows than you can afford? Probably yes because we can develop economy later and only if we have money to develop town. We can try to use fuzzy to decide what to build dwelling or economy building. We can use income, army grows, and current amount of gold. What else? Maybe alternative gold sources as not captured mines? Capturing mine is equivalent to building city hall but usually more cheap. Also some army pressure might shift as to build dwelling. Small area and strong desire to explore sectors blocked by monsters/enemy? Garrisons?

Guys, do you always use all 7 levels of creatures in town? Is there any sense to skip some?

Also I think that towns are not absolutely equal and some are better for army development than others. I try to calculate overall town development cost needed to build all dwellings. Also it might be worth to consider strength of creatures and disabled buildings and amount of towns with such alignment.

Any other ideas?

What do you think about AIValue? Could we trust it? Or is it also considered as a hardcode and should be calculated dunamically?

#35

What point in hardcoding them in any case?

In good proper building algorithm dwellings should be evaluated exactly same way as other buildings. E.g as you said yourself in Building city hall vs capturing gold mine. Obviously AI shouldn’t strive to build dwellings if it’s don’t have budget to purchase creatures from it.

I’m hate this AIValue thing and in long run it’s should be replaced. Yet it’s not as important as it’s will be only required for modded game.

#36

Simple, I need to understand that this is a dwelling or its upgrade so I can extract its creature and evaluate them. I think I can add army score and economics score for each building in town and use them together with amount of gold, income and so on. I just take all possible buildings with all these parameters and feed fuzzy with them so it can answer which is more important now? and rules like if armyCostRatio is high and economicsScore is high than value is high. armyCostRatio = cost of all grows/weekly income. I do not know exactly how fuzzy works so I have to understand it first. Same for citadel, its army score is 50% of current grows. Still some things do not fit in this schema, some walls of knowledge and so on. They have to be considered separately.
Also I need to track down dependencies like I need to build mage guild in order to build some dwelling so mage gild also receives army score.

I do not suggest to hardcode which level to skip, I suggest to analyze if we should skip some particular unit because we do not have enough gold income but can build/already built a dwelling with some better units. I expect that damage per each gold coin is less for higher level units also they usually have better speed and skills. Fact of skipping does not tell us to not build dwelling but we can avoid counting it in army cost ratio and avoid buying them until we have no other chose.

#37

Lets go back to wandering. I have an empty map. It has just a few things on it. AI finds a mine which it can’t get and just wants to capture it badly. So it stops exploring. And there is no visitable objects so wandering do nothing. They just stand and do nothing. Wait until there is some army in town. What I think should be done:

  • wandering should be able to explore
  • wandering should prioritize objects to get because now it just takes nearest one
  • wandering logic should understand when it is better to explore and when to take objects
  • wandering should consider if current hero is the nearest one to an object it wants to get
  • conquer should understand when it is more important to explore rather than gather army
  • gather army could do nothing if it understands that it can’t get enough army anyway. Do not try to get half of it.
#38

Lets go back to wandering. I have an empty map. It has just a few things on it. AI finds a mine which it can’t get and just wants to capture it badly. So it stops exploring. And there is no visitable objects so wandering do nothing. They just stand and do nothing. Wait until there is some army in town.

Could you please upload that map in question? AI shouldn’t behave that way to my knowledge.

#39

In map list it is called a test map AI is not frozen completely, it waits capitol and army. It starts moving on 3rd week or something and captures that mine. Also there are a lot logs saying that not all mp are spent. I suspect problem with goal switching. AI can’t switch from ClearWayTo to Explore on the same turn.
Test.h3m (9.8 KB)
Saves.zip (495.1 KB)

1 Like
#40

Also there is a problem when AI wants to take a thing on the adjacent tile. We use movement logic which checks if AI hero has actually moved and fails if not. But AI does not need to move because his path contains only one node with BLOCKVIS. It just takes the thing without move but logic crashes and wandering exits for this hero so he has some movement points left. I will address this issue on a separate PR. You can see the issue very often when there is a mine and a resource near it. AI first captures the mine and then the resource and then the hero is skipped even if he still has movement points…

1 Like
#41

AFAIK there is no goal switching at all. Each turn goals being evaluated and only one per hero selected to achieve. After that only “wandering”.

#42

I thought a lot I do not want to develop current AI. I do not like GOAP because I do not see for now how it will help. I do not want hierarchical goal decomposition at all but I want goals to affect prioritizing engine instead. Fuzzy Logic looks like a great instrument for prioritization, so I will use it definitely. So I will continue development in my own branch nullkiller/vcmi/ai-rework.
For now I want 4 stages

  • Analysis - get data from game state and represent it in more convenient way
  • Task generation - generate all possible tasks. Here might be some logic to avoid tasks which can not be executed at all. Tasks can consist from other tasks and each such composite task should be executed only as an atomic thing.
  • Task prioritization - use data from analysis stage to properly prioritize tasks.
  • Task execution - get top priority task and execute it.

The branch with exploration improvement can be merged to develop if there is no objections. I will merge it to my branch anyway.

My decision is mostly driven by a desire to build a bicycle so no offence. I will do what I like quietly in my branch and come back if I will see the result. Still I am open to any ideas.

#43

It’s true that the AI needs major improvements because it’s sad to play with an AI that is controlling heroes without armies. Every time I want to capture a town, I just end up cleaning every CPU’s armies just to see them buying ~3 heroes without any troops or taking the castle with too few creatures. Also, they’re just spending their times with those troopless heroes and taking mines and stuff without any “challenge” nor fun at all. That AI simply have an annoying behavior.

Also, I don’t know if it’s a bug or something but CPU without castle wander until their banishment instead of attacking the nearby castle as original H3 CPU used to. They sometimes visit the same map objects several times in the same turn. Sometimes their turns are abnormally long and/or they’re doing pause between each steps.

PS: I know the last reply was 1-year ago but since there isn’t that much activity on this forum, this can only be beneficial. Thanks for reading.

#44

I work on AI in a separate branch unofficially. The progress is quite slow so. Regarding AI without towns left to tell the truth it is already the end. It hardly can win if you took his last town. There are much more important problems.

#45

Well, if the AI was building armies instead of buying troopless heroes, it might have a chance of recapturing a town. Sometimes, they don’t do anything because of AI Value while they technically should win the fight if they engage it.

What kind of tools are used in the H3AI creation process? I didn’t find any documentation about that and I’m curious to know more about.

#46

Actually I am not sure that H3 original AI has a rule to capture town if has none. I remember that I wanted a few times to force AI recapture its town back so that I do not win and have some time to explore map and it failed to do this.

Regarding the tools I do not know anything about original AI tools.

#47

It’s funny 'cause I remember the opposite: having good garrisons in town and getting attacked while the enemy was going to be banished. Though, I might’ve made a mistake if it was on WoG. I might do few tests someday and give you the results.

#48

Maybe you are right. I also remember such behavior when AI attacks the last lost town and dies. I might mistake it with VCMI. VCAI definitely does not have such rule and will continue wandering even without towns. And yes it loves to buy heroes. And it has no fear and defense. My AI at least has some logic around defense and will behave a bit better. But it also has no rule regarding no towns. If you want to try out my AI I can provide you a build at weekends. But it will definitely be less stable and much slower.

1 Like
#49

Yes, I’d like to try it !

#50

https://builds.vcmi.download/branch/nullkiller_ai/Windows/?C=N&O=A
Builds from my branch. Finally it sucseeded.
Also latest build from app veyor https://ci.appveyor.com/project/vcmi/vcmi/builds/28345470/job/rsbq7ydj980dd0jy/artifacts
Not sure if appveyor build works but if it does it is usually faster on windows

#51

Map - Ascesion. AI durn. Day - 1.

image
image.png1920×1080 267 KB

#52

Yeah. A lot of bugs. Needs stabilization. It can take months to acquire more or less stable build for beta testing. And I became lazy.