I am working on project in which I need to find best optimised path from 1 point to another in continuous space in multi agent scenario. I am looking for best algorithm which suits this problem using Reinforcement learning. I have tried "Multi-agent actor-critic for mixed cooperative-competitive environment" but it does not seems to reach goals in 10000 epesidoes. How can I improve this algorithm or is there any other algorithm that can help me with this.
Multi-agent reinforcement learning is quite hard to master and has yet to prove effective for general cases.
The problem is that in multi-agent the environment becomes non-stationary from the perspective of each individual agent. This means that an agents action cannot be mapped to the state directly because other agents are performing action seperately, which "confuse" all of the agents. There is an in-depth collection of multi-agent research here: https://github.com/LantaoYu/MARL-Papers
If you would like you to pursue the actor-critic method you mentioned, I recommend this for you further research: https://arxiv.org/pdf/1706.02275.pdf if you would like to perfect Multi-Agent Actor Critic (MADDPG)