Chinese Academy of Sciences trains macaques to play Pac-Man game

IT House news on May 27, the Chinese Academy of Sciences released news, recently, eLife online published an article entitled “Layered Combination Strategy of Macaques in Pac-Man Game》research paper. The research was completed by Yang Tianming’s research group from the Center for Excellence in Brain Science and Intelligent Technology (Institute of Neuroscience) of the Chinese Academy of Sciences and the State Key Laboratory of Primate Neurobiology.

The study designed a novel and interesting experimental paradigm – Pac-Man game, and train the macaques to use the joystick to complete the main tasks of this game paradigm. This study combines complex behavioral paradigms and artificial intelligence modeling to quantitatively explore the characteristics of heuristic behavioral strategies of macaques to solve complex problems.Provides a new method and important inspiration for explaining the computational mechanism of the brain to achieve advanced cognitive functions.

According to reports, in daily life, most of the important goals of people are usually beyond the scope of simple decision-making, and these goals can be achieved by designing a series of meticulous combinations of basic strategies. Individuals can prioritize the benefits and risks of each strategy according to the current situation, and analyze specific problems in subtasks that are easier to complete.

In a highly dynamic environment with unexpected surprises and distractions, maintaining camera flexibility is critical in the decision-making process. Although studying the complex behavior and underlying neural mechanisms of animals is a continuing scientific concern in the fields of neuroscience and cognitive science, most animal behavior paradigms are not sufficiently complex to support the study of how animals simplify dynamic and diverse strategies to Complete complex high-level cognitive tasks.

To address these issues, the study adapted the classic arcade game Pac-Man (Figure A),And train the macaques to learn to use the joystick to control the Pac-Man movement in a closed maze to collect food, the macaques will get real-time juice as a reward for avoiding the enemy’s pursuit. After a period of training, the macaques can understand the relationship between the various elements in the game and the rewards and punishments, and make continuous movement choices accordingly to avoid the pursuit of the enemy, get more rewards, and even under certain rules. Counter-kill the enemy. Although the game is highly dynamic and has complex elements, it is essentially similar to an animal foraging task in the wild, which may be a key element of the successful training of the animals in this study.

In order to quantitatively describe the characteristics of macaque behavioral strategies, researchers used machine learning and statistical methods to dynamically fit and match game play and multiple intelligent strategy models. This multi-agent collaborative decision-making model is also the design key to achieve the highest score in the Pac-Man game in the field of artificial intelligence. The computational model designs a set of strategy basis groups, each strategy in the strategy basis group only solves one sub-task in the game, such as foraging for the nearest food, avoiding the enemy’s pursuit, or changing the enemy’s state through energy bean food.

The model compared different strategy basis sets to the game behavior data of macaques to infer the dynamic weights of the strategies.The computational model has an accuracy of more than 90% in predicting the movement of the handle of the macaque. More importantly, the policy dynamic weight analysis found that the macaques solved these problems distributedly by using a divide-and-conquer heuristic method, focusing on only one subtask of the game at each time, thereby achieving the optimization of the overall goal of the game. The study found that macaques were able to temporally combine these strategy basis sets to construct more complex composite strategies to tackle specific and more challenging tasks. research shows,Macaques are able to optimally master a set of policy bases and employ hierarchical decision-making to solve complex tasks(Panel B).

This study looks at the intersection of systems cognitive neuroscience and artificial intelligence with broad interest, combining complex behavioral paradigms with rigorous computational modeling,Provides important experimental evidence and novel analytical methods for future exploration of higher cognition in primates. IT House learned that the research work was supported by the Ministry of Science and Technology, the Chinese Academy of Sciences, the Shanghai Municipal Science and Technology Commission and the National Natural Science Foundation of China.