At the Schwartz Reisman Institute for Technology and Society’s Absolutely Interdisciplinary Conference this June, Nicolas Papernot — an assistant professor at U of T — moderated a session titled ‘Testing social cognitive theory with AI.’ The two presenters at the session were William Cunningham, a psychology professor at U of T, and Joel Z. Leibo, a senior staff research scientist at Google DeepMind.
In their discussion, Cunningham and Leibo explored how multi-agent reinforcement learning (MARL) can test social cognitive theory. According to the social cognitive theory, learning occurs within a social setting, where individuals, their behaviours, and the environment interact and influence each other. Using reinforcement learning models, we can create simulations of agents and model their behaviours. This approach can help us gain valuable insights into the emergence of human social processes, offering valuable implications for aligning AI with human values.
Reinforcement Learning (RL) is a type of machine learning method where an AI agent learns by trying things out in an interactive environment, using feedback from the results of its actions and its experiences. Essentially, RL allows a computer to figure things out on its own without being told exactly what to do. Instead, it learns by getting rewards for ‘good’ behaviours and penalties for ‘bad’ behaviours. You treat the computer, or “agent,” when it does something correctly, and it learns to do more of it. Vice versa, you punish the agent when it does something incorrectly, and it learns to do less of it. It’s like training a pet to do tricks.
Historically, social psychology has made an effort to pinpoint the core elements of social cognition by taking a ‘domain-general’ approach — that is to say, by believing that social phenomena can be explained solely through basic mental processes, which include perception, memory, attribution, and attention. However, according to the speakers, this simplistic approach was inadequate for explaining social behaviour in real-world contexts. It failed to take into account external influences. Social psychology now attempts to incorporate many motivational factors, such as self-esteem, group dominance, and inequality aversion, into its models explaining social behaviour.
This is where AI comes into play. Cunningham and Leibo’s research used MARL models, a form of AI, as a “null model” for social cognition: a model with as few assumptions as possible about how the MARL models would behave. This allows them to focus on reinforcement learning. They aimed to explore the extent to which they could explain human social behaviour without including additional motivational factors.
The researchers designed a coordination game that involved two different groups of ‘sprites’ — as in AI agents — interacting and collecting objects. Some conditions allowed sprites to interact only with members of their own group, while others allowed interactions with members of both groups. Over time, the sprites began to interact with members of their own group more frequently, demonstrating an emergent in-group bias.
The researchers then introduced unique ‘pixel’ identifications to each sprite, which allowed the AI agents to distinguish between individuals. This enhanced the in-group bias, revealing that familiarity and the learning processes of individuals could drive group preferences. The AI agents did not require any hardcoded motivations — instead, the in-group bias was derived solely from reinforcement learning.
The implications of this research are profound. By leveraging AI and reinforcement learning, researchers can create models that simulate complex social behaviours without explicitly including additional motives. This opens up new avenues for understanding the origins of social biases and group formations without the need to postulate specific innate motivations.
As AI evolves, the intersection of AI and social psychology will open up a new frontier of possibilities, enabling us to explore the depths of human interaction like never before.
