Mehul Damani

Hello! I am a second year Ph.D. student at MIT, where I am advised by Jacob Andreas.

My high level research goal is to develop generally capable reinforcement learning (RL) systems that are aligned with human values and goals. I am particularly interested in methods that harness the common-sense knowledge of LLM’s to guide RL agents. I am also interested in studying cooperation in multi-agent systems, which encompasses both pure agent teams as well as human-AI teams. Finally, I am excited by the prospect of applying RL to solve real-world problems such as warehouse automation and traffic signal control.

Previously, I worked with Lerrel Pinto at NYU on developing automatic curriculum learning methods for RL agents. Before that, I was a part of the MARMot Lab at NUS, where I worked with Guillaume Sartoretti on applying multi-agent reinforcement learning to traffic signal control and multi-agent pathfinding.

Selected Publications

NeurIPS

Mitigating Generative Agent Social Dilemmas

Yocum, Julian, Christoffersen, Phillip, Damani, Mehul, Svegliato, Justin, Hadfield-Menell, Dylan, and Russell, Stuart

In NeurIPS 2023 Foundation Models for Decision Making Workshop 2023
TMLR

Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback

Casper, Stephen, Davies, Xander, Shi, Claudia, Gilbert, Thomas Krendl, Scheurer, Jeremy, Rando, Javier, Freedman, Rachel, Korbak, Tomasz, Lindner, David, Freire, Pedro, and others,

arXiv preprint arXiv:2307.15217 2023
AAMAS

SocialLight: Distributed Cooperation Learning towards Network-Wide Traffic Signal Control

Goel, Harsh, Zhang, Yifeng, Damani, Mehul, and Sartoretti, Guillaume

In Proceedings of the 2023 International Conference on Autonomous Agents and Multiagent Systems 2023

Abs

Many recent works have turned to multi-agent reinforcement learning (MARL) for adaptive traffic signal control to optimize the travel time of vehicles over large urban networks. However, achieving effective and scalable cooperation among junctions (agents) remains an open challenge, as existing methods often rely on extensive, non-generalizable reward shaping or on non-scalable centralized learning. To address these problems, we propose a new MARL method for traffic signal control, SocialLight, which learns cooperative traffic control policies by distributedly estimating the individual marginal contribution of agents on their local neighborhood. SocialLight relies on the Asynchronous Actor Critic (A3C) framework, and makes learning scalable by learning a locally-centralized critic conditioned over the states and actions of neighboring agents, used by agents to estimate individual contributions by counterfactual reasoning. We further introduce important modifications to the advantage calculation that help stabilize policy updates. These modifications decouple the impact of the neighbors' actions on the computed advantages, thereby reducing the variance in the gradient updates. We benchmark our trained network against state-of-the-art traffic signal control methods on standard benchmarks in two traffic simulators, SUMO and CityFlow. Our results show that SocialLight exhibits improved scalability to larger road networks and better performance across usual traffic metrics.
IEEE-RAL, ICRA

PRIMAL2: Pathfinding Via Reinforcement and Imitation Multi-Agent Learning - Lifelong

Damani, Mehul, Luo, Zhiyao, Wenzel, Emerson, and Sartoretti, Guillaume

IEEE Robotics and Automation Letters 2021

arXiv PDF Code
Springer

Distributed Reinforcement Learning for Robot Teams: a Review

Wang, Yutong, Damani, Mehul, Wang, Pamela, Cao, Yuhong, and Sartoretti, Guillaume

Current Robotics Reports 2022

Abs

Recent advances in sensing, actuation, and computation have opened the door to multi-robot systems consisting of hundreds/thousands of robots, with promising applications to automated manufacturing, disaster relief, harvesting, last-mile delivery, port/airport operations, or search and rescue. The community has leveraged model-free multi-agent reinforcement learning (MARL) to devise efficient, scalable controllers for multi-robot systems (MRS). This review aims to provide an analysis of the state-of-the-art in distributed MARL for multi-robot cooperation.