Convergence of dramatic increase in the available data and processing power, enabled by ubiquitous sensing and computing capabilities is rapidly changing engineered systems. Cyber-Physical Systems (CPS) refers to such systems with tightly integrated computational, control and physical capabilities, like transportation networks, smart cities, autonomous vehicles, sensor networks, power grids and healthcare systems. However, designing and implementing CPS involve an array of complex and challenging tasks: learning and making inference from data, designing scalable optimization and control methods, and developing decentralized and adaptive decision making algorithms.

My research aims to address these challenging questions in the context of cyber-physical systems, while contributing to the fields such as reinforcement learning, control theory and game theory. I am especially interested in many applications in the areas of multi-agent systems, intelligent transportation systems and renewable energy systems.

Some specific topics are:

Reinforcement Learning

Theory: Exploration vs Exploitation in RL, Safe RL, Multi-Armed Bandits, Recurrent Neural Networks.

How do we learn to control a stochastic dynamical system in order to achieve some desired objectives? Reinforcement learning is the standard paradigm to address these class of problems. I am especially interested in three specific questions in this context: How do we learn efficiently, with minimum data and minimum cost for learning? How do we learn optimal control policies while respecting the safety constraints? How do we address the multi-agent learning problem with strategic and information constraints? My objective is to enable the design of learning-based control algorithms for autonomous systems, with provable guarantees on performance, safety and security.

I am also working on developing data-driven learning and optimization algorithms for intelligent transportation systems. For example, how do we exploit the historical data and high-resolution real time data to predict future traffic flow and signal timings? I use machine learning techniques to develop scalable learning and prediction algorithms in this context. These algorithms can be used to develop traffic control and routing protocols in transportation networks, to optimize the throughput and reduce queues. I am also interested in using tools from learning theory to model consumer behavior and preferences, especially in the context of energy systems and online markets.

Control Theory:

Theory: Markov Decision Processes, Approximate Dynamic Programming, Decentralized Control, Optimal Control.

Applications: How do we systematically design a scalable distributed control architecture that will give provable stability, robustness and high performance in the presence of uncertainty and stochasticity? This is the fundamental question that I am interested in. In particular, I am working on developing decentralized control algorithms for multi-agent systems, motivated by the problems in robotics and UAVs. I am also interested in control problems in the context of intelligent transportation systems. Vehicle-to-Vehicle (V2V), Vehicle-to-Infrastructure (V2I) and Vehicle-to-Cloud (V2C) communication capabilities have the potential to dramatically change the transportation systems operation in terms of improving road safety, alleviating traffic congestion and reducing fuel consumption. However, this requires innovative (decentralized) control design coupled with scalable learning and prediction algorithms. Some of my recent works are in this context.

Game Theory:

Theory: Stochastic Games, Mechanism Design, Learning in Games

Applications: My recent interest is in sharing economy models. In my past works, I have analyzed sharing economy models in the context of electricity networks. I plan to continue my work on sharing economy and extend it to other important application areas. Mechanism design for renewable energy systems, especially for demand response, is another area that I am interested in.



  1. Rajarshi Bhattacharyya, Bainan Xia, Desik Rengarajan, Srinivas Shakkottai, Dileep Kalathil, “FlowBazaar: A Market-Mediated Software Defined Communications Ecosystem at the Wireless Edge”, January, 2019. PDF

  2. R. Bhattacharyya, A. Bura, D. Rengarajan, M. Rumuly, S. Shakkottai, D. Kalathil, R. Mok, A. Dhamdhere, “QFlow: A Reinforcement Learning Approach to High QoE Video Streaming over Wireless Networks”, December, 2018. PDF

  3. Deepan Muthirayan, Dileep Kalathil, Kameshwar Poolla, Pravin Varaiya, “Mechanism Design for Demand Response Programs”, submitted to IEEE Transactions on Smart Grids, September, 2018. PDF

  4. Vamsi Krishna Vegamoor, Dileep Kalathil, Sivakumar Rathinam, Swaroop Darbha, “Reducing Time Headway in Homogeneous CACC Vehicle Platoons in the Presence of Packet Drops”, European Control Conference (ECC), 2019


  1. Shahana Ibrahim, Dileep Kalathil, Rene O. Sanches, Pravin Varaiya, “Estimating Phase Duration for SPaT Messages”, IEEE Transactions on Intelligent Transportation Systems, November, 2018. PDF

  2. Dileep Kalathil, Chenye Wu, Kameshwar Poolla, Pravin Varaiya, “Sharing Economy for the Smart Grid”, accepted to IEEE Transactions on Smart Grids, August, 2017. PDF

  3. Naumaan Nayyar, Dileep Kalathil, Rahul Jain, “On Regret-Optimal Learning in Decentralized Multi-player Multi-armed Bandits”, IEEE Transactions on Control of Network Systems, December, 2016. PDF

  4. Naumaan Nayyar, Dileep Kalathil, Rahul Jain, “Optimal Decentralized Control with Asymmetric One-Step Delayed Information Sharing”, IEEE Transactions on Control of Network Systems, December, 2016. PDF

  5. Dileep Kalathil, Vivek Borkar, Rahul Jain, “Approachability in Stackelberg Stochastic Games with Vector Costs”, Dynamic Games and Applications, 7(3):422-42, September, 2017. PDF

  6. (Authors are in the alphabetical order) William Haskel, Rahul Jain, Dileep Kalathil, “Empirical Dynamic Programming”, Mathematics of Operations Research, 41(2):402 - 429, January, 2016. PDF

  7. Dileep Kalathil, Naumaan Nayyar, Rahul Jain, “Decentralized Learning for Multi-Player Multi-Armed Bandits”, IEEE Transactions on Information Theory, 60(4):2331-2345, April, 2014. PDF

  8. Dileep Kalathil, Rahul Jain, “Spectrum Sharing through Contracts for Cognitive Radios”, IEEE Transactions on Mobile Computing, 12(10):1999-2011, October, 2013. PDF


  1. Dileep Kalathil, Ram Rajagopal, “Learning Demand Response with Bandits”, for submission to IEEE Transactions on Smart Grids, August, 2017. PDF

  2. Dileep Kalathil, Deepan Muthirayan, Dai Wang, Kameshwar Poolla, Pravin Varaiya, “Selling Demand Response Using Options”, for submission to IEEE Transactions on Smart Grid, August, 2017. PDF

  3. Jonathan Mather, Enrique Baeyens, Dileep Kalathil, Kameshwar Poolla, “The Geometry of Locational Marginal Prices”, for submission to IEEE Transactions on Power Systems, August, 2017. PDF

Recent Conference Publications:

  1. Deepan Muthirayan, Dileep Kalathil, Kameshwar Poolla, Pravin Varaiya, “Baseline Estimation and Scheduling for Demand Response”, IEEE Conference on Decision and Control (CDC), December, 2018.

  2. Rodrigo Henriquez-Auba, Patricia Pauli, Dileep Kalathil, Duncan S. Callaway, Kameshwar Poolla, “The Sharing Economy for Residential Solar Generation”, IEEE Conference on Decision and Control (CDC), December, 2018.

For the past conference publications, please check my CV.


  • Kishan Badrinath. Joined the PhD program in Fall 2018
  • Desik Rengarajan (co-advised with Prof. Srinivas Shakkottai). Joined the PhD program in Fall 2017


  • Spring 2019
    • ECEN 489: Artificial Intelligence and Applications
  • Fall 2018
    • ECEN 689: Reinforcement Learning
  • Fall 2017:
    • ECEN 303: Random Signals and Systems




  • dileep [dot] kalathil [at] tamu [dot] edu
  • Room 334G, WEB, Department of Electrical and Computer Engineering, Texas A&M University, College Station, TX 77843-3128