Dr. Amir-massoud Farahmand’s career in reinforcement learning has led him across Canada to work with some of the world’s top researchers in the field in Edmonton and Montreal. Now, after completing two postdocs and working as a research scientist in the US, he has chosen to make Toronto his home as he becomes the newest faculty member at the Vector Institute.

“I knew I wanted to come back to Canada,” said Amir-massoud. “When I compared options, Vector had just launched and seemed to have a lot of potential. People are trying to make it a great place to both train talent and do research. There is also a lot of support from industry sponsors.” Amir-massoud had also met Vector researchers, including David Duvenaud, Richard Zemel, and Geoffrey Hinton at conferences. He noted, “The opportunity to collaborate with them was a big factor in my decision to come to Toronto.”

Leading reinforcement learning research in Canada

Early in his education, Amir-massoud thought he might become a neurosurgeon. He was curious about how people think, but he also enjoyed physics and computers. During his Master’s studies at the University of Tehran, he came across the concept of reinforcement learning, which piqued his interest and led him to pursue his research in Canada.

Amir-massoud had heard of Toronto and Montreal as hot spots for AI, but the team of researchers in the field of reinforcement learning at the University of Alberta drew him to Edmonton. It was there that he completed his PhD, working with Csaba Szepesvari, and received the Outstanding PhD Thesis Award from the University of Alberta’s Department of Computing Science.

In 2011, Amir-massoud found himself in Montreal for his postdoc in a lab with Doina Precup and Joelle Pineau at McGill University. Reflecting on his time there, he said,

“There weren’t so many job opportunities in Canada at the time. Since then, the emergence of the AI community has been very fast with the opening of new corporate labs and government investments. My hope is that the launch of the Vector Institute and growth of the surrounding ecosystem will help retain talent already trained in Canada.”

Fundamental and applied research

Amir-massoud’s research interests are in machine learning and reinforcement learning. His focus is on understanding principles required to design reinforcement learning agents that can solve challenging industrial problems. These agents, such as robots or web software, can adapt their behaviour based on the rewards or punishments they receive. For example, one of Amir-massoud’s research projects at Mitsubishi Electric Research Laboratories (MERL) in Boston used these agents to optimize air conditioning systems. In another project, he used reinforcement learning to teach an agent when to switch between battery and fuel in a hybrid electric vehicle to minimize the operating cost.

As he returns to Canada from his time in Boston, Amir-massoud is looking forward to advancing both fundamental and applied research at the Vector Institute. “Here at Vector, there is more flexibility in terms of application. My research can be more or less fundamental. There is also the possibility of starting a student group of my own to help scale research and move faster.”

“I want to work on applications that will make life easier for people,”

said Amir-massoud. At the same time, he said,

“A lot of theoretical questions need to be answered before we can easily solve a vast range of real world problems.”

He noted the importance of researchers being aware of the implications of their work and how people may or may not adapt to new technologies.

Collaborating and sharing knowledge

In his first few weeks at Vector, Amir-massoud has hit the ground running. This year he is a member of the organizing committee for the Canadian Institute for Advanced Research (CIFAR) Reinforcement Learning Summer School, which will be hosted by the Vector Institute in August.

“There may not be reinforcement learning or deep learning courses at a lot of universities, so this summer school is an opportunity for people to learn a lot in a short period of time. We’re trying to bring a lot of good researchers — some of the best in the world — and good coverage of topics. I would want to attend if I were a student.”

Vector’s Research Director, Dr. Richard Zemel, leads recruitment for the Vector Institute. Dr. Zemel said, “Amir-massoud is highly-accomplished, and we’re very excited that he has chosen to join the team at Vector. His background in reinforcement learning will add to the diversity of expertise among the Vector faculty.” Beyond Amir-massoud’s research accomplishments, Dr. Zemel said,

“His interests in collaborating with other researchers and working with students will help make Vector a vibrant, active hub for world-class AI research.”