Title: Feature learning and “the linear representation hypothesis” for monitoring and steering LLMs

Abstract: A trained Large Language Model (LLM) contains much of human knowledge. Yet, it is difficult to gauge the extent or accuracy of that knowledge, as LLMs do not always “know what they know” and may even be unintentionally or actively misleading. In this talk I will discuss feature learning  introducing Recursive Feature Machines—a powerful method originally designed for extracting relevant features from tabular data. I will demonstrate how this technique enables us to detect and precisely guide LLM behaviors toward almost any desired concept by manipulating a single fixed vector in the LLM activation space.

About the Vector Distinguished Lecture Series
The Vector Distinguished Lecture Series is a formal gathering of academic and industrial data scientists across the Greater Toronto Area (GTA) to discuss advanced topics in machine learning and its goal is to build a stronger machine learning community in Toronto. The talks will be given by international and local faculty and industry professionals.

The seminar series is intended for university faculty and graduate students in machine learning across computer science, ECE, statistics, mathematics, linguistics, and medicine, as well as PhD-level data scientists doing interesting applied research in the GTA. The Toronto machine learning community will be stronger when we know each other and know what problems people are working on.

Vector Distinguished Lecture Series is currently open to the public remotely. Researchers in the Vector community will have opportunities to meet speakers in person. All talks will be streamed online and be posted on the Vector YouTube Channel.