Felix Hill

SIMA 2: A Generalist Embodied Agent for Virtual Worlds

SIMA team, Adrian Bolton, Alexander Lerchner, Alexandra Cordell et autres

We introduce SIMA 2, a generalist embodied agent that understands and acts in a wide variety of 3D virtual worlds. Built upon a Gemini foundation model, SIMA 2 represents a significant step toward active, goal-directed interaction within an embodied environment. Unlike prior …

Why transformers are obviously good models of language

Felix Hill

Nobody knows how language works, but many theories abound. Transformers are a class of neural networks that process language automatically with more success than alternatives, both those based on neural computations and those that rely on other (e.g. more symbolic) mechanisms. Here, …

Language models, like humans, show content effects on reasoning tasks

Andrew K. Lampinen, Ishita Dasgupta, Stephanie C. Y. Chan, Hannah Sheahan et autres

reasoning is a key ability for an intelligent system. Large language models (LMs) achieve above-chance performance on abstract reasoning tasks but exhibit many imperfections. However, human abstract reasoning is also imperfect. Human reasoning is affected by our real-world knowledge and beliefs, and …

us, gb (code pays fourni par la source)

SODA: Bottleneck Diffusion Models for Representation Learning

Drew A. Hudson, Daniel Zoran, Mateusz Malinowski, Andrew K. Lampinen et autres

We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorpo-rates an image encoder, which distills a source view into a compact representation, that, in turn, guides the generation of related novel views. We show that by imposing a …

us, gb (code pays fourni par la source)

What needs to go right for an induction head? A mechanistic study of in-context learning circuits and their formation

Aaditya K. Singh, Ted Moskovitz, Felix Hill, Stephanie C. Y. Chan et autres

In-context learning is a powerful emergent ability in transformer models. Prior work in mechanistic interpretability has identified a circuit element that may be critical for in-context learning -- the induction head (IH), which performs a match-and-copy operation. During training of large transformers …

Scaling Instructable Agents Across Many Simulated Worlds

SIMA team, Maria Abi Raad, Arun Kumar Ahuja, Catarina Barros et autres

Building embodied AI systems that can follow arbitrary language instructions in any 3D environment is a key challenge for creating general AI. Accomplishing this goal requires learning to ground language in perception and embodied actions, in order to accomplish complex tasks. The …

LGBTQIA+ Cultural Competence in Physical Therapy: An Exploratory Qualitative Study From the Clinician’s Perspective

Melissa C. Hofmann, Nancy F. Mulligan, Karla A Bell, Chris W. Condran et autres

OBJECTIVE: The purpose of this study was to understand the lesbian, gay, bisexual, transgender, queer, intersex, agender, and other gender and sexually diverse identities (LGBTQIA+) health care experience and associated cultural competence from the physical therapist perspective (physical therapist and physical therapist …

us (code pays fourni par la source)

SODA: Bottleneck Diffusion Models for Representation Learning

Drew A. Hudson, Daniel Zoran, Mateusz Malinowski, Andrew K. Lampinen et autres

We introduce SODA, a self-supervised diffusion model, designed for representation learning. The model incorporates an image encoder, which distills a source view into a compact representation, that, in turn, guides the generation of related novel views. We show that by imposing a …

The Transient Nature of Emergent In-Context Learning in Transformers

Aaditya K. Singh, Stephanie C. Y. Chan, Ted Moskovitz, Erin Grant et autres

Transformer neural networks can exhibit a surprising capacity for in-context learning (ICL) despite not being explicitly trained for it. Prior work has provided a deeper understanding of how ICL emerges in transformers, e.g. through the lens of mechanistic interpretability, Bayesian inference, or …

Vision-Language Models as Success Detectors

Yuqing Du, Ksenia Konyushkova, Misha Denil, Jessica Landon et autres

Detecting successful behaviour is crucial for training intelligent agents. As such, generalisable reward models are a prerequisite for agents that can learn to generalise their behaviour. In this work we focus on developing robust success detectors that leverage large, pretrained vision-language models …

The Edge of Orthogonality: A Simple View of What Makes BYOL Tick

Pierre H. Richemond, Allison Tam, Yunhao Tang, Florian Strub et autres

Self-predictive unsupervised learning methods such as BYOL or SimSiam have shown impressive results, and counter-intuitively, do not collapse to trivial representations. In this work, we aim at exploring the simplest possible mathematical arguments towards explaining the underlying mechanisms behind self-predictive unsupervised learning. …

Collaborating with language models for embodied reasoning

Ishita Dasgupta, Christine Kaeser‐Chen, Kenneth Marino, Arun Kumar Ahuja et autres

Reasoning in a complex and ambiguous environment is a key goal for Reinforcement Learning (RL) agents. While some sophisticated RL agents can successfully solve difficult tasks, they require a large amount of training data and often struggle to generalize to new unseen …