Hongkang Yang

A First-Principles Theory of Slow Thinking and Active Perception

Hongkang Yang, Zhewei Xu, Feiyu Xiong, E Weinan

As part of a series on first-principles modeling of cognitive functions, this paper attempts to provide a mathematical formulation of thinking and perception. It formally derives slow thinking or more generally, active perception, and encompasses the design, training and inference of slow …

Small Initialization Matters for Large Language Models

Liangkai Hang, Junjie Yao, Zhiyu Li, Feiyu Xiong et autres

Large language models provide a tractable system for asking how intelligence itself emerges, rather than only how LLMs can be engineered. Although progress is usually attributed to scale, data and architecture, we show that parameter initialization is a gene-like determinant of training …

Small Initialization Matters for Large Language Models

Liangkai Hang, Junjie Yao, Zhiyu Li, Feiyu Xiong et autres

Large language models provide a tractable system for asking how intelligence itself emerges, rather than only how LLMs can be engineered. Although progress is usually attributed to scale, data and architecture, we show that parameter initialization is a gene-like determinant of training …

cn, at, kp (code pays fourni par la source)

A First-Principles Theory of Slow Thinking and Active Perception

Hongkang Yang, Z Xu, Feiyu Xiong, E Weinan

As part of a series on first-principles modeling of cognitive functions, this paper attempts to provide a mathematical formulation of thinking and perception. It formally derives slow thinking or more generally, active perception, and encompasses the design, training and inference of slow …

at, cn (code pays fourni par la source)

A First-Principles Theory of Slow Thinking and Active Perception

Hongkang Yang, Zhiqin John Xu, Feiyu Xiong, Weinan Ee

MemOS: A Memory OS for AI System

Zhiyu Li, Chunyan Xi, Chunyu Li, Shichao Song et autres

Large Language Models (LLMs) have become an essential infrastructure for Artificial General Intelligence (AGI), yet their lack of well-defined memory management systems hinders the development of long-context reasoning, continual personalization, and knowledge consistency.Existing models mainly rely on static parameters and short-lived contextual …

Adaptive Preconditioners Trigger Loss Spikes in Adam

Zhiwei Bai, Zhangchen Zhou, Jiajie Zhao, Xiaolong Li et autres

Loss spikes commonly emerge during neural network training with the Adam optimizer across diverse architectures and scales, yet their underlying mechanism remains elusive. While previous explanations attribute these phenomena to sharper loss landscapes at lower loss, we show that landscape geometry alone …

Scalable Complexity Control Facilitates Reasoning Ability of LLMs

Liangkai Hang, Junjie Yao, Zhiwei Bai, Tianyi Chen et autres

The reasoning ability of large language models (LLMs) has been rapidly advancing in recent years, attracting interest in more fundamental approaches that can reliably enhance their generalizability. This work demonstrates that model complexity control, conveniently implementable by adjusting the initialization rate and …

MemOS: An Operating System for Memory-Augmented Generation (MAG) in Large Language Models

Zhiyu Li, Shichao Song, Hanyu Wang, Simin Niu et autres

Large Language Models (LLMs) have emerged as foundational infrastructure in the pursuit of Artificial General Intelligence (AGI). Despite their remarkable capabilities in language perception and generation, current LLMs fundamentally lack a unified and structured architecture for handling memory. They primarily rely on …

Memory$^3$: Language Modeling with Explicit Memory

Hongkang Yang, Zehao Lin, Wenjin Wang, Hao Wu et autres

The training and inference of large language models (LLMs) are together a costly process that transports knowledge from raw data to meaningful computation. Inspired by the memory hierarchy of the human brain, we reduce this cost by equipping LLMs with explicit memory, …

cn (code pays fourni par la source)

$\text{Memory}^3$: Language Modeling with Explicit Memory

Hongkang Yang, Zehao Lin, Wenjin Wang, Hao Wu et autres

The training and inference of large language models (LLMs) are together a costly process that transports knowledge from raw data to meaningful computation. Inspired by the memory hierarchy of the human brain, we reduce this cost by equipping LLMs with explicit memory, …

Operation of Dual-T-Type Modular Multilevel Converter for Uninterrupted Power Supply Under Bridge Failures

Cheng Wang, Hongkang Yang, Hongting Hua, Yaosuo Xue

Dual-T-type modular multilevel converter (DTMMC) is an alternative to the existing uninterrupted power supply (UPS) due to its advantage in flexible leg reusing, multiple power ports, and high operating efficiency. In this article, a DTMMC-UPS, which has series configurations on both input/output …

cn, us (code pays fourni par la source)