Zhenting Qi

Prioritization of Risks from Artificial Intelligence: A Delphi Study of 272 International Experts

Alexander K. Saeri, Jess Graham, Michael Noetel, Peter Slattery et autres

Artificial intelligence poses many risks, ranging from familiar present-day harms to unprecedented and potentially catastrophic ones. Effective risk management requires prioritization: we must understand which risks are most severe, who is most vulnerable, and who is most responsible for addressing them. We …

Prioritization of Risks from Artificial Intelligence: A Delphi Study of 272 International Experts

Alexander K. Saeri, Jess Graham, Michael Noetel, Peter Slattery et autres

Artificial intelligence poses many risks, ranging from familiar present-day harms to unprecedented and potentially catastrophic ones. Effective risk management requires prioritization: we must understand which risks are most severe, who is most vulnerable, and who is most responsible for addressing them. We …

au, ca, us, gb, cn (code pays fourni par la source)

MoCo: A One-Stop Shop for Model Collaboration Research

Shangbin Feng, Yuyang Bai, Ziyuan Yang, Yike Wang et autres

Advancing beyond single monolithic language models (LMs), recent research increasingly recognizes the importance of model collaboration, where multiple LMs collaborate, compose, and complement each other. Existing research on this topic has mostly been disparate and disconnected, from different research communities, and lacks …

MoCo: A One-Stop Shop for Model Collaboration Research

Shangbin Feng, Yuyang Bai, Ziyuan Yang, Yike Wang et autres

Advancing beyond single monolithic language models (LMs), recent research increasingly recognizes the importance of model collaboration, where multiple LMs collaborate, compose, and complement each other. Existing research on this topic has mostly been disparate and disconnected, from different research communities, and lacks …

us, cn (code pays fourni par la source)

Confucius Code Agent: Scalable Agent Scaffolding for Real-World Codebases

Sherman Wong, Zhenting Qi, Zhaodong Wang, Nan Hu et autres

Real-world software engineering tasks require coding agents that can operate on massive repositories, sustain long-horizon sessions, and reliably coordinate complex toolchains at test time. Existing research-grade coding agents offer transparency but struggle when scaled to heavier, production-level workloads, while production-grade systems achieve …

MuTIS: Enhancing Reasoning Efficiency through Multi Turn Intervention Sampling in Reinforcement Learning

Wenshuo Zhao, Xinyu Qiu, Zhenting Qi, Shuanglin Li et autres

Recently, large reasoning models (LRMs) have demonstrated state-of-the-art performance across a wide range of benchmarks.However, a common challenge for these models is the "overthinking" problem, which leads to excessive reasoning steps and significant computational overhead.Furthermore, the issues with long Chain-of-Thought (CoT) are …

cn, us (code pays fourni par la source)

Measuring the Faithfulness of Thinking Drafts in Large Reasoning Models

Zidi Xiong, Shan Chen, Zhenting Qi, Himabindu Lakkaraju

Large Reasoning Models (LRMs) have significantly enhanced their capabilities in complex problem-solving by introducing a thinking draft that enables multi-path Chain-of-Thought explorations before producing final answers. Ensuring the faithfulness of these intermediate reasoning processes is crucial for reliable monitoring, interpretation, and effective …

us (code pays fourni par la source)

Generalizing Trust: Weak-to-Strong Trustworthiness in Language Models

Martin Pawelczyk, Zhenting Qi, Aounon Kumar, Himabindu Lakkaraju

The rapid proliferation of generative AI, especially large language models, has led to their integration into a variety of applications. A key phenomenon known as weak-to-strong generalization - where a strong model trained on a weak model's outputs surpasses the weak model …

Constrained Human-AI Cooperation: An Inclusive Embodied Social Intelligence Challenge

Weihua Du, Qiushi Lyu, Jiaming Shan, Zhenting Qi et autres

We introduce Constrained Human-AI Cooperation (CHAIC), an inclusive embodied social intelligence challenge designed to test social perception and cooperation in embodied agents. In CHAIC, the goal is for an embodied agent equipped with egocentric observations to assist a human who may be …

P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains

Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi et autres

Existing methods on understanding the capabilities of LLMs in logical reasoning rely on binary entailment classification or synthetically derived rationales, which are not sufficient for proper investigation of model's capabilities. We present P-FOLIO, a human-annotated dataset consisting of diverse and complex reasoning …

Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems

Zhenting Qi, Hanlin Zhang, Eric P. Xing, Sham M. Kakade et autres

Retrieval-Augmented Generation (RAG) improves pre-trained models by incorporating external knowledge at test time to enable customized adaptation. We study the risk of datastore leakage in Retrieval-In-Context RAG Language Models (LMs). We show that an adversary can exploit LMs' instruction-following capabilities to easily …

P-FOLIO: Evaluating and Improving Logical Reasoning with Abundant Human-Written Reasoning Chains

Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi et autres

Simeng Han, Aaron Yu, Rui Shen, Zhenting Qi, Martin Riddell, Wenfei Zhou, Yujie Qiao, Yilun Zhao, Semih Yavuz, Ye Liu, Shafiq Joty, Yingbo Zhou, Caiming Xiong, Dragomir Radev, Rex Ying, Arman Cohan. Findings of the Association for Computational Linguistics: EMNLP 2024. 2024.

us, gb (code pays fourni par la source)