James Flemings

Personalizing Agent Privacy Decisions via Logical Entailment

James Flemings, Ren Yi, Octavian Suciu, Kassem Fawaz et autres

Personal large language model (LLM) agents increasingly perform tasks that require access to user data, raising concerns about appropriate data disclosure. We show that relying solely on LLMs to make data-sharing decisions is insufficient. Prompting LLMs to ground their decisions on contextual …

us (code pays fourni par la source)

PrivacySIM: Evaluating LLM Simulation of User Privacy Behavior

James Flemings, Murali Annavaram

Large language models (LLMs) are increasingly used to simulate human behavior, but their ability to simulate $individual$ privacy decisions is not well understood. In this paper, we address the problem of evaluating whether a core set of user persona attributes can drive …

Differentially Private Retrieval-Augmented Generation

Tingting Tang, James Flemings, Yongqin Wang, Murali Annavaram

Retrieval-augmented generation (RAG) is a widely used framework for reducing hallucinations in large language models (LLMs) on domain-specific tasks by retrieving relevant documents from a database to support accurate responses. However, when the database contains sensitive corpora, such as medical records or …

Hubble: a Model Suite to Advance the Study of LLM Memorization

Johnny Tian-Zheng Wei, Ameya Godbole, Mohammad Aflah Khan, Qi Wang et autres

We present Hubble, a suite of fully open-source large language models (LLMs) for the scientific study of LLM memorization. Hubble models come in standard and perturbed variants: standard models are pretrained on a large English corpus, and perturbed models are trained in …

TokenSmith: Streamlining Data Editing, Search, and Inspection for Large-Scale Language Model Training and Interpretability

Mohammad Aflah Khan, Ameya Godbole, Johnny Tian-Zheng Wei, Ryan Z. Wang et autres

Understanding the relationship between training data and model behavior during pretraining is crucial, but existing workflows make this process cumbersome, fragmented, and often inaccessible to researchers. We present TokenSmith, an open-source library for interactive editing, inspection, and analysis of datasets used in …

Memory-Efficient Differentially Private Training with Gradient Random Projection

Alex Mulrooney, Devansh Gupta, James Flemings, Huanyu Zhang et autres

Differential privacy (DP) protects sensitive data during neural network training, but standard methods like DP-Adam suffer from high memory overhead due to per-sample gradient clipping, limiting scalability. We introduce DP-GRAPE (Gradient RAndom ProjEction), a DP training method that significantly reduces memory usage …

Differentially Private In-context Learning via Sampling Few-shot Mixed with Zero-shot Outputs

James Flemings, Hongyi Li, Meisam Razaviyayn, Murali Annavaram

In-context learning (ICL) has shown promising improvement in downstream task adaptation of LLMs by augmenting prompts with relevant input-output examples (demonstrations). However, the ICL demonstrations can contain privacy-sensitive information, which can be leaked and/or regurgitated by the LLM output. Differential Privacy (DP), …

Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Models

James Flemings, Bo Jiang, Wanrong Zhang, Zafar Takhirov et autres

Language models (LMs) rely on their parametric knowledge augmented with relevant contextual knowledge for certain tasks, such as question answering.However, the contextual knowledge can contain private information that may be leaked when answering queries, and estimating this privacy leakage is not well …

us (code pays fourni par la source)

TokenSmith: Streamlining Data Editing, Search, and Inspection for Large-Scale Language Model Training and Interpretability

Mohammad Aflah Khan, Ameya Godbole, Johnny Tian-Zheng Wei, James Flemings et autres

Mohammad Aflah Khan, Ameya Godbole, Johnny Wei, Ryan Yixiang Wang, James Flemings, Krishna P. Gummadi, Willie Neiswanger, Robin Jia. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. 2025.

de, us (code pays fourni par la source)

Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Models

James Flemings, Bo Jiang, Zafar Takhirov, Murali Annavaram

Language models (LMs) rely on their parametric knowledge augmented with relevant contextual knowledge for certain tasks, such as question answering. However, the contextual knowledge can contain private information that may be leaked when answering queries, and estimating this privacy leakage is not …

Adaptively Private Next-Token Prediction of Large Language Models

James Flemings, Meisam Razaviyayn, Murali Annavaram

As Large Language Models (LLMs) proliferate, developing privacy safeguards for these models is crucial. One popular safeguard involves training LLMs in a differentially private manner. However, such solutions are shown to be computationally expensive and detrimental to the utility of these models. …

Differentially Private Next-Token Prediction of Large Language Models

James Flemings, Meisam Razaviyayn, Murali Annavaram

Ensuring the privacy of Large Language Models (LLMs) is becoming increasingly important. The most widely adopted technique to accomplish this is DP-SGD, which trains a model to guarantee Differential Privacy (DP). However, DP-SGD overestimates an adversary's capabilities in having white box access …