Afra Feyza Akyürek

PRBench: Large-Scale Expert Rubrics for Evaluating High-Stakes Professional Reasoning

Afra Feyza Akyürek, Advait Gosai, Vipul Kumar Gupta, Jaehwan Jeong et autres

Afra Feyza Akyürek, Advait Gosai, Chen Bo Calvin Zhang, Vipul Gupta, Jaehwan Jeong, Anisha Gunjal, Tahseen Rabbani, Maria Mazzone, David Randolph IV, Mohammad Mahmoudi Meymand, Gurshaan Chattha, Paula Rodriguez, Diego A. Mares Buendia, Pavit Singh, Michael Liu, Subodh Chawla, Peter Cline, Lucy …

Online Rubrics Elicitation from Pairwise Comparisons

MohammadHossein Rezaei, Robert Vacareanu, Zihao Wang, Clinton Wang et autres

Rubrics provide a flexible way to train LLMs on open-ended long-form answers where verifiable rewards are not applicable and human preferences provide coarse signals. Prior work shows that reinforcement learning with rubric-based rewards leads to consistent gains in LLM post-training. Most existing …

Is Active Persona Inference Necessary for Aligning Small Models to Personal Preferences?

Zilu Tang, Afra Feyza Akyürek, Ekin Akyürek, Derry Tanti Wijaya

A prominent issue in aligning language models (LMs) to personalized preferences is underspecification -- the lack of information from users about their preferences. A popular trend of injecting such specification is adding a prefix (e.g. prior relevant conversations) to the current user's …

Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability

Afra Feyza Akyürek, Ekin Akyürek, Leshem Choshen, Derry Tanti Wijaya et autres

While language models (LMs) can sometimes generate factually correct text and estimate truth values of individual claims, these generally do not reflect a globally coherent, manipulable model of the world. As a consequence, current LMs also generate incorrect or nonsensical content, and …

Deductive Closure Training of Language Models for Coherence, Accuracy, and Updatability

Afra Feyza Akyürek, Ekin Akyürek, Leshem Choshen, Derry Tanti Wijaya et autres

While language models (LMs) can sometimes generate factually correct text and estimate truth values of individual claims, these generally do not reflect a globally coherent, manipulable model of the world.As a consequence, current LMs also generate incorrect or nonsensical content, and are …

ru (code pays fourni par la source)

DUnE: Dataset for Unified Editing

Afra Feyza Akyürek, E. T-S. Pan, Garry Kuwanto, Derry Tanti Wijaya

Even the most advanced language models remain susceptible to errors necessitating to modify these models without initiating a comprehensive retraining process. Model editing refers to the modification of a model's knowledge or representations in a manner that produces the desired outcomes. Prior …

Low-Resource Machine Translation Training Curriculum Fit for Low-Resource Languages

Garry Kuwanto, Afra Feyza Akyürek, Isidora Chara Tourni, Siyang Li et autres

us (code pays fourni par la source)

RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs

Afra Feyza Akyürek, Ekin Akyürek, Aman Madaan, Ashwin Kalyan et autres

Despite their unprecedented success, even the largest language models make mistakes. Similar to how humans learn and improve using feedback, previous work proposed providing language models with natural language feedback to guide them in repairing their outputs. Because human-generated critiques are expensive …

Agenda Setting, Cross-cutting Effects, and Political Expression on Social Media: The Gun Violence Case

Лэй Гуо, Yiyan Zhang, Kate K. Mays, Afra Feyza Akyürek et autres

Focusing on a polarized issue—U.S. gun violence—this study examines agenda setting as an antecedent of political expression on social media. A state-of-the-art machine-learning model was used to analyze news coverage from 25 media outlets—mainstream and partisan. Those results were paired with a …

cn, us (code pays fourni par la source)

RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs

Afra Feyza Akyürek, Ekin Akyürek, Ashwin Kalyan, Peter E. Clark et autres

Afra Feyza Akyurek, Ekin Akyurek, Ashwin Kalyan, Peter Clark, Derry Tanti Wijaya, Niket Tandon. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2023.

us (code pays fourni par la source)

DUnE: Dataset for Unified Editing

Afra Feyza Akyürek, E. T-S. Pan, Garry Kuwanto, Derry Tanti Wijaya

Even the most advanced language models remain susceptible to errors necessitating to modify these models without initiating a comprehensive retraining process.Model editing refers to the modification of a model's knowledge or representations in a manner that produces the desired outcomes.Prior research primarily …

us (code pays fourni par la source)

Challenges in Measuring Bias via Open-Ended Language Generation

Afra Feyza Akyürek, Muhammed Yusuf Kocyigit, Sejin Paik, Derry Tanti Wijaya

Researchers have devised numerous ways to quantify social biases vested in pretrained language models. As some language models are capable of generating coherent completions given a set of textual prompts, several prompting datasets have been proposed to measure biases between social groups …