José Cambronero

Grounding AI Agents in Contracts: An Empirical Evaluation of Spec-Driven Test Generation

Michele Tufano, James McClure, José Cambronero, Runxiang Cheng et autres

LLM-based agents are increasingly used for coding tasks, where they have outperformed many classical approaches and scaled to repository-level tasks, such as test generation. However, when directly prompted to generate tests, these agents can fail to reason about the code and its …

Towards a Human-in-the-Loop Framework for Reliable Patch Evaluation Using an LLM-as-a-Judge

Sherry Shi, Michele Tufano, José Cambronero, Runxiang Cheng et autres

Reliable evaluation is crucial for advancing Automated Program Repair (APR), but prevailing benchmarks rely on execution-based evaluation metrics (pass@k), which fail to capture true patch validity. Determining validity can require costly manual annotation. To reduce this cost, we introduce a human-in-the-loop framework …

us (code pays fourni par la source)