Comparative Analysis of Text-Embedding Models for Enhanced Search Functionality: A Case Study on Harvard Course Data

Le résumé fourni par la source

Embeddings, as numerical representations of words and phrases, play a crucial role in natural language processing, particularly in applications like search, recommendation systems, and sentiment analysis. This paper investigates the efficacy of OpenAI’s latest embedding model, text-embedding-ada-002, in comparison to traditional models such as BERT and fuzzy matching techniques. Despite claims of its superior performance, there is limited empirical research quantifying these differences. To address this gap, we implement a search functionality utilizing text-embedding-ada-002, BERT, and fuzzy matching on a custom dataset comprising Harvard University course information. By systematically applying these search methods, we analyze their respective efficiencies and accuracies in retrieving relevant course data based on semantic similarity to user queries. Our findings will highlight the advantages and limitations of each approach, providing valuable insights for improving search algorithms in educational contexts. Additionally, we will develop a user-friendly web application that enables the Harvard community to search and explore course offerings, making the results of our research accessible and practical for students and faculty alike.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.