Identificação de temas emergentes em notícias através de métodos não-supervisionados

Abstract

Novelty Detection in documents is a challenging task of great relevance today. The literature addresses this problem through different approaches, using supervised learning, unsupervised learning, self-supervised learning, among others. This topic is especially significant in the field of Natural Language Processing, as the goal is to differentiate texts belonging to a known set from those bringing new or emerging information. In this work, we propose to investigate different unsupervised models for novelty detection in a news dataset, with the objective of comparing them with supervised models. Models such as Local Outlier Factor, Isolation Forest, and Elliptic Envelope were evaluated, along with a modern approach based on the RAG architecture with Large Language Models, comparing them with baselines established in the literature. The metrics used include precision, recall, F1-score, and accuracy. The results demonstrated that the Local Outlier Factor model achieved promising performance, with an F1 of 80.90% and accuracy of 85.80%, compared to state-of-the-art baselines.