Enhancing neural topic modeling for social media text via semantic bag of word clusters and log-domain Sinkhorn transport

Résumé fourni par la source

Topic modeling has been widely applied to analyze text data from social media platforms. Under this scenario, traditional Neural Topic Models (NTMs) encounter three primary challenges: (1) initial text representation; (2) the long-tail nature of topic distributions in social network texts; (3) approximation of Optimal Transport. Motivated by these challenges, we propose an end-to-end solution spanning from text representation to topic modeling. First, we propose SBoWC, a novel text representation method that performs dimensionality reduction while absorbing semantic information through base terms, achieved by combining word embeddings with clustering statistics. Subsequently, we propose GSWTM, a Wasserstein-based autoencoder topic model that fits the long-tail topic distribution in social network texts via Gamma priors and innovatively employs log-domain Sinkhorn to approximate Optimal Transport. Ablation studies demonstrate the transferability and effectiveness of SBoWC in text representation. GSWTM demonstrates significantly better performance than baselines in TU, C V , and the comprehensive metrics TQ across four real social network datasets of varying sizes. The log-domain Sinkhorn approximation exhibits excellent stability, allowing the regularization parameter ϵ to be reduced to 0.1–0.01, thereby approaching the original Optimal Transport.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.