Empowering Machine Learning Models with Contextual Knowledge for Enhancing the Detection of Eating Disorders in Social Media Posts

Tracking #: 3269-4483

Jose Alberto Benitez-Andrades
Maria Teresa García-Ordás
Mayra Russo
Ahmad Sakor
Luis Daniel Fernandes Rotger
Maria-Esther Vidal

Responsible editor: 
Guest Editors SW Meets Health Data Management 2022

Submission type: 
Full Paper
Social networks have become information dissemination channels, where announcements are posted frequently; they also serve as frameworks for debates in various areas (e.g., scientific, political, and social). In particular, in the health area, social networks represent a channel to communicate and disseminate novel treatments' success; they also allow ordinary people to express their concerns about a disease or disorder. The Artificial Intelligence (AI) community has developed analytical methods to uncover and predict patterns from posts that enable it to explain news about a particular topic, e.g., mental disorders expressed as eating disorders or depression. Albeit potentially rich while expressing an idea or concern, posts are presented as short texts, preventing, thus, AI models from accurately encoding these posts' contextual knowledge. We propose a hybrid approach where knowledge encoded in community-maintained knowledge graphs (e.g., Wikidata) is combined with deep learning to categorize social media posts using existing classification models. The proposed approach resorts to state-of-the-art named entity recognizers and linkers (e.g., Falcon 2.0) to extract entities in short posts and link them to concepts in knowledge graphs. Then, knowledge graph embeddings (KGEs) are utilized to compute latent representations of the extracted entities, which result in vector representations of the posts that encode these entities' contextual knowledge extracted from the knowledge graphs. These KGEs are combined with contextualized word embeddings (e.g., BERT) to generate a context-based representation of the posts that empower prediction models. We apply our proposed approach in the health domain to detect whether a publication is related to an eating disorder (e.g., anorexia or bulimia) and uncover concepts within the discourse that could help healthcare providers diagnose this type of mental disorder. We evaluate our approach on a dataset of 2,000 tweets about eating disorders. Our experimental results suggest that combining contextual knowledge encoded in word embeddings with the one built from knowledge graphs increases the reliability of the predictive models. The ambition is that the proposed method can support health domain experts in discovering patterns that may forecast a mental disorder, enhancing early detection and more precise diagnosis towards personalized medicine.
Full PDF Version: 


Solicited Reviews:
Click to Expand/Collapse
Review #1
By João Rafael Almeida submitted on 24/Sep/2022
Review Comment:

The authors have addressed all my previous comments.

Review #2
Anonymous submitted on 14/Oct/2022
Review Comment:

The authors have addressed my comments and much more in this new version. The issues about the English-writing, structure and explanation about the database are now satisfactory. Additionally, the experiments are very extensive and the methodology proposed is coherent. I think the work is acceptable and will be useful for the scientific community.

Review #3
Anonymous submitted on 07/Nov/2022
Review Comment:

This paper presents an approach for finding tweets about eating disorders. The main contribution is to add the use of knowledge graph embeddings into the data for the classification process. A 2K tweet dataset has been labeled and used for training and testing.

Strong points:
-Contributing new labeled dataset to the community
-Well-written and well-structured paper.

Weak points:
-Low novelty.
-Motivation for finding ED tweets is not clear.
-Questionable evaluation protocol.

Additional comments:
-It appears as the main contribution is the use of knowledge base embeddings for this particular problem, otherwise the approach is quite similar to what has been used for other domains.
-It's not easy to understand exactly why it's important to find tweets about ED (except those you can already find by search/filter).
-A separate validation set should have been used, not only training and test, otherwise overfitting is likely (and contributing to the high accuracy).
-As the authors also point out, the way the dataset is created (including labeling), might bias the sampling and results. I would have liked to be convinced that this is indeed not affecting the results, otherwise the performance is quite meaningless to me.
-Except for the fact that knowledge graph embeddings are used, I don't see a strong contribution wrt. the semantic web research area.
-"ED 1" is likely to be very inaccurate, I don't expect the majority of those that have an ED would have it in their bio.
-Results are not very useful without knowing the number of elements for each class (e.g., is there class imbalance, which significantly affects the classification problem?).