Master's Thesis: Danish Clinical Event Extraction

1.6.2020•2 min read

thesisNLPMachine LearningHealthcare

The full document is available as a PDF.

Keeping this here as a record. This is my master's thesis in Software Development (Design) from the IT University of Copenhagen, written with Frederik Wonsild and submitted on June 1, 2020, supervised by Leon Derczynski.

The full thesis is 60 pages. The abstract is below.

Danish Clinical Event Extraction

Developing a clinical event extraction system for electronic health records using deep learning and active learning

Danish electronic health records contain valuable information that is not fully utilized today. Given the unstructured nature of clinical texts, natural language processing techniques are required to extract relevant entities from these records. This study develops a deep learning based system capable of recognizing clinical events in Danish electronic health records. A total of 2.090 breast cancer-related sentences are annotated, comprising 42.753 words and 5.809 events. We fine-tune a pretrained multilingual BERT model on this data, reaching an F1-score of 87.5% on recognizing event spans.

An active learning system is implemented for sentence querying to explore how the cost of annotation can be reduced. We implement two active learning strategies, uncertainty sampling and diversity sampling. We find that both strategies lead to a higher F1-score than random sampling when trained on the same amount of sentences. Specifically, we see an F1-score increase of 1.3% and 2.6% compared to random sampling, corresponding to an error reduction of 7.2% and 14.5%. However, in terms of the amount of annotated words, we find random sampling to be less costly than both active learning strategies.

We test our fine-tuned BERT model on sentences related to pneumonia in order to establish its cross-domain capabilities. We observe a 5.7 percentage point drop in F1-score, leading to a score of 81.8%.

While the benefits of active learning in terms of annotation cost remain unclear and require more research, these findings do prove the feasibility of a Danish clinical event extraction system.