• Corpus ID: 240288799

Using Text Analytics for Health to Get Meaningful Insights from a Corpus of COVID Scientific Papers

@article{Soshnikov2021UsingTA,
  title={Using Text Analytics for Health to Get Meaningful Insights from a Corpus of COVID Scientific Papers},
  author={Dmitry Soshnikov and Vickie Soshnikova},
  journal={ArXiv},
  year={2021},
  volume={abs/2110.15453}
}

• Dmitry Soshnikov, Vickie Soshnikova
• Published 28 October 2021
• Computer Science
• ArXiv

Since the beginning of COVID pandemic, there have been around 700000 scientific papers published on the subject. A human researcher cannot possibly get acquainted with such a huge text corpus — and therefore developing AI-based tools to help navigating this corpus and deriving some useful insights from it is highly needed. In this paper, we will use Text Analytics for Health pre-trained service together with some cloud tools to extract some knowledge from scientific papers, gain insights, and build a tool to help researcher navigate the paper collection in a meaningful way. Code. The code for experiments described in this paper is available from http://github.com/CloudAdvocacyAzurePaperExplorationWorkshop. Introduction: Automatic Paper Analysis Automatic scientific paper analysis is fast growing area of studies, and due to recent improvements in NLP techniques is has been greatly improved in the recent years. In this paper, we will show you how to derive specific insights from COVID papers, such as changes in medical treatment over time, or joint treatment strategies using several medications. The main approach we will describe in this paper is to extract as much semi-structured information from text as possible, and then store it into some NoSQL database for further processing. Storing information in the database would allow us to make some very specific queries to answer some of the questions, as well as to provide visual exploration tool for medical expert for structured search and insight generation. Processing large volume of papers is done by running parallel sweep job on Azure Machie Learning cluster. The overall architecture of the proposed system is shown in Fig. 1: Fig.1: Architecture of a system to extract knowledge insights from a corpus of scientific papers. Note that this architecture is built on top of the platform components of Microsoft Azure, which allows us to delegate many complex issues (such as scalability) to the cloud provider. COVID Scientific Papers and CORD Dataset The idea to apply NLP methods to scientific literature seems quite natural and has been proposed in many different works [1,2,3]. First of all, scientific texts are already well-structured, they contain things like keywords, abstract, as well as well-defined terms. Thus, at the very beginning of COVID pandemic, a research challenge has been launched on Kaggle to analyze scientific papers on the subject. The dataset behind this competition is called CORD [4], and it contains constantly updated corpus of everything that is published on topics related to COVID. This dataset consists of the following parts: • Metadata file Metadata.csv contains most important information for all publications in one place. Each paper in this table has unique identifier cord_uid (which in fact does not happen to be completely unique, once you start working with the dataset). The information includes: Title of publication, Journal, Authors, Abstract, Date of publication, doi • Full-text papers in document_parses directory that contain structured text in JSON format, which greatly simplifies the analysis. • Pre-built Document Embeddings that maps cord_uids to float vectors that reflect some overall semantics of the paper. In this paper, we will focus on paper abstracts, because they contain the most important information from the paper. However, for full analysis of the dataset, it makes sense to use the same approach on full texts as well. Natural Language Processing Tasks In the recent years, there has been a huge progress in the field of Natural Language Processing, and very powerful neural network language models have been trained. In the area of NLP, the following tasks are typically considered: • Text classification / intent recognition — In this task, we need to classify a piece of text into a number of categories. This is a typical classification task. • Sentiment Analysis — We need to return a number that shows how positive or negative the text is. This is a typical regression task. • Named Entity Recognition (NER) — In NER, we need to extract named entities from text, and determine their type. For example, we may be looking for names of medicines, or diagnoses. Another task similar to NER is keyword extraction. • Text summarization — Here we want to be able to produce a short version of the original text, or to select the most important pieces of text. • Question Answering — In this task, we are given a piece of text and a question, and our goal is to find the exact answer to this question from text. • Open-Domain Question Answering (ODQA) — The main difference from previous task is that we are given a large corpus of text, and we need to find the answer to our question somewhere in the whole corpus. In [5] have described how we can use ODQA approach to automatically find answers to specific COVID questions. However, this approach does not provide insights into the text corpus. To make some insights from text, NER seems to be the most prominent technique to use. If we can find specific entities that are present in text, we could then perform semantically rich search in text that answers specific questions, as well as obtain data on co-occurrence of different entities, figuring out specific scenarios that interest us. To train NER model, as well as any other neural language model, we need a reasonably large dataset that is properly marked up. Finding those datasets is often not an easy task, and producing them for new problem domain often requires initial human effort to mark up the data. Pre-Trained Language Models and Text Analytics for Health Cognitive Service Luckily, modern transformer language models can be trained in semi-supervised manner using transfer learning. First, the base language model (for example, BERT [6]) is trained on a large corpus of text first, and then can be specialized to a specific task such as classification or NER on a smaller dataset. This transfer learning process can also contain additional step — further training of generic pretrained model on a domain-specific dataset. For example, in the area of medical science Microsoft Research has pre-trained a model called PubMedBERT [7], using texts from PubMed repository. This model can then be further adopted to different specific tasks, provided we have some specialized datasets available. However, training a model requires a lot of skills and computational power, in addition to a dataset. Microsoft (as well as some other large cloud vendors) also make some pre-trained models available through the REST API. Those services are called Cognitive Services, and one of those services for working with text is called Text Analytics [8]. It can do the following: • Keyword extraction and NER for some common entity types, such as people, organizations, dates/times, etc. • Sentiment analysis • Language Detection • Entity Linking, by automatically adding internet links to some most common entities. This also performs disambiguation, for example Mars can refer to both the planet or a chocolate bar, and correct link would be used depending on the context. For example, here is the result of analyzing one medical paper abstract by Text Analytics: As you can see, some specific entities (for example, HCQ, which is short for hydroxychloroquine) are not recognized at all. Recently, a special version of the service, called Text Analytics for Health [9] was released, which exposes pre-trained PubMedBERT model with some additional capabilities. Here is the result of extracting entities from the same piece of text using Text Analytics for Health: Text Analytics is a REST service, which can be called by using Text Analytics Python SDK in the following manner: poller = text_analytics_client.begin_analyze_healthcare_entities([txt]) res = list(poller.result()) print(res) In addition to just the list of entities, we also get the following: • Enity Mapping of entities to standard medical ontologies, such as UMLS [10]. • Relations between entities inside the text, such as TimeOfCondition, etc. • Negation, which indicated that an entity was used in negative context, for example COVID-19 diagnosis did not occur. Fig. 2: Results of entity extraction, linking and ontology mapping returned by Text Analytics for Health In addition to using Python SDK, we can also call Text Analytics using REST API directly. This is useful if you are using a programming language that does not have a corresponding SDK, or if you prefer to receive Text Analytics result in the JSON format for further storage or processing. In Python, this can be easily done using requests library: uri = f"{endpoint}/text/analytics/v3.1/entities/ health/jobs?model-version=v3.1" headers = { "Ocp-Apim-Subscription-Key" : key } resp = requests.post(uri,headers=headers,data=doc) res = resp.json() if res['status'] == 'succeeded': result = t['results'] else: result = None Resulting JSON file will look like this: {"id": "jk62qn0z", "entities": [ {"offset": 24, "length": 28, "text": "coronavirus disease pandemic", "category": "Diagnosis", "confidenceScore": 0.98, "isNegated": false}, {"offset": 54, "length": 8, "text": "COVID-19", "category": "Diagnosis", "confidenceScore": 1.0, "isNegated": false, "links": [ {"dataSource": "UMLS", "id": "C5203670"}, {"dataSource": "ICD10CM", "id": "U07.1"}, ... ]}, "relations": [ {"relationType": "Abbreviation", "bidirectional": true, "source": "#/results/documents/2/entities/6", "target": "#/results/documents/2/entities/7"}, ...], } In production code, one may want to incorporate a mechanism that will retry the operation when an error is returned by the service. Parallel Paper Processing using Azure Machine Learning Cluster Since the dataset currently contains ~700K paper abstracts, processing them sequentially through Text Analytics would be quite time-consuming. To run this code in parallel, we can use technologies such as Azure Batch or Azure Machine Learning [11]. Both allow you to create a cluster of ide