Overview of BioCreAtIvE: critical assessment of information extraction for biology

@article{Hirschman2005OverviewOB,
  title={Overview of BioCreAtIvE: critical assessment of information extraction for biology},
  author={L. Hirschman and A. Yeh and C. Blaschke and A. Valencia},
  journal={BMC Bioinformatics},
  year={2005},
  volume={6},
  pages={S1 - S1}
}
BackgroundThe goal of the first BioCreAtIvE challenge (Critical Assessment of Information Extraction in Biology) was to provide a set of common evaluation tasks to assess the state of the art for text mining applied to biological problems. The results were presented in a workshop held in Granada, Spain March 28–31, 2004. The articles collected in this BMC Bioinformatics supplement entitled "A critical assessment of text mining methods in molecular biology" describe the BioCreAtIvE tasks… Expand
Evaluation of text-mining systems for biology: overview of the Second BioCreative community challenge
TLDR
A common characteristic observed in all three tasks was that the combination of system outputs could yield better results than any single system, including the development of the first text-mining meta-server. Expand
BioCreative-IV virtual issue
TLDR
This DATABASE virtual issue captures the major results from the Fourth bioCreative Challenge Evaluation Workshop, and is the sixth special issue devoted to BioCreative. Expand
BioCreative-IV virtual issue
BioCreative: Critical Assessment of Information Extraction in Biology is an international community-wide effort for evaluating text mining (TM) and information extraction systems applied to theExpand
BioCreative-2012 Virtual Issue
TLDR
This DATABASE virtual issue captures the major results from the BioCreative-2012 Workshop on Interactive Text Mining in the Biocuration Workflow and is the fifth special issue devoted to Biocreative. Expand
Overview of the BioCreative III Workshop
TLDR
It is concluded that the best performing systems for GN, P PI-ACT and PPI-IMT in realistic settings are not sufficient for fully automatic use and the importance of interactive systems is presented. Expand
Integrating Text Mining into the MGI Biocuration Workflow
TLDR
Mouse Genome Informatics proves the potential for the further incorporation of semi-automated processes into the curation of the biomedical literature by providing an overview of its pilot projects with NCBO's Open Biomedical Annotator and Fraunhofer SCAI's ProMiner. Expand
Integrating text mining into the MGI biocuration workflow
TLDR
Mouse Genome Informatics proves the potential for the further incorporation of semi-automated processes into the curation of the biomedical literature by discussing the search process, performance metrics and success criteria, and how a short list of potential text mining tools were identified. Expand
Benchmarking of the 2010 BioCreative Challenge III text-mining competition by the BioGRID and MINT interaction databases
TLDR
NLP systems can in the first instance facilitate expert curation by refining the list of candidate publications that contain PPI data, and more ambitiously, NLP approaches may be able to directly extract relevant information from full-text articles for rapid inspection by expert curators. Expand
BioCreative III interactive task: an overview
TLDR
The InterActive Task (IAT) was introduced to address the utility and usability of text mining tools for real-life biocuration tasks and provides the first steps toward the definition of metrics and functional requirements that are necessary for designing a formal evaluation of interactive curation systems in the BioCreative IV challenge. Expand
An overview of the BioCreative 2012 Workshop Track III: interactive text mining task
TLDR
An effort to bring together text mining tool developers and database biocurators to test the utility and usability of tools and indicates that some of the systems were able to improve efficiency of curation by speeding up the curation task significantly and improve annotation accuracy when compared with the performance on the manually curated set. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 34 REFERENCES
Evaluation of BioCreAtIvE assessment of task 2
TLDR
Concepts provided by GO are currently the most extended set of terms used for annotating gene products, thus they were explored to assess how effectively text mining tools are able to extract those annotations automatically. Expand
Overview of BioCreAtIvE task 1B: normalized gene lists
TLDR
This assessment demonstrates that multiple groups were able to perform a real biological task across a range of organisms, and holds out promise that the technology can provide partial automation of the curation process in the near future. Expand
BioCreAtIvE Task 1A: gene mention finding evaluation
TLDR
The 80% plus F-measure results are good, but still somewhat lag the best scores achieved in some other domains such as newswire, due in part to the complexity and length of gene names, compared to person or organization names in newswire. Expand
Data preparation and interannotator agreement: BioCreAtIvE Task 1B
TLDR
The preparation and evaluation of answer keys for training and testing found that clear annotation guidelines are important, along with careful interannotator experiments, to validate the generated gene lists, and answer pooling was much faster and allowed us to identify more conflicting genes than interannotATOR analysis. Expand
An evaluation of GO annotation retrieval for BioCreAtIvE and GOA
TLDR
A biological perspective on the evaluation, how the GOA team annotate GO using literature is explained, and some suggestions to improve the precision of future text-retrieval and extraction techniques are offered. Expand
A sentence sliding window approach to extract protein annotations from biomedical articles
TLDR
The "sentence sliding window" approach proposed here was found to efficiently extract text fragments from full text articles containing annotations on proteins, providing the highest number of correctly predicted annotations. Expand
Learning Statistical Models for Annotating Proteins with Function Information using Biomedical Text
TLDR
This work built a system to automatically annotate a given protein with codes from the Gene Ontology using the text of an article from the biomedical literature as evidence, and observes that the Naïve Bayes models were effective in filtering and ranking the initially hypothesized annotations. Expand
Finding genomic ontology terms in text using evidence content
TLDR
It is concluded that an automatic annotation system can effectively use the method introduced to identify biological properties in unstructured text to identify Gene Ontology annotations and their evidence in a set of articles. Expand
Protein annotation as term categorization in the gene ontology using word proximity networks
TLDR
The evaluation results indicate that the method for expanding words associated with GO nodes is quite powerful; it was able to successfully select appropriate evidence text for a given annotation in 38% of Task 2.1 queries. Expand
A simple approach for protein name identification: prospects and limits
TLDR
A simple and efficient approach that identifies gene and protein names in texts and returns database identifiers for matches and is evaluated in the recent BioCreAtIvE entity extraction and mention normalization task by an independent jury. Expand
...
1
2
3
4
...