Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning
- Beliz Gunel, Jingfei Du, Alexis Conneau, V. Stoyanov
- Computer ScienceInternational Conference on Learning…
- 3 November 2020
This work proposes a supervised contrastive learning (SCL) objective for the fine-tuning stage of natural language understanding classification models and demonstrates that the new objective leads to models that are more robust to different levels of noise in the training data, and can generalize better to related tasks with limited labeled task data.
Learning Mixed-Curvature Representations in Product Spaces
- Albert Gu, Frederic Sala, Beliz Gunel, C. Ré
- MathematicsInternational Conference on Learning…
- 27 September 2018
Self-training Improves Pre-training for Natural Language Understanding
- Jingfei Du, Edouard Grave, Alexis Conneau
- Computer ScienceNorth American Chapter of the Association for…
- 5 October 2020
SentAugment, a data augmentation method which computes task-specific query embeddings from labeled data to retrieve sentences from a bank of billions of unlabeled sentences crawled from the web, is introduced.
Quantitative Magnetic Particle Imaging Monitors the Transplantation, Biodistribution, and Clearance of Stem Cells In Vivo
- B. Zheng, Marc P von See, S. Conolly
- BiologyTheranostics
- 1 January 2016
Magnetic Particle Imaging (MPI), a novel technique that directly images iron-oxide nanoparticle-tagged cells, can longitudinally monitor and quantify MSC administration in vivo and demonstrates that MPI offers strong utility for noninvasively imaging and quantifying the systemic distribution of cell therapies and other therapeutic agents.
Mind The Facts: Knowledge-Boosted Coherent Abstractive Text Summarization
- Beliz Gunel, Chenguang Zhu, Michael Zeng, Xuedong Huang
- Computer ScienceArXiv
- 27 June 2020
A novel architecture that extends Transformer encoder-decoder architecture is proposed that incorporates entity-level knowledge from the Wikidata knowledge graph and utilizes the ideas used in Transformer-XL language model in a bid to improve on these shortcomings.
Optimal Broadband Noise Matching to Inductive Sensors: Application to Magnetic Particle Imaging
- B. Zheng, P. Goodwill, S. Conolly
- PhysicsIEEE Transactions on Biomedical Circuits and…
- 20 July 2017
The fundamental limits of noise performance and bandwidth for inductive sensor-based measurement techniques in combination with a low-noise amplifier are described and three equivalent methods of noise matching to inductive sensors using transformer-like network topologies are presented.
VORTEX: Physics-Driven Data Augmentations Using Consistency Training for Robust Accelerated MRI Reconstruction
- Arjun D Desai, Beliz Gunel, A. Chaudhari
- Computer Science
- 3 November 2021
This work proposes applying physics-driven data augmentations for consistency training that leverage the domain knowledge of the forward MRI data acquisition process and MRI physics to achieve improved label efficiency and robustness to clinically-relevant distribution drifts.
Glean: Structured Extractions from Templatic Documents
- Sandeep Tata, Navneet Potti, James Bradley Wendt, L. Costa, Marc-Alexander Najork, Beliz Gunel
- Computer ScienceProceedings of the VLDB Endowment
- 1 February 2021
The overall architecture of Glean is described, and three key data management challenges are discussed : managing the quality of ground truth data, generating training data for the machine learning model using labeled documents, and building tools that help a developer rapidly build and improve a model for a given document type.
Data-Limited Tissue Segmentation using Inpainting-Based Self-Supervised Learning
- J. Dominic, Nandita Bhaskhar, A. Chaudhari
- Computer ScienceArXiv
- 14 October 2022
It is demonstrated how SSL can overcome paucity of labels for improving tissue segmentation by using unlabeled datasets, and outperformed baseline supervised models in the computations of clinically-relevant metrics in scenarios with very low amounts of labeled data.
Scale-Equivariant Unrolled Neural Networks for Data-Efficient Accelerated MRI Reconstruction
- Beliz Gunel, Arda Sahiner, J. Pauly
- Computer ScienceInternational Conference on Medical Image…
- 21 April 2022
This work proposes modeling the proximal operators of unrolled neural networks with scale-equivariant convolutional neural networks in order to improve the data-efficiency and robustness to drifts in scale of the images that might stem from the variability of patient anatomies or change in change in different MRI scanners.
...
...