• Corpus ID: 14589670

Analysis and Optimization of fastText Linear Text Classifier

@article{Zolotov2017AnalysisAO,
  title={Analysis and Optimization of fastText Linear Text Classifier},
  author={Vladimir Zolotov and David Kung},
  journal={ArXiv},
  year={2017},
  volume={abs/1702.05531}
}
The paper [1] shows that simple linear classifier can compete with complex deep learning algorithms in text classification applications. [] Key Method We proved formally that fastText can be transformed into a simpler equivalent classifier, which unlike fastText does not have any hidden layer. We also proved that the necessary and sufficient dimensionality of the word vector embedding space is exactly the number of document classes. These results help constructing more optimal linear text classifiers with…

Figures from this paper

A Deep Investigation into fastText
  • Jincheng Xu, Qingfeng Du
  • Computer Science
    2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS)
  • 2019
TLDR
A mathematical approach to interpret the behavior of fastText into a score matrix is proposed and the deep investigation into fastText helps to understand the decision making process as well as provides heuristic ideas on how to design more effective text classification models in the future.
Text Classification Based on TextCNN for Power Grid User Fault Repairing Information
  • Yukun Cao, Tian Zhao
  • Computer Science
    2018 5th International Conference on Systems and Informatics (ICSAI)
  • 2018
TLDR
A text classification model based on TextCNN (Text Convolutional Neural Network, TCNN) for fault repairing information of Power grid users is proposed, using CBOW (Continuous Bag-of-Words) to learn the distributed representation of text and obtained by softmax classifier.
Learning supervised embeddings for large scale sequence comparisons
TLDR
Two supervised approaches are introduced—SuperVec and SuperVecX—to learn sequence embeddings that are an order of magnitude faster than BLAST for the database retrieval task, supporting hybrid approaches that rapidly filter the collection so that only potentially relevant records remain.
Context-based sentiment analysis on customer reviews using machine learning linear models
TLDR
The proposed fastText model is a high performance yet cost-effective model for predicting accurate sentiments from large datasets containing customer reviews, that uses the fastText library from Facebook’s AI research (FAIR) Lab, as well as the traditional Linear Support Vector Machine (LSVM) to classify text and word embedding.
Multi-dialect Arabic BERT for Country-level Dialect Identification
TLDR
This paper presents the experiments conducted, and the models developed by the competing team, Mawdoo3 AI, along the way to achieving the winning solution to subtask 1 of the Nuanced Arabic Dialect Identification (NADI) shared task.
Machine Reading of Hypotheses for Organizational Research Reviews and Pre-trained Models via R Shiny App for Non-Programmers
TLDR
This paper introduces natural language processing (NLP) models to accelerate the discovery, extraction, and organization of theoretical developments from social science publications and suggests the next paths to extend the methodology for computer-assisted knowledge synthesis.
Temporal Word Embeddings for Dynamic User Profiling in Twitter
TLDR
The research described in this paper focused on exploring the domain of user profiling, where a set of vectors is used to describe the evolution of a Twitter user’s interests over time through their use of language, and resulted in a novel technique for temporal user profiling.
DeepRelease: Language-agnostic Release Notes Generation from Pull Requests of Open-source Software
TLDR
A deep learning based approach named DeepRelease (Deep learning based Release notes generator) to generate release notes according to pull requests, which is language-agnostic and can be used for projects in any language.
Threat Action Extraction using Information Retrieval
TLDR
This research proposes a novel automatic threat action retrieval system, which collects and analyzes various data sources and develops an improved data preprocessing method to reduce feature dimension and a novel query match algorithm to capture effective threat actions automatically without manually predefined ontology applied by the past research.
Secondary Use of Clinical Problem List Entries for Neural Network-Based Disease Code Assignment
TLDR
This work explored automated coding of 50 character long clinical problem list entries using the International Classification of Diseases (ICD-10) and evaluated three different types of network architectures on the top 100 ICD-10 three-digit codes.
...
1
2
...

References

SHOWING 1-10 OF 12 REFERENCES
Bag of Tricks for Efficient Text Classification
TLDR
A simple and efficient baseline for text classification is explored that shows that the fast text classifier fastText is often on par with deep learning classifiers in terms of accuracy, and many orders of magnitude faster for training and evaluation.
Enriching Word Vectors with Subword Information
TLDR
A new approach based on the skipgram model, where each word is represented as a bag of character n-grams, with words being represented as the sum of these representations, which achieves state-of-the-art performance on word similarity and analogy tasks.
Very Deep Convolutional Networks for Text Classification
TLDR
This work presents a new architecture (VDCNN) for text processing which operates directly at the character level and uses only small convolutions and pooling operations, and is able to show that the performance of this model increases with the depth.
Efficient Character-level Document Classification by Combining Convolution and Recurrent Layers
TLDR
A neural network architecture that utilizes both convolution and recurrent layers to efficiently encode character inputs is proposed and validated on eight large scale document classification tasks and compared with character-level convolution-only models.
Text Understanding from Scratch
TLDR
It is shown that temporal ConvNets can achieve astonishing performance without the knowledge of words, phrases, sentences and any other syntactic or semantic structures with regards to a human language.
Character-level Convolutional Networks for Text Classification
TLDR
This article constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results in text classification.
Very Deep Convolutional Networks for Natural Language Processing
TLDR
This work presents a new architecture for text processing which operates directly on the character level and uses only small convolutions and pooling operations, and is able to show that the performance of this model increases with the depth.
#TagSpace: Semantic Embeddings from Hashtags
TLDR
A convolutional neural network that learns feature representations for short textual posts using hashtags as a supervised signal that outperforms a number of baselines on a document recommendation task and is useful for other tasks as well.
Document Modeling with Gated Recurrent Neural Network for Sentiment Classification
TLDR
A neural network model is introduced to learn vector-based document representation in a unified, bottom-up fashion and dramatically outperforms standard recurrent neural network in document modeling for sentiment classification.
WSABIE: Scaling Up to Large Vocabulary Image Annotation
TLDR
This work proposes a strongly performing method that scales to image annotation datasets by simultaneously learning to optimize precision at the top of the ranked list of annotations for a given image and learning a low-dimensional joint embedding space for both images and annotations.
...
1
2
...