Stolen Probability: A Structural Weakness of Neural Language Models
- David Demeter, Gregory J. Kimmel, Doug Downey
- Computer ScienceAnnual Meeting of the Association for…
- 5 May 2020
This work presents numerical, theoretical and empirical analyses which show that words on the interior of the convex hull in the embedding space have their probability bounded by the probabilities of the word vectors on the hull.
Discovery of highly insecticidal synthetic spinosyn mimics - CAMD enabled de novo design simplifying a complex natural product.
- T. C. Sparks, G. D. Crouse, David Demeter, Geno Samaritoni, Casandra L. McLeod
- BiologyPest Management Science
- 30 October 2018
Simplifying complex natural products: Computer modeling-based design leads to highly insecticidal, chemically simpler synthetic mimics of the spinosyn natural products that are active in the field. ©…
De Novo Design of Potent, Insecticidal Synthetic Mimics of the Spinosyn Macrolide Natural Products
- G. D. Crouse, David Demeter, Geno Samaritoni, Casandra L. McLeod, T. C. Sparks
- Biology, ChemistryScientific Reports
- 20 March 2018
Molecular modeling studies coupled with bioactivity-directed chemical modifications were used to define a less complex, synthetically accessible replacement for the spinosyn tetracycle, leading to the discovery of highly insecticidal analogs.
Just Add Functions: A Neural-Symbolic Language Model
- David Demeter, Doug Downey
- Computer ScienceAAAI Conference on Artificial Intelligence
- 11 December 2019
A general methodology to enhance the inductive bias of NNLMs by incorporating simple functions into a neural architecture to form a hierarchical neural-symbolic language model (NSLM), which explicitly encode symbolic deterministic relationships to form probability distributions over words.
Controlling Global Statistics in Recurrent Neural Network Text Generation
- Thanapon Noraset, David Demeter, Doug Downey
- Computer ScienceAAAI Conference on Artificial Intelligence
- 27 April 2018
A dynamic regularizer that updates as training proceeds, based on the generative behavior of the RNNLMs is presented, which is successful at improving word-level repetition statistics and improves model perplexity when the statistical constraints are $n$-gram statistics taken from a large corpus.
Who’s on First?: Probing the Learning and Representation Capabilities of Language Models on Deterministic Closed Domains
- David Demeter, Doug Downey
- Computer ScienceConference on Computational Natural Language…
- 2021
It is shown that language models of three different architectures can answer questions about world states using only verb-like encodings of activity, which is extensible to new language models and additional question-answering tasks.
CODE: Compiler-based Neuron-aware Ensemble training
- E. Trainiti, Thanapon Noraset, David Demeter, Doug Downey, Simone Campanoni
- Computer ScienceConference on Machine Learning and Systems
- 2021
CODE introduces neuron-level analyses and transformations aimed at identifying and removing redundant computation from the networks that compose the ensemble that enables CODE to train large DNN ensembles in a fraction of the time and memory footprint needed by current techniques.
Learning to Perform Complex Tasks through Compositional Fine-Tuning of Language Models
- Victor S. Bursztyn, David Demeter, Doug Downey, Larry Birnbaum
- Computer ScienceConference on Empirical Methods in Natural…
- 23 October 2022
This work presents compositional fine-tuning (CFT): an approach based on explicitly decomposing a target task into component tasks, and then fine- Tuning smaller LMs on a curriculum of such component tasks.
Testing of the permeable Fe0-Barrier at dumpField Podlipa (Lubietova Cu-Deposit, Slovakia)
- J. Kupka, P. András, J. Dadová, David Demeter
- Environmental Science
- 2015
Material pochodzący z miejsca zrzutu wod kopalni Podlipa w opuszczonym zlozu Cu w Lubietovie wykazuje duzą zawartośc Cu, Fe,
Cd, Co, Mn, Ni oraz Pb. W wiekszości przypadkow wyzsze zawartości metalu…
De Novo Design of Potent, Insecticidal Synthetic Mimics of the Spinosyn Macrolide Natural Products
- G. D. Crouse, David Demeter, Geno Samaritoni, Casandra L. McLeod, T. C. Sparks
- Biology, ChemistryScientific Reports
- 20 March 2018
Molecular modeling studies coupled with bioactivity-directed chemical modifications were used to define a less complex, synthetically accessible replacement for the spinosyn tetracycle, leading to the discovery of highly insecticidal analogs.