• Corpus ID: 221266537

Translating Paintings Into Music Using Neural Networks

  title={Translating Paintings Into Music Using Neural Networks},
  author={Prateek Verma and Constantin Basica and Pamela Davis Kivelson},
We propose a system that learns from artistic pairings of music and corresponding album cover art. The goal is to 'translate' paintings into music and, in further stages of development, the converse. We aim to deploy this system as an artistic tool for real time 'translations' between musicians and painters. The system's outputs serve as elements to be employed in a joint live performance of music and painting, or as generative material to be used by the artists as inspiration for their… 

Figures from this paper


A Neural Algorithm of Artistic Style
This work introduces an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality and offers a path forward to an algorithmic understanding of how humans create and perceive artistic imagery.
Multimodal Deep Learning for Music Genre Classification
An approach to learn and combine multimodal data representations for music genre classification is proposed, and a proposed approach for dimensionality reduction of target labels yields major improvements in multi-label classification.
Conditional End-to-End Audio Transforms
An end-to-end method for transforming audio from one style to another based on convolutional and hierarchical recurrent neural networks, designed to capture long-term acoustic dependencies, requires minimal post-processing, and produces realistic audio transforms.
Neuralogram: A Deep Neural Network Based Representation for Audio Signals
The Neuralogram is proposed -- a deep neural network based representation for understanding audio signals which transforms an audio signal to a dense, compact representation based upon embeddings learned via a neural architecture.
Real-time Melodic Accompaniment System for Indian Music Using TMS320C6713
An instrumental accompaniment system for Indian classical vocal music is designed and implemented on a Texas Instruments Digital Signal Processor TMS320C6713. This will act as a virtual accompanist
Look, Listen and Learn
There is a valuable, but so far untapped, source of information contained in the video itself – the correspondence between the visual and the audio streams, and a novel “Audio-Visual Correspondence” learning task that makes use of this.
Audio-linguistic Embeddings for Spoken Sentences
This work illustrates the viability of generic, multi-modal sentence embeddings for spoken language understanding and learns long-term dependencies by modeling speech at the sentence level as an audio-linguistic multitask learning problem.
Audio Set: An ontology and human-labeled dataset for audio events
The creation of Audio Set is described, a large-scale dataset of manually-annotated audio events that endeavors to bridge the gap in data availability between image and audio research and substantially stimulate the development of high-performance audio event recognizers.
The Million Song Dataset
The Million Song Dataset, a freely-available collection of audio features and metadata for a million contemporary popular music tracks, is introduced and positive results on year prediction are shown, and the future development of the dataset is discussed.
Densely Connected Convolutional Networks
The Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion, and has several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters.