Melody-Conditioned Lyrics Generation with SeqGANs

  title={Melody-Conditioned Lyrics Generation with SeqGANs},
  author={Yihao Chen and Alexander Lerch},
  journal={2020 IEEE International Symposium on Multimedia (ISM)},
Automatic lyrics generation has received attention from both music and AI communities for years. Early rule-based approaches have -due to increases in computational power and evolution in data-driven modelsmostly been replaced with deep-learning-based systems. Many existing approaches, however, either rely heavily on prior knowledge in music and lyrics writing or oversimplify the task by largely discarding melodic information and its relationship with the text. We propose an end-to-end melody… Expand

Figures and Tables from this paper

AI-Lyricist: Generating Music and Vocabulary Constrained Lyrics
AI-Lyricist is proposed, a system to generate novel yet meaningful lyrics given a required vocabulary and a MIDI file as inputs, and its superior performance against the state-of-the-art for the proposed tasks is shown. Expand
DeepRapper: Neural Rap Generation with Rhyme and Rhythm Modeling
To the knowledge, DeepRapper is the first system to generate rap with both rhymes and rhythms, and a Transformerbased autoregressive language model which carefully models rh rhyme and rhythms. Expand


Lyrics-Conditioned Neural Melody Generation
By exploiting a large music dataset with 12,197 pairs of English lyrics and melodies, a lyrics-conditioned AI neural melody generation system that consists of three components: lyrics encoder network, melody generation network, and MIDI sequence tuner is developed. Expand
Conditional Rap Lyrics Generation with Denoising Autoencoders
A method for automatically synthesizing a rap verse given an input text written in another form, such as a summary of a news article, to reconstruct rap lyrics from content words is developed. Expand
A Melody-Conditioned Lyrics Language Model
A novel, data-driven language model that produces entire lyrics for a given input melody conditioned on a featurized melody, and experimental results show that the proposed model generates fluent lyrics while maintaining the compatibility between boundaries of lyrics and melody structures. Expand
A Syllable-Structured, Contextually-Based Conditionally Generation of Chinese Lyrics
This work proposes to interpret lyrics-melody alignments as syllable structural information and use a multi-channel sequence-to-sequence model with considering both phrasal structures and semantics and demonstrates the effectiveness of the proposed lyrics generation model. Expand
A Hierarchical Attention Based Seq2seq Model for Chinese Lyrics Generation
Results of automatic and human evaluations demonstrate that the proposed hierarchical attention based Seq2Seq (Sequence-to-Sequence) model is able to compose complete Chinese lyrics with one united topic constraint. Expand
Neural Text Generation: Past, Present and Beyond
The recently proposed methods for text generation based on reinforcement learning, re-parametrization tricks and generative adversarial nets (GAN) techniques are introduced and compared to handle their common problems such as gradient vanishing and generation diversity. Expand
Chinese Poetry Generation with Recurrent Neural Networks
A model for Chinese poem generation based on recurrent neural networks which is ideally suited to capturing poetic content and form is proposed which outperforms competitive Chinese poetry generation systems using both automatic and manual evaluation methods. Expand
GhostWriter: Using an LSTM for Automatic Rap Lyric Generation
This paper demonstrates the effectiveness of a Long Short-Term Memory language model in the initial efforts to generate unconstrained rap lyrics, which produces better “ghostwritten” lyrics than a baseline model. Expand
How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?
This paper presents a critique of scheduled sampling, a state-of-the-art training method that contributed to the winning entry to the MSCOCO image captioning benchmark in 2015, and presents the first theoretical analysis that explains why adversarial training tends to produce samples with higher perceived quality. Expand
Adversarial Learning for Neural Dialogue Generation
This work applies adversarial training to open-domain dialogue generation, training a system to produce sequences that are indistinguishable from human-generated dialogue utterances, and investigates models for adversarial evaluation that uses success in fooling an adversary as a dialogue evaluation metric, while avoiding a number of potential pitfalls. Expand