Lyrics Generator Twitter Bot

This repository contains code to load a GPT-2 model, perform text generation and create a Twitter Bot that interact with Twitter users when it is mentioned.

Examples

I fine-tuned 2 small GPT-2 models (124M parameters) to generate Eminem lyrics as well as Storytelling lyrics. The following samples correspond to the outputs of such models:

Eminem Bot Lyrics (@rap_god_bot)

Music Storytelling Bot Lyrics (@musicstorytell)

How to run the code

1. Make sure you have python 3 and pip installed:

sudo apt-get update
sudo apt-get install python3
sudo apt install python3-pip

2. Package will be installed if you write:

script/setup

3. Download the example model (a small 124M parameters pre-trained GPT-2 model) or place yours under models/ folder:

python3 download_example_model.py 

4. Open the file run_twitter_bot.py and modify it with your model file path and twitter user information.

5. Modify the authentication files to match your user / keys. More info on how to create a Twitter development account in the section below.

Cool Stuff that I Used/Learned Doing the Project

GPT-2 and Transformer Models

This year, OpenAI released a new set of language generation models: GPT-2. These large-scale unsupervised language models were able to generate coherent paragraphs of text while achieving state-of-the-art performance on many language modeling benchmarks.

For storage and memory purposes I decided to fine-tune the smallest one (124M) though the 355M was genenerating more diverse outputs.

These are some resources that I used:

• Understand the Transformer architecture: [Paper], [Blog_1], [Blog_2]
• Understand GPT-2: [Paper], [Blog_1], [Blog_2]
• Understand code, download models and fine-tune them: [gpt-2-simple], [OpenAI]

LyricsGenious Package

In order to download all the song lyrics that I used to fine-tune the GPT-2 model, I used a great library called LyricsGenious. This package offers a really clean interface that interacts with the Genious API and makes easy the download of lyrics.

import lyricsgenius
genius = lyricsgenius.Genius("my_client_access_token_here")
artist = genius.search_artist("Eminem", max_songs=10, sort="title")
print(artist.songs)

Twitter API and Tweepy

Tweepy is an easy-to-use Python library for accessing and interacting with the Twitter API. Getting started is as simple as:

import tweepy

# Authenticate to Twitter
auth = tweepy.OAuthHandler("CONSUMER_KEY", "CONSUMER_SECRET")
auth.set_access_token("ACCESS_TOKEN", "ACCESS_TOKEN_SECRET")

# Create API object
api = tweepy.API(auth)

# Create a tweet
api.update_status("Hello Tweepy")

To use the Twitter API you will need to create a developer account. This post was very useful to understand the API calls, functions and details that I needed to create my own Twitter Bot.

Swap files

I used the Free Tier of Amazon EC2 instances to deploy the models. Even though they were the smallest GPT-2 models, they weren't fitting on RAM memory. The solution I opted for was creating a Swap space in the system.

Swap is a space on a disk that is used when the amount of physical RAM memory is full. When a Linux system runs out of RAM, inactive pages are moved from the RAM to the swap space.

I used the following code to allocate 2GB of space:

# Create a file which will be used for swap
sudo fallocate -l 2G /swapfile

# Set the correct permissions
sudo chmod 600 /swapfile

# Set up a Linux swap area
sudo mkswap /swapfile

# Enable the swap
sudo swapon /swapfile

# Verify the swap status
sudo swapon --show

Check this blogpost for more information.

Using PIL to Print Text in Images

I created a function to create an image and draw text on it using PIL.

def print_text_in_image(text, font_path='Pillow/Tests/fonts/FreeMono.ttf', image_color=(255, 255, 225)):
    # Create a blank image
    # image = np.uint8(np.ones((1100, 1000, 3)) * 255)
    image = np.ones((1100, 1000, 3))

    # Give some color to the base image
    image[:, :, 0] *= image_color[0]
    image[:, :, 1] *= image_color[1]
    image[:, :, 2] *= image_color[2]
    image = np.uint8(image)

    # Create a PIL Image
    pil_image = Image.fromarray(image, 'RGB')

    font = ImageFont.truetype(font_path, 40)

    # Get a drawing context
    d = ImageDraw.Draw(pil_image)

    # Margins
    vertical_coord = 50
    horizontal_margin = 50

    # Draw text, full opacity
    for sentence in text:
        d.text((horizontal_margin, vertical_coord), sentence, font=font, fill=(0, 0, 0, 255))
        vertical_coord += 40
    return pil_image

Sending E-mails with Python

I wanted to set up an automatic e-mail messaging service so every time the bot is down I could get a notification. I ended up using SMTP_SSL and a gmail account:

import ssl
import json
import smtplib


class EmailSender:
    def __init__(self, authentication_json_path):
        with open(authentication_json_path, 'r') as f:
            authentication_params = json.load(f)

        self.password = authentication_params['PASSWORD']
        self.email = authentication_params['EMAIL']

        # Port for SSL
        self.port = 465

        # Create a secure SSL context
        self.context = ssl.create_default_context()

    def send_email(self, email_receiver, message):
        with smtplib.SMTP_SSL("smtp.gmail.com", self.port, context=self.context) as server:
            server.login(self.email, self.password)
            server.sendmail(self.email, email_receiver, message)

More information about how to set an e-mail service can be found here.