Project: Text Generation with RNN using TensorFlow and Keras

Let’s create a project for text generation using Recurrent Neural Networks (RNNs) with Python, TensorFlow, and Keras. In this example, we’ll generate text in a similar style to a given input text.

1. Project Setup:

• Create a new Python project or script.
• Install necessary libraries:

pip install tensorflow numpy

2. Data Loading:

• Choose a text dataset for training. For this example, we’ll use the Shakespeare dataset:

from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
import requests

# Download the Shakespeare dataset
url = "https://raw.githubusercontent.com/karpathy/char-rnn/master/data/tinyshakespeare/input.txt"
text = requests.get(url).text

3. Text Preprocessing:

• Tokenize and preprocess the text data:

# Tokenize the text
tokenizer = Tokenizer()
tokenizer.fit_on_texts([text])
total_words = len(tokenizer.word_index) + 1

# Create input sequences and labels
input_sequences = []
for line in text.split('\n'):
    token_list = tokenizer.texts_to_sequences([line])[0]
    for i in range(1, len(token_list)):
        n_gram_sequence = token_list[:i+1]
        input_sequences.append(n_gram_sequence)

# Pad sequences for equal length
max_sequence_length = max([len(seq) for seq in input_sequences])
input_sequences = pad_sequences(input_sequences, maxlen=max_sequence_length, padding='pre')

4. Model Definition:

• Define a simple Recurrent Neural Network (RNN) using TensorFlow and Keras:

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Embedding, LSTM, Dense

model = Sequential()
model.add(Embedding(total_words, 100, input_length=max_sequence_length - 1))
model.add(LSTM(100))
model.add(Dense(total_words, activation='softmax'))

5. Model Compilation:

• Compile the model, specifying the loss function, optimizer, and metrics:

model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

6. Model Training:

• Train the RNN model on the text data:

from tensorflow.keras.utils import to_categorical

X, y = input_sequences[:, :-1], input_sequences[:, -1]
y = to_categorical(y, num_classes=total_words)

model.fit(X, y, epochs=10, verbose=1)

7. Text Generation:

• Use the trained model to generate new text:

seed_text = "To be or not to be"
next_words = 100

for _ in range(next_words):
    token_list = tokenizer.texts_to_sequences([seed_text])[0]
    token_list = pad_sequences([token_list], maxlen=max_sequence_length - 1, padding='pre')
    predicted = model.predict_classes(token_list, verbose=0)
    output_word = ""
    for word, index in tokenizer.word_index.items():
        if index == predicted:
            output_word = word
            break
    seed_text += " " + output_word

print(seed_text)

8. Project Conclusion:

• Summarize the project’s goals, outcomes, and potential improvements.
• Include any insights gained from analyzing the generated text.

This project provides a basic example of text generation using an RNN. You can experiment with different architectures, hyperparameter tuning, and datasets for more creative and complex text generation tasks.