Seq2Seq Explained With a Coffee Shop

A customer gives the cashier a complicated coffee order.

The cashier turns the words into numbers. The shift lead reads those numbers one at a time and squeezes the entire order into a single hidden state. The barista receives only that hidden state and reconstructs the order one word at a time.

That is the core idea behind sequence-to-sequence learning:

source sequence → encoder → hidden state → decoder → target sequence

In this lesson, we rebuild the basic architecture behind Sutskever, Vinyals, and Le’s landmark 2014 seq2seq paper using a tiny PyTorch model you can run on your own computer.

What You’ll Learn

By the end of this lesson, you should understand:

• how words become integer token IDs
• how an embedding layer turns IDs into vectors
• how an LSTM reads a sentence one token at a time
• what the hidden state represents
• why the encoder compresses the source sentence
• how the decoder produces an output sequence one token at a time
• why the source sentence is reversed in the 2014 model
• what teacher forcing means
• the difference between training and inference
• how an EOS token tells the decoder to stop
• how next-word probabilities produce the final output

The Coffee-Shop Model

We use two parallel explanations throughout the lesson.

The metaphor

customer’s note
→ cashier
→ shift lead’s cup markings
→ barista
→ spoken callout

The neural network

source tokens
→ token IDs
→ input embeddings
→ encoder LSTM
→ hidden state
→ decoder LSTM
→ output tokens

The note is the source sequence.

The cup markings are our human-readable metaphor for the hidden state.

The barista’s callout is the target sequence.

The Demo

The code trains two tiny translators.

Barista translator

two_shot half_caf vanilla coffee
→ half-caf double vanilla americano

The customer’s ordinary order becomes compact barista-speak.

Spanish-to-English translator

yo quiero tacos
→ i want tacos

This uses the same architecture. Only the training data changes.

Run the Lesson

Run both demonstrations:

python lesson_04_seq2seq.py

Run only the barista example:

python lesson_04_seq2seq.py --dataset barista

Run only Spanish-to-English:

python lesson_04_seq2seq.py --dataset spanish_english

Skip the pauses:

python lesson_04_seq2seq.py --fast

Change the number of training epochs:

python lesson_04_seq2seq.py --epochs 1200

What to Watch For

As the program runs, notice the four major steps:

1. The vocabulary maps each word to an integer ID.
2. The embedding layer turns each ID into a learned vector.
3. The encoder LSTM compresses the source sequence into its final hidden state.
4. The decoder LSTM predicts the target sequence one word at a time.

During decoding, the program also prints the model’s top three candidates for each next word.

That lets you watch the barista think.

Your Assignment

Do not merely run the code once and move on.

Change something.

Try:

• adding a new drink
• adding a new syrup
• adding another Spanish phrase
• reducing the training epochs
• increasing the hidden-state size
• removing source reversal
• changing the random seed
• testing an unseen combination of known words

Break the model. Fix it. Make it yours.

The One-Sentence Takeaway

Seq2seq reads one ordered sequence into a hidden state, then decodes that state into another ordered sequence.

Or, in coffee-shop language:

The cashier reads the note, the shift lead marks the cup, and the barista calls out the drink.

Where This Leads

The weakness of this original architecture is the bottleneck.

The encoder must squeeze the entire source sentence into one fixed hidden state. That becomes increasingly difficult as sentences grow longer.

The next breakthrough is attention:

Instead of forcing the decoder to remember everything from one vector, let it look back at the source whenever it needs to.

That is the next brick.