Decoding strategies for text generation and their use-cases

A practitioner guide for when to use different text decoding strategies.

Free stock image from Canva by Author

If you have worked with text generation models you would have encountered several decoding strategies like Greedy Search, Beam Search, Diverse beam search, Topk Sampling, TopP sampling, TopKP sampling.

Here is a guide on when to use each of these:

TLDR:

Greedy, Beam Search: Translation, Summarization

Diverse Beam Search: Paraphrasing, Image Captioning.

Constrained beam search: Smart paraphrasing, Copywriting, SEO

Topk, TopP, TopKP Sampling: Creative writing.

Image by author

1. Greedy Search:

Greedy search always selects the word with the highest probability as its next word.

💡 Usecases: Translation, Summarization.

If you have a high-quality translator all you need to do is convert a sentence from one language to another.