Developer Tools
Alternative Language Models
Francesco ChiaramonteFrancesco Chiaramonte
Home   >   DeepMind RETRO

With the help of a large database containing 2 trillion tokens, DeepMind’s RETRO employs a novel method to enhance language models. By obtaining document segments from a large corpus that are chosen based on their similarity to recent input tokens, this method improves auto-regressive language models.

Remarkably, despite using 25 times fewer parameters than advanced models like GPT-3 and Jurassic-1, RETRO performs on par with them on the Pile benchmark. When further developed, RETRO’s abilities can handle more challenging jobs, especially those requiring a depth of knowledge, like answering questions. RETRO essentially shows the effectiveness of utilising massive data retrieval to improve language model performance.

Researchers, data scientists, developers, and professionals in knowledge-intensive fields can use DeepMind’s RETRO.

>>> Use ChatGPT Free Online to make your work more convenient



Francesco Chiaramonte

Francesco Chiaramonte is renowned for over 10 years of experience, from machine learning to AI entrepreneurship. He shares knowledge and is committed to advancing artificial intelligence, hoping that AI will drive societal progress.

Similar Apps

Openai Codex

Alternative Language Models

nanoGPT minGPT

Alternative Language Models


Alternative Language Models

DeepMind RETRO

Alternative Language Models