Google GLaM

The Generalist Language Model (GLaM), developed by Google, is a cutting-edge mixture of experts (MoE) model intended for a variety of inputs. This structure consists of a number of specialised submodels, or experts, each of which is designed with a particular data type in mind. A gating network controls how the model behaves by locating and activating the two experts who are most appropriate for each token (such as words or word fragments).

The extensive GLaM model strategically uses a 97 billion-parameter subnetwork (roughly 8% of the full capacity) for each token prediction during the inference phase, despite having a whopping 1.2 trillion parameters spread across 64 experts for each of its 32 MoE layers.

User objects:

– NLP researchers

– AI developers

– Data scientists

– Tech companies focusing on language processing

– Academics in linguistics and computational linguistics

– Search engine developers

– Content recommendation system developers

– Chatbot developers

>>> Please use: ChatGPT without login for Free

DEMO

Previous Next