The architecture eliminates the need to decompose large models for distributed computing to train: Push-button AI? The hottest trend in AI is the emergence of massive models such as Open AI’s GPT-3.

Up until late 2024, no one has been able to massively increase the amount of compute dedicated to a single model beyond the OpenAI GPT 4 model level. This information is from semianalysis and EIA.