The Rise of Smaller Language Models
Although LLMs are leading the Generative AI adoption in industry, here are some challenges which demands a need for new innovations in language models.
OpenAI’s ChatGPT serves around 200 million user interactions operates on half a million KWH of energy each day, which is over 17,000 times the daily electricity use of an average US household. — New Yorker report
Training a language model with the computational complexity of GPT-3 could require the yearly electricity equivalent of over a thousand households. — University of Washington research
I think we’re at the end of the era where it’s gonna be these giant models, and we’ll make them better in other ways — Sam Altman at MIT’s imagination in action event.
Small Language Models (SLMs) have emerged as alternate to LLMs that address above drawbacks without compromise in quality of output.
What are SLMs:
