What are LLMs? An intro into AI, models, tokens, parameters, weights, quantization and more

• TinyLlama

  14 6,915 8.7 Python

  The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

Small models: Less than ~1B parameters. TinyLlama and tinydolphin are examples of small models.

• gemma

  3 2,057 5.6 Jupyter Notebook

  Open weights LLM from Google DeepMind.

  

Medium models: Roughly between 1B to 10B parameters. This is where Mistral 7B, Phi-3, Gemma from Google DeepMind, and wizardlm2 sit. Fun fact: GPT 2 was a medium sized model, much smaller than its latest versions.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• gpt-2

  64 21,259 2.5 Python

  Code for the paper "Language Models are Unsupervised Multitask Learners"

  

Medium models: Roughly between 1B to 10B parameters. This is where Mistral 7B, Phi-3, Gemma from Google DeepMind, and wizardlm2 sit. Fun fact: GPT 2 was a medium sized model, much smaller than its latest versions.

• gpt-3

  41 9,406 3.5

  Discontinued GPT-3: Language Models are Few-Shot Learners

  

Large models: Everything above 10B of parameters. This is where Llama 3, Llama 2, Mistral 8x22B, GPT 3, and most likely GPT 4 sit.

Suggest a related project