New Open-source LLMs! 🤯 The Falcon has landed! 7B and 40B

• sparsegpt

  16 650 3.6 Python

  Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

  

There is this : https://github.com/IST-DASLab/sparsegpt

• trl

  13 8,412 9.6 Python

  Train transformer language models with reinforcement learning.

  

For lora - PEFT seems to work. I don't have patience to wait 5 hours, but modifying this example seems to work. You don't even need to modify that much, as their model just as neo-x uses query_key_value name for self-attention.

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

Suggest a related project