alpaca_eval
alpaca_farm
alpaca_eval | alpaca_farm | |
---|---|---|
4 | 1 | |
1,160 | 718 | |
12.0% | 2.4% | |
9.6 | 7.5 | |
3 days ago | 3 months ago | |
Jupyter Notebook | Python | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
alpaca_eval
-
UltraLM-13B reaches top of AlpacaEval leaderboard
Alpaca Eval is open source and was developed by the same team who trained the alpaca model afaik. It is not like what you said in the other comment
-
[P] AlpacaEval : An Automatic Evaluator for Instruction-following Language Models
I have been going deep in this space for my can-ai-code project and was looking at the config that WizardLM was run with: https://github.com/tatsu-lab/alpaca_eval/blob/main/src/alpaca_eval/models_configs/wizardlm-13b/configs.yaml
an automatic evaluator that is easy to use, fast, cheap and validated against 20K human annotations. It actually has a higher agreement with majority vote of humans than a single human annotator! Of course, our method still has limitations which we discuss here!
alpaca_farm
-
[P] AlpacaEval : An Automatic Evaluator for Instruction-following Language Models
AlpacaEval dataset: 805 instructions, which are a simplification of AlpacaFarm's evaluation set.
What are some alternatives?
Local-LLM-Comparison-Colab-UI - Compare the performance of different LLM that can be deployed locally on consumer hardware. Run yourself with Colab WebUI.
HugNLP - CIKM2023 Best Demo Paper Award. HugNLP is a unified and comprehensive NLP library based on HuggingFace Transformer. Please hugging for NLP now!π
DeepLearningExamples - State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.
Awesome-Multimodal-Large-Language-Models - :sparkles::sparkles:Latest Papers and Datasets on Multimodal Large Language Models, and Their Evaluation.
FinGPT - FinGPT: Open-Source Financial Large Language Models! Revolutionize π₯ We release the trained model on HuggingFace.
MOSS - An open-source tool-augmented conversational language model from Fudan University
llm-search - Querying local documents, powered by LLM
camel - π« CAMEL: Communicative Agents for βMindβ Exploration of Large Scale Language Model Society [Moved to: https://github.com/camel-ai/camel]
medmcqa - A large-scale (194k), Multiple-Choice Question Answering (MCQA) dataset designed to address realworld medical entrance exam questions.
language-planner - Official Code for "Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents"
UltraChat - Large-scale, Informative, and Diverse Multi-round Chat Data (and Models)