[D] Is all the talk about what GPT can do on Twitter and Reddit exaggerated or fairly accurate?

• JARVIS

  52 23,193 7.2 Python

  JARVIS, a system to connect LLMs with ML community. Paper: https://arxiv.org/pdf/2303.17580.pdf

  

https://github.com/microsoft/JARVIS this has that capability as far as I know, I don’t know the individual hugging face model it utilizes for whatever task because there are a ton

• guardrails

  13 3,444 9.9 Python

  Adding guardrails to large language models.

  

not vouching for it, but I know this is at least a thing that exists and I like the general idea: https://github.com/shreyar/guardrails

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• TruthfulQA

  4 517 2.8 Jupyter Notebook

  TruthfulQA: Measuring How Models Imitate Human Falsehoods

I agree they show that you can brute-force mimick uncertainty estimates to some degree, and that the model is generally well calibrated (though on what is basically a set of trivia questions, so YMMV)... yet:

Suggest a related project