LLMTest_NeedleInAHaystack VS Lobe Chat

This research follows the “haystack test” Greg Kamradt published when the update GPT-4 came out (twitter, code). That test provided useful insight into (the lack of) context recall performance. But it was performed on a very small sample test (limiting its statistical significance) and was initially limited to GPT-4 (he has since published an updated version that also uses Claude 2.1). Moreover, the test data consists of essays that were likely already used pretraining LLMs, and the results were evaluated by GPT-4, potentially introducing confounding variables into the mix.

Get your OpenAI API key and then use it on one of the hundreds of open source frontends available, such as: https://github.com/lobehub/lobe-chat

The workbench UI sucks, what's nifty about it? It's cumbersome and slow. I would recommend using a ChatUI (huggingface ChatUI, or https://github.com/lobehub/lobe-chat) and use the API that way.

LobeChat is a open-source, extensible (Function Calling), high-performance chatbot framework. It supports one-click free deployment of your private ChatGPT/LLM web application.By building a powerful plugin ecosystem, ChatGPT not only can provide real-time news updates, but it can also assist you in easily querying documents and accessing various e-commerce data. This allows ChatGPT to play a key role in a wider range of fields. If you are interested in writing plugins, itprovide detailed component development documentation, SDKs, and template files.