openai-messages-token-helper

A utility library for dealing with token counting for messages sent to an LLM (currently OpenAI models only) (by pamelafox)

Openai-messages-token-helper Alternatives

Similar projects and alternatives to openai-messages-token-helper

openai-messages-token-helper
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a better openai-messages-token-helper alternative or higher similarity.

openai-messages-token-helper discussion

Log in or Post with

openai-messages-token-helper reviews and mentions

Posts with mentions or reviews of openai-messages-token-helper. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2024-06-17.
  • Show HN: Token price calculator for 400+ LLMs
    12 projects | news.ycombinator.com | 17 Jun 2024
    I grappled with that issue for https://github.com/pamelafox/openai-messages-token-helper as I wanted to be able to use it for a quick token check with SLMs as well, so I ended up adding a parameter "fallback_to_default" for developers to indicate they're okay with assuming gpt-35 BPE encoding.
  • Lessons after a Half-billion GPT Tokens
    2 projects | news.ycombinator.com | 13 Apr 2024
    Lol, nice truncation logic! If anyone’s looking for something slightly fancier, I made a micro-package for our tiktoken-based truncation here: https://github.com/pamelafox/llm-messages-token-helper

Stats

Basic openai-messages-token-helper repo stats
3
14
7.6
25 days ago

pamelafox/openai-messages-token-helper is an open source project licensed under MIT License which is an OSI approved license.

The primary programming language of openai-messages-token-helper is Python.


Sponsored
Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com