anthropic-tokenizer
Approximation of the Claude 3 tokenizer by inspecting generation stream (by javirandor)
anthropic-tokenizer | anthropic-tokenizer-typescript | |
---|---|---|
3 | 1 | |
87 | 45 | |
- | - | |
7.8 | 2.9 | |
6 days ago | 4 months ago | |
Python | TypeScript | |
MIT License | MIT License |
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
anthropic-tokenizer
Posts with mentions or reviews of anthropic-tokenizer.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2024-06-17.
anthropic-tokenizer-typescript
Posts with mentions or reviews of anthropic-tokenizer-typescript.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2024-06-17.
-
Show HN: Token price calculator for 400+ LLMs
Anthropic actually has a Claude 3 tokenizer tucked away in one of their repos: https://github.com/anthropics/anthropic-tokenizer-typescript
At this moment, Tokencost uses the OpenAI tokenizer as a default tokenizer, but this would be a welcome PR!
What are some alternatives?
When comparing anthropic-tokenizer and anthropic-tokenizer-typescript you can also consider the following projects:
tokencost - Easy token price estimates for 400+ LLMs