Python Tokenization

Open-source Python projects categorized as Tokenization
Edit details
Topics: Python NLP Natural Language Processing Machine Learning Artificial intelligence
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

Top 6 Python Tokenization Projects

  • spaCy

    💫 Industrial-strength Natural Language Processing (NLP) in Python

    • Project mention: How I discovered Named Entity Recognition while trying to remove gibberish from a string. | dev.to | 2024-05-06

  • trankit

    Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing

  • Scout Monitoring

    Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

    Scout Monitoring logo

  • razdel

    Rule-based token, sentence segmentation for Russian language

  • simplemma

    Simple multilingual lemmatizer for Python, especially useful for speed and efficiency

  • python-fpe

    FPE - Format Preserving Encryption with FF3 in Python

  • xontrib-output-search

    Get identifiers, paths, URLs and words from the previous command output and use them for the next command in xonsh shell.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Python Tokenization related posts

  • Cdk8s: CNCF-Backed Infrastructure-as-Code (IaC) for Kubernetes

    2 projects | news.ycombinator.com | 6 Sep 2022

  • Road Plan For Ravencoin?

    2 projects | /r/Ravencoin | 3 Jun 2021

  • resources on how to develop a token?

    2 projects | /r/CryptoCurrency | 26 Apr 2021

  • Dev Meeting Transcript (March 26, 2021)

    3 projects | /r/Ravencoin | 26 Mar 2021

  • Document sur les outils de la finance décentralisai (DEFI) en anglais.

    2 projects | /r/vosfinances | 13 Feb 2021
  • A note from our sponsor - Scout Monitoring
    www.scoutapm.com | 1 Jun 2024
    Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today. Learn more →

Index

What are some of the best open-source Tokenization projects in Python? This list will help you:

Project Stars
1 spaCy 28,934
2 trankit 716
3 razdel 248
4 simplemma 130
5 python-fpe 80
6 xontrib-output-search 36

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com