Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Openai-whisper-cpu Alternatives
Similar projects and alternatives to openai-whisper-cpu
-
text-generation-webui
A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.
-
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
-
buzz
Buzz transcribes and translates audio offline on your personal computer. Powered by OpenAI's Whisper.
-
intel-extension-for-pytorch
A Python package for extending the official PyTorch that can easily obtain performance on Intel platform
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
-
BentoML
The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!
-
kernl
Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackable.
-
-
-
-
whisper-playground
Build real time speech2text web apps using OpenAI's Whisper https://openai.com/blog/whisper/
-
transformer-deploy
Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The number of mentions on this list indicates mentions on common posts plus user suggested alternatives.
Hence, a higher number means a better openai-whisper-cpu alternative or higher similarity.
openai-whisper-cpu reviews and mentions
Posts with mentions or reviews of openai-whisper-cpu.
We have used some of these posts to build our list of alternatives
and similar projects. The last one was on 2023-05-14.
-
How to run Llama 13B with a 6GB graphics card
I feel the same.
For example some stats from Whisper [0] (audio transcoding) show the following for the medium model (see other models in the link):
---
GPU medium fp32 Linear 1.7s
CPU medium fp32 nn.Linear 60.7
CPU medium qint8 (quant) nn.Linear 23.1
---
So the same model runs 35.7 times faster on GPU, and compared to an CPU-optimized model still 13.6.
I was expecting around an order or magnitude of improvement. Then again, I do not know if in the case of this article the entire model was in the GPU, or just a fraction of it (22 layers), which might explain the result.
[0] https://github.com/MiscellaneousStuff/openai-whisper-cpu
-
Whispers AI Modular Future
According to https://github.com/MiscellaneousStuff/openai-whisper-cpu the medium model needs 1.7 seconds to transcribe 30 seconds of audio when run on a GPU.
-
[P] Transcribe any podcast episode in just 1 minute with optimized OpenAI/whisper
There is a very simple method built-in to PyTorch which can give you over 3x speed improvement for the large model, which you could also combine with the method proposed in this post. https://github.com/MiscellaneousStuff/openai-whisper-cpu
-
[D] How to get the fastest PyTorch inference and what is the "best" model serving framework?
For CPU inference, model quantization is a very easy to apply method with great average speedups which is already built-in to PyTorch. For example, I applied dynamic quantization to the OpenAI Whisper model (speech recognition) across a range of model sizes (ranging from tiny which had 39M params to large which had 1.5B params). Refer to the below table for performance increases:
-
[P] OpenAI Whisper - 3x CPU Inference Speedup
GitHub
-
A note from our sponsor - InfluxDB
www.influxdata.com | 23 May 2024
Stats
Basic openai-whisper-cpu repo stats
5
221
10.0
over 1 year ago
MiscellaneousStuff/openai-whisper-cpu is an open source project licensed under MIT License which is an OSI approved license.
The primary programming language of openai-whisper-cpu is Jupyter Notebook.
Popular Comparisons
- openai-whisper-cpu VS intel-extension-for-pytorch
- openai-whisper-cpu VS llama-cpp-python
- openai-whisper-cpu VS whisperX
- openai-whisper-cpu VS FlexGen
- openai-whisper-cpu VS buzz
- openai-whisper-cpu VS kernl
- openai-whisper-cpu VS BentoML
- openai-whisper-cpu VS text-generation-webui
- openai-whisper-cpu VS whisper-playground
- openai-whisper-cpu VS whisper-asr-webservice
Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com