Why is 4bit llama slower on a 32GB RAM 3090 windows machine vs. a M1 Pro 32GB ram with llama.cpp?

This page summarizes the projects mentioned and recommended in the original post on /r/LocalLLaMA
Wsl2 GUI Windows Flutter Wsl

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured

  • text-generation-webui

    A Gradio web UI for Large Language Models. Supports transformers, GPTQ, AWQ, EXL2, llama.cpp (GGUF), Llama models.

    • There's a mistake in that doc, as version 2 is supported on Windows 10 version 21H2 or later (right click start, system to confirm) and you'll definitely want that. It basically gives you a GPU accelerated Ubuntu virtual machine inside of Windows. Once you get it set up, then you can just follow Linux instructions to set Oobabooga up (https://github.com/oobabooga/text-generation-webui).

  • wsl2-distro-manager

    A GUI to quickly manage your WSL2 instances

    • https://github.com/bostrot/wsl2-distro-manager/ (scroll down for the link to the releases or the MS Store page)

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • WSL Distro Manager: GUI to quickly manage your WSL2 instances

    1 project | news.ycombinator.com | 21 Oct 2023

  • Stevedore: free and open-source Docker Desktop alternative for Windows

    3 projects | /r/docker | 8 Mar 2022

  • Open with Emacs in Windows

    1 project | /r/emacs | 29 Jul 2021

  • Is WinUI 3 Dead?

    1 project | news.ycombinator.com | 25 May 2024

  • LuaRT 1.8.0 – open-source Windows programming framework for Lua

    2 projects | news.ycombinator.com | 22 May 2024