Show HN: Prompts as (WASM) Programs

• aici

  6 1,743 9.9 Rust

  AICI: Prompts as (Wasm) Programs

  

It's only the controller that runs in Wasm, not the inference.

The pyctrl is a just a sample controller, you can write a controller that takes any kind of grammar (eg., a yacc grammar [0] - the python code in that example is only used for glueing).

Llama.cpp grammars were quite slow in my testing (20ms per token or so, compared to 2ms for the yacc grammar referenced above).

[0] https://github.com/microsoft/aici/blob/main/controllers/pyct...

• ollama

  204 64,536 9.9 Go

  Get up and running with Llama 3, Mistral, Gemma, and other large language models.

  

This is great generic idea.

It's also possible to wrap it in something user friendly a'la [0].

[0] https://github.com/ollama/ollama/issues/3019

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• transformers-CFG

  1 55 8.7 Python

  🤗 A specialized library for integrating context-free grammars (CFG) in EBNF with the Hugging Face Transformers

  

Does it support constrained generation during training?

This is what we need for the large language models I am training for health care use cases.

For example, constraining LLM output is currently done by masking, and having this rust based library would enable novel ways to train LLMs.

Relevant papers:

https://github.com/epfl-dlab/transformers-CFG

https://neurips.cc/virtual/2023/poster/70782

• outlines

  33 5,799 9.7 Python

  Structured Text Generation

  

> The most obvious usage of this is forcing a model to output valid JSON

Isn't this something that Outlines [0], Guidance [1] and others [2] already solve much more elegantly?

0. https://github.com/outlines-dev/outlines

1. https://github.com/guidance-ai/guidance

2. https://github.com/sgl-project/sglang

• guidance

  23 17,461 9.8 Jupyter Notebook

  A guidance language for controlling large language models.

  

> The most obvious usage of this is forcing a model to output valid JSON

Isn't this something that Outlines [0], Guidance [1] and others [2] already solve much more elegantly?

0. https://github.com/outlines-dev/outlines

1. https://github.com/guidance-ai/guidance

2. https://github.com/sgl-project/sglang

• sglang

  3 2,393 9.4 Python

  SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

  

> The most obvious usage of this is forcing a model to output valid JSON

Isn't this something that Outlines [0], Guidance [1] and others [2] already solve much more elegantly?

0. https://github.com/outlines-dev/outlines

1. https://github.com/guidance-ai/guidance

2. https://github.com/sgl-project/sglang

• autogen

  32 25,506 9.9 Jupyter Notebook

  A programming framework for agentic AI. Discord: https://aka.ms/autogen-dc. Roadmap: https://aka.ms/autogen-roadmap

  

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project