aici
sglang
aici | sglang | |
---|---|---|
7 | 3 | |
1,756 | 2,440 | |
7.5% | 16.7% | |
9.9 | 9.4 | |
6 days ago | 3 days ago | |
Rust | Python | |
MIT License | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
aici
-
HonoJS: Small, simple, and ultrafast web framework for the Edges
Have you looked at AICI by Microsoft yet?
https://github.com/microsoft/aici/
-
LLM4Decompile: Decompiling Binary Code with LLM
I have been planning to work on something like this. I think that eventually, someone will crack the "binary in -> good source code out of LLM" pipeline but we are probably a few years away from that still. I say a few years because I don't think there's a huge pile of money sitting at the end of this problem, but maybe I'm wrong.
A really good "stop-gap" approach would be to build a decompilation pipeline using Ghidra in headless mode and then combine the strict syntax correctness of a decompiler with the "intuition/system 1 skills" of an LLM. My inspiration for this setup comes from two recent advancements, both shared here on HN:
1. AlphaGeometry: The Decompiler and the LLM should complement each other, covering each other's weaknesses. https://deepmind.google/discover/blog/alphageometry-an-olymp...
2. AICI: We need a better way of "hacking" on top of these models, and being able to use something like AICI as the "glue" to coordinate the generation of C source. I don't really want the weights of my LLM to be used to generate syntactically correct C source, I want the LLM to think in terms of variable names, "snippet patterns" and architectural choices while other tools (Ghidra, LLVM) worry about the rest. https://github.com/microsoft/aici
Obviously this is all hand-wavey armchair commentary from a former grad student who just thinks this stuff is cool. Huge props to these researchers for diving into this. I know the authors already mentioned incorporating Ghidra into their future work, so I know they're on the right track.
-
Show HN: Prompts as (WASM) Programs
We believe Guidance can run on top of AICI (we're working on efficient Earley parser for that [0], together with local Guidance folks). AICI is generally lower level (though our sample controllers are at similar level to Guidance).
[0] https://github.com/microsoft/aici/blob/main/controllers/aici...
- AI Controller Interface (AICI)
sglang
-
Show HN: Prompts as (WASM) Programs
> The most obvious usage of this is forcing a model to output valid JSON
Isn't this something that Outlines [0], Guidance [1] and others [2] already solve much more elegantly?
0. https://github.com/outlines-dev/outlines
1. https://github.com/guidance-ai/guidance
2. https://github.com/sgl-project/sglang
-
AMD ROCm Software Blogs
Thanks! Again, partnerships over customers. If you're experienced and have the technical chops to make a MI300x sing, we want to work with you. Our model is that we are the capex/opex investor for businesses. As much as I love software, Hot Aisle is more of a hardware business. Running super high end large scale compute is an extreme challenge in itself. We are less interested in building the software side of things and want to foster those who can focus on that side.
https://github.com/unslothai/unsloth/issues/160
https://github.com/search?q=repo%3Apredibase%2Florax+rocm&ty...
https://github.com/sgl-project/sglang/issues/157
https://github.com/casper-hansen/AutoAWQ (supports rocm)
- FLaNK Stack Weekly 22 January 2024
What are some alternatives?
transformers-CFG - 🤗 A specialized library for integrating context-free grammars (CFG) in EBNF with the Hugging Face Transformers
Mermaid - Edit, preview and share mermaid charts/diagrams. New implementation of the live editor.
ghidra_tools - A collection of Ghidra scripts, including the GPT-3 powered code analyser and annotator, G-3PO.
spotube - 🎧 Open source Spotify client that doesn't require Premium nor uses Electron! Available for both desktop & mobile!
pingora - A library for building fast, reliable and evolvable network services.
excalidraw-claymate - A tool based on Excalidraw to create stop motion animations and slides.
Awesome-LLM-Productization - Awesome-LLM-Productization: a curated list of tools/tricks/news/regulations about AI and Large Language Model (LLM) productization
flink-kubernetes-operator - Apache Flink Kubernetes Operator
deepcompyle - Pretraining transformers to decompile Python bytecodes
TinyLlama - The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.