Efficient LLM inference solution on Intel GPU

• intel-extension-for-pytorch

  16 1,403 9.7 Python

  A Python package for extending the official PyTorch that can easily obtain performance on Intel platform

  

OK I found it. Looks like they use SYCL (which for some reason they've rebranded to DPC++): https://github.com/intel/intel-extension-for-pytorch/tree/v2...

• llama.cpp

  788 59,389 10.0 C++

  LLM inference in C/C++

• Scout Monitoring

  www.scoutapm.com featured

  Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.

  

• Cgml

  23 40 8.6 C++

  GPU-targeted vendor-agnostic AI library for Windows, and Mistral model implementation.

Suggest a related project