okuvshynov / llama_duoLinks
asynchronous/distributed speculative evaluation for llama3
☆38Updated last year
Alternatives and similar repositories for llama_duo
Users that are interested in llama_duo are comparing it to the libraries listed below
Sorting:
- Inference of Mamba models in pure C☆192Updated last year
- A minimalistic C++ Jinja templating engine for LLM chat templates☆193Updated last month
- Python bindings for ggml☆146Updated last year
- High-Performance SGEMM on CUDA devices☆107Updated 9 months ago
- GGML implementation of BERT model with Python bindings and quantization.☆55Updated last year
- iterate quickly with llama.cpp hot reloading. use the llama.cpp bindings with bun.sh☆50Updated 2 years ago
- Experiments with BitNet inference on CPU☆54Updated last year
- The Finite Field Assembly Programming Language☆36Updated 5 months ago
- LLM training in simple, raw C/CUDA☆107Updated last year
- A faithful clone of Karpathy's llama2.c (one file inference, zero dependency) but fully functional with LLaMA 3 8B base and instruct mode…☆139Updated last week
- General purpose GPU compute framework built on Vulkan to support 1000s of cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). …☆52Updated 8 months ago
- Thin wrapper around GGML to make life easier☆40Updated 4 months ago
- Inference RWKV v7 in pure C.☆40Updated 3 weeks ago
- tinygrad port of the RWKV large language model.☆44Updated 7 months ago
- Custom PTX Instruction Benchmark☆131Updated 8 months ago
- GPT2 implementation in C++ using Ort☆26Updated 4 years ago
- GGUF implementation in C as a library and a tools CLI program☆292Updated 2 months ago
- Advanced Ultra-Low Bitrate Compression Techniques for the LLaMA Family of LLMs☆110Updated last year
- Inference Vision Transformer (ViT) in plain C/C++ with ggml☆295Updated last year
- Lightweight Llama 3 8B Inference Engine in CUDA C☆48Updated 7 months ago
- LLM-based code completion engine☆190Updated 9 months ago
- C API for MLX☆144Updated last month
- minimal C implementation of speculative decoding based on llama2.c☆25Updated last year
- Fast and vectorizable algorithms for searching in a vector of sorted floating point numbers☆152Updated 10 months ago
- RWKV in nanoGPT style☆193Updated last year
- llama.cpp to PyTorch Converter☆34Updated last year
- Stable Diffusion in pure C/C++☆60Updated 2 years ago
- Course Project for COMP4471 on RWKV☆17Updated last year
- ctypes wrappers for HIP, CUDA, and OpenCL☆130Updated last year
- GGUF parser in Python☆28Updated last year