okuvshynov / llama_duoLinks
asynchronous/distributed speculative evaluation for llama3
☆39Updated last year
Alternatives and similar repositories for llama_duo
Users that are interested in llama_duo are comparing it to the libraries listed below
Sorting:
- Inference of Mamba models in pure C☆191Updated last year
- The Finite Field Assembly Programming Language☆36Updated 3 months ago
- A minimalistic C++ Jinja templating engine for LLM chat templates☆170Updated 3 weeks ago
- Thin wrapper around GGML to make life easier☆40Updated 2 months ago
- Samples of good AI generated CUDA kernels☆89Updated 3 months ago
- High-Performance SGEMM on CUDA devices☆97Updated 7 months ago
- GGUF implementation in C as a library and a tools CLI program☆284Updated this week
- iterate quickly with llama.cpp hot reloading. use the llama.cpp bindings with bun.sh☆51Updated last year
- Python bindings for ggml☆146Updated 11 months ago
- ☆54Updated last year
- LLM training in simple, raw C/CUDA☆104Updated last year
- Fast and vectorizable algorithms for searching in a vector of sorted floating point numbers☆146Updated 8 months ago
- Prepare for DeekSeek R1 inference: Benchmark CPU, DRAM, SSD, iGPU, GPU, ... with efficient code.☆73Updated 6 months ago
- Lightweight Llama 3 8B Inference Engine in CUDA C☆48Updated 5 months ago
- Simple high-throughput inference library☆127Updated 3 months ago
- GGML implementation of BERT model with Python bindings and quantization.☆56Updated last year
- Custom PTX Instruction Benchmark☆126Updated 6 months ago
- Inference Vision Transformer (ViT) in plain C/C++ with ggml☆293Updated last year
- Train your own small bitnet model☆75Updated 10 months ago
- C API for MLX☆124Updated last month
- llama.cpp to PyTorch Converter☆34Updated last year
- Inference RWKV v7 in pure C.☆38Updated this week
- General purpose GPU compute framework built on Vulkan to support 1000s of cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). …☆52Updated 6 months ago
- tinygrad port of the RWKV large language model.☆45Updated 5 months ago
- Gpu benchmark☆66Updated 7 months ago
- GPT2 implementation in C++ using Ort☆26Updated 4 years ago
- Editor with LLM generation tree exploration☆73Updated 6 months ago
- Tiny Dream - An embedded, Header Only, Stable Diffusion C++ implementation☆265Updated last year
- Multi-Threaded FP32 Matrix Multiplication on x86 CPUs☆351Updated 4 months ago
- SIMD quantization kernels☆83Updated last week