gf712 / gpt2-cpp
GPT2 implementation in C++ using Ort
☆24Updated 3 years ago
Related projects ⓘ
Alternatives and complementary repositories for gpt2-cpp
- LLM training in simple, raw C/CUDA☆86Updated 6 months ago
- ☆52Updated last year
- Inference Llama 2 in one file of pure C++☆79Updated last year
- Experiments with BitNet inference on CPU☆50Updated 7 months ago
- ☆123Updated 10 months ago
- vLLM: A high-throughput and memory-efficient inference and serving engine for LLMs☆89Updated this week
- Tiny C++11 GPT-2 inference implementation from scratch☆46Updated 10 months ago
- Inference of Mamba models in pure C☆177Updated 8 months ago
- Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees" adapted for Llama models☆36Updated last year
- GGML implementation of BERT model with Python bindings and quantization.☆51Updated 8 months ago
- General purpose GPU compute framework built on Vulkan to support 1000s of cross vendor graphics cards (AMD, Qualcomm, NVIDIA & friends). …☆41Updated last month
- A faithful clone of Karpathy's llama2.c (one file inference, zero dependency) but fully functional with LLaMA 3 8B base and instruct mode…☆48Updated 3 months ago
- RWKV in nanoGPT style☆177Updated 5 months ago
- Train your own small bitnet model☆55Updated 3 weeks ago
- A torchless, c++ rwkv implementation using 8bit quantization, written in cuda/hip/vulkan for maximum compatibility and minimum dependenci…☆307Updated 9 months ago
- SparseGPT + GPTQ Compression of LLMs like LLaMa, OPT, Pythia☆41Updated last year
- QuIP quantization☆46Updated 7 months ago
- Inference Llama 2 in one file of pure C & one file with CUDA☆16Updated last year
- Python bindings for ggml☆132Updated 2 months ago
- Inference Vision Transformer (ViT) in plain C/C++ with ggml☆229Updated 7 months ago
- Fork of llama.cpp, extended for GPT-NeoX, RWKV-v4, and Falcon models☆29Updated last year
- RWKV, in easy to read code☆55Updated last week
- tinygrad port of the RWKV large language model.☆43Updated 4 months ago
- A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ, and export to onnx/onnx-runtime easily.☆148Updated last month
- A safetensors extension to efficiently store sparse quantized tensors on disk☆46Updated this week
- Standalone Flash Attention v2 kernel without libtorch dependency☆98Updated 2 months ago
- llama.cpp fork with additional SOTA quants and improved performance☆89Updated this week
- Universal cross-platform tokenizers binding to HF and sentencepiece☆273Updated 3 months ago
- Minimal example of using a traced huggingface transformers model with libtorch☆35Updated 4 years ago