abetlen / ggml-pythonLinks
Python bindings for ggml
☆140Updated 9 months ago
Alternatives and similar repositories for ggml-python
Users that are interested in ggml-python are comparing it to the libraries listed below
Sorting:
- Code for the paper "QMoE: Practical Sub-1-Bit Compression of Trillion-Parameter Models".☆275Updated last year
- Inference of Mamba models in pure C☆186Updated last year
- Inference Vision Transformer (ViT) in plain C/C++ with ggml☆285Updated last year
- GPTQ inference Triton kernel☆300Updated 2 years ago
- A high-throughput and memory-efficient inference and serving engine for LLMs☆263Updated 7 months ago
- RWKV in nanoGPT style☆189Updated 11 months ago
- ☆536Updated 7 months ago
- An efficent implementation of the method proposed in "The Era of 1-bit LLMs"☆153Updated 7 months ago
- Official implementation of Half-Quadratic Quantization (HQQ)☆814Updated last week
- LLaVA server (llama.cpp).☆179Updated last year
- Code for paper: "QuIP: 2-Bit Quantization of Large Language Models With Guarantees"☆365Updated last year
- LLM-based code completion engine☆189Updated 4 months ago
- This is our own implementation of 'Layer Selective Rank Reduction'☆238Updated last year
- A safetensors extension to efficiently store sparse quantized tensors on disk☆117Updated this week
- Micro Llama is a small Llama based model with 300M parameters trained from scratch with $500 budget☆150Updated last year
- an implementation of Self-Extend, to expand the context window via grouped attention☆119Updated last year
- An innovative library for efficient LLM inference via low-bit quantization☆348Updated 9 months ago
- Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.☆199Updated 10 months ago
- Fast parallel LLM inference for MLX☆189Updated 10 months ago
- Experiments on speculative sampling with Llama models☆126Updated last year
- Inference Vision Transformer (ViT) in plain C/C++ with ggml☆30Updated last year
- A torchless, c++ rwkv implementation using 8bit quantization, written in cuda/hip/vulkan for maximum compatibility and minimum dependenci…☆312Updated last year
- inference code for mixtral-8x7b-32kseqlen☆99Updated last year
- OpenAI compatible API for TensorRT LLM triton backend☆208Updated 10 months ago
- CLIP inference in plain C/C++ with no extra dependencies☆498Updated 9 months ago
- Landmark Attention: Random-Access Infinite Context Length for Transformers QLoRA☆122Updated last year
- ☆119Updated last year
- Advanced Quantization Algorithm for LLMs and VLMs, with support for CPU, Intel GPU, CUDA and HPU. Seamlessly integrated with Torchao, Tra…☆483Updated this week
- Easy and Efficient Quantization for Transformers☆198Updated 3 months ago
- Simple high-throughput inference library☆115Updated 2 weeks ago