experiments with inference on llama
☆103Jun 6, 2024Updated 2 years ago
Alternatives and similar repositories for llama-inference
Users that are interested in llama-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆13May 25, 2023Updated 3 years ago
- Kernels by MakoraGenerate☆15May 26, 2026Updated 5 months ago
- ☆20Jan 27, 2024Updated 2 years ago
- ☆10May 21, 2023Updated 3 years ago
- A dashboard for exploring timm learning rate schedulers☆20Nov 22, 2024Updated last year
- Wordpress hosting with auto-scaling - Free Trial Offer • AdFully Managed hosting for WordPress and WooCommerce businesses that need reliable, auto-scalable performance. Cloudways SafeUpdates now available.
- Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀☆1,695Oct 23, 2024Updated 2 years ago
- Performant, composable online learning☆16Feb 22, 2021Updated 5 years ago
- ☆12Oct 25, 2023Updated 3 years ago
- ☆16Aug 10, 2022Updated 4 years ago
- QLoRA for Masked Language Modeling☆23Sep 11, 2023Updated 3 years ago
- Simple speculative decoding technique, integrated in vLLM and transformers☆622Aug 23, 2024Updated 2 years ago
- Experiments on speculative sampling with Llama models☆129Jun 8, 2023Updated 3 years ago
- batched loras☆352Sep 6, 2023Updated 3 years ago
- CNER: Concept and Named Entity Recognition☆13Jul 15, 2024Updated 2 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- Exploring NLP weak supervision approaches to train text classification models. The project is also a prototype for a semi-automated text …☆22Feb 18, 2024Updated 2 years ago
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆2,343May 11, 2025Updated last year
- 9th solution☆11Oct 11, 2022Updated 4 years ago
- Minimal example scripts of the Hugging Face Trainer, focused on staying under 150 lines☆196May 6, 2024Updated 2 years ago
- Salesforce open-source LLMs with 8k sequence length.☆727Jun 2, 2026Updated 5 months ago
- Benchmarking code for running quantized kernels from vLLM and other libraries☆14Dec 3, 2024Updated last year
- An open source implementation of CLIP.☆33Nov 7, 2022Updated 4 years ago
- Serving multiple LoRA finetuned LLM as one☆1,177May 8, 2024Updated 2 years ago
- 1st place (Efficiency Track) solution for Feedback Prize - English Language Learning Kaggle competition☆11Dec 17, 2022Updated 3 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,921Jan 21, 2024Updated 2 years ago
- Project repository of the paper "Less Annotating, More Classifying – Addressing the Data Scarcity Issue of Supervised Machine Learning wi…☆35Mar 19, 2024Updated 2 years ago
- An unofficial Python client library for Lambda Lab's Cloud Computing Platform☆12Feb 19, 2023Updated 3 years ago
- Test pytorch code with minimal computational overhead☆26Jun 8, 2023Updated 3 years ago
- Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.☆6,271Aug 22, 2025Updated last year
- Large Language Model Text Generation Inference☆10,879Mar 21, 2026Updated 7 months ago
- ☆20Jul 12, 2023Updated 3 years ago
- A reference implementation of an end to end, open-source MLOps platform.☆15Nov 20, 2022Updated 3 years ago
- This repo lets you run mistral-7b in Google Colab.☆16Oct 1, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- ☆127Jun 25, 2026Updated 4 months ago
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆2,117Jun 30, 2025Updated last year
- ☆74Jul 2, 2021Updated 5 years ago
- Official implementation of the ACL Findings 2023 paper: Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarizatio…☆15Jan 25, 2024Updated 2 years ago
- Python bindings for the Transformer models implemented in C/C++ using GGML library.☆1,888Jan 28, 2024Updated 2 years ago
- ☆20Apr 12, 2024Updated 2 years ago
- Merge Transformers language models by use of gradient parameters.☆215Aug 8, 2024Updated 2 years ago
Are these results useful?