experiments with inference on llama
☆103Jun 6, 2024Updated 2 years ago
Alternatives and similar repositories for llama-inference
Users that are interested in llama-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆20Nov 23, 2022Updated 3 years ago
- ☆20Jan 27, 2024Updated 2 years ago
- Leverage your LangChain trace data for fine tuning☆45Aug 2, 2024Updated 2 years ago
- A dashboard for exploring timm learning rate schedulers☆20Nov 22, 2024Updated last year
- Simple (fast) transformer inference in PyTorch with torch.compile + lit-llama code☆11Aug 29, 2023Updated 3 years ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆16Aug 10, 2022Updated 4 years ago
- Project for the AI Fellowship at Insight - Fact checking of text summarization models using Transformers.☆12Feb 19, 2020Updated 6 years ago
- Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀☆1,696Oct 23, 2024Updated last year
- Performant, composable online learning☆16Feb 22, 2021Updated 5 years ago
- ☆12Oct 25, 2023Updated 2 years ago
- Experiments on speculative sampling with Llama models☆129Jun 8, 2023Updated 3 years ago
- ☆122Apr 22, 2024Updated 2 years ago
- 1st place (Efficiency Track) solution for Feedback Prize - English Language Learning Kaggle competition☆11Dec 17, 2022Updated 3 years ago
- Project repository of the paper "Less Annotating, More Classifying – Addressing the Data Scarcity Issue of Supervised Machine Learning wi…☆35Mar 19, 2024Updated 2 years ago
- Virtual machines for every use case on DigitalOcean • AdGet dependable uptime with 99.99% SLA, simple security tools, and predictable monthly pricing with DigitalOcean's virtual machines, called Droplets.
- A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.☆2,943Sep 30, 2023Updated 3 years ago
- An open source implementation of CLIP.☆33Nov 7, 2022Updated 3 years ago
- Test pytorch code with minimal computational overhead☆26Jun 8, 2023Updated 3 years ago
- ☆20Jul 12, 2023Updated 3 years ago
- Mutual Information Predicts Hallucinations in Abstractive Summarization☆13Nov 14, 2022Updated 3 years ago
- ☆126Jun 25, 2026Updated 3 months ago
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆2,115Jun 30, 2025Updated last year
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆2,344May 11, 2025Updated last year
- A reference implementation of an end to end, open-source MLOps platform.☆15Nov 20, 2022Updated 3 years ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- batched loras☆352Sep 6, 2023Updated 3 years ago
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,924Jan 21, 2024Updated 2 years ago
- Salesforce open-source LLMs with 8k sequence length.☆728Jun 2, 2026Updated 4 months ago
- The Plumber framework for KG completion and structured triples extraction☆23May 23, 2023Updated 3 years ago
- Non-local Modeling for Image Quality Assessment☆14Dec 20, 2023Updated 2 years ago
- ☆74Jul 2, 2021Updated 5 years ago
- extensible collectives library in triton☆98Mar 31, 2025Updated last year
- Merge Transformers language models by use of gradient parameters.☆215Aug 8, 2024Updated 2 years ago
- 🏙 Interactive performance profiling and debugging tool for PyTorch neural networks.☆65Jan 21, 2025Updated last year
- End-to-end encrypted email - Proton Mail • AdSpecial offer: 40% Off Yearly / 80% Off First Month. All Proton services are open source and independently audited for security.
- ☆453Oct 15, 2023Updated 3 years ago
- Large Language Model Text Generation Inference☆10,879Mar 21, 2026Updated 7 months ago
- ☆133Nov 24, 2023Updated 2 years ago
- Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackab…☆1,584Jan 28, 2026Updated 8 months ago
- Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.☆6,260Aug 22, 2025Updated last year
- Dataset accompanying "PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data" (pu…☆14Dec 4, 2024Updated last year
- Applied AI experiments and examples for PyTorch☆322Aug 22, 2025Updated last year