experiments with inference on llama
☆103Jun 6, 2024Updated 2 years ago
Alternatives and similar repositories for llama-inference
Users that are interested in llama-inference are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- ☆20Nov 23, 2022Updated 3 years ago
- ☆20Jan 27, 2024Updated 2 years ago
- Leverage your LangChain trace data for fine tuning☆45Aug 2, 2024Updated 2 years ago
- Simple (fast) transformer inference in PyTorch with torch.compile + lit-llama code☆11Aug 29, 2023Updated 3 years ago
- Serving multiple LoRA finetuned LLM as one☆1,175May 8, 2024Updated 2 years ago
- Simple, predictable pricing with DigitalOcean hosting • AdAlways know what you'll pay with monthly caps and flat pricing. Enterprise-grade infrastructure trusted by 600k+ customers.
- 9th solution☆11Oct 11, 2022Updated 3 years ago
- A block oriented training approach for inference time optimization.☆34Aug 19, 2024Updated 2 years ago
- 1st place (Efficiency Track) solution for Feedback Prize - English Language Learning Kaggle competition☆11Dec 17, 2022Updated 3 years ago
- Project repository of the paper "Less Annotating, More Classifying – Addressing the Data Scarcity Issue of Supervised Machine Learning wi…☆35Mar 19, 2024Updated 2 years ago
- ☆18Jun 25, 2026Updated 2 months ago
- Official implementation of the ACL Findings 2023 paper: Interpretable Automatic Fine-grained Inconsistency Detection in Text Summarizatio…☆15Jan 25, 2024Updated 2 years ago
- Test pytorch code with minimal computational overhead☆26Jun 8, 2023Updated 3 years ago
- Simple speculative decoding technique, integrated in vLLM and transformers☆616Aug 23, 2024Updated 2 years ago
- batched loras☆352Sep 6, 2023Updated 3 years ago
- GPU virtual machines on DigitalOcean Gradient AI • AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- Mutual Information Predicts Hallucinations in Abstractive Summarization☆13Nov 14, 2022Updated 3 years ago
- ☆20Jul 12, 2023Updated 3 years ago
- Example of applying CUDA graphs to LLaMA-v2☆11Aug 25, 2023Updated 3 years ago
- Merge Transformers language models by use of gradient parameters.☆215Aug 8, 2024Updated 2 years ago
- This repo lets you run mistral-7b in Google Colab.☆16Oct 1, 2023Updated 2 years ago
- ☆74Jul 2, 2021Updated 5 years ago
- Experimenting text-embeddings-inference server on both CPU and GPU☆18Oct 25, 2023Updated 2 years ago
- A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.☆2,943Sep 30, 2023Updated 2 years ago
- Non-local Modeling for Image Quality Assessment☆13Dec 20, 2023Updated 2 years ago
- GPUs on demand by Runpod - Special Offer Available • AdRun AI, ML, and HPC workloads on powerful cloud GPUs—without limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,923Jan 21, 2024Updated 2 years ago
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆2,112Jun 30, 2025Updated last year
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆2,349May 11, 2025Updated last year
- Microsoft Phi 2 Streamlit App, deployed on HuggingFace Spaces is based on the Microsoft Phi 2 small language model (SLM) for text generat…☆15May 1, 2024Updated 2 years ago
- Experiments on speculative sampling with Llama models☆129Jun 8, 2023Updated 3 years ago
- Salesforce open-source LLMs with 8k sequence length.☆726Jun 2, 2026Updated 3 months ago
- GPTQ inference Triton kernel☆322May 18, 2023Updated 3 years ago
- Minimal example scripts of the Hugging Face Trainer, focused on staying under 150 lines☆196May 6, 2024Updated 2 years ago
- An automation webcrawler based on Selenium library for retrieving parliamentary questions on The Website of Taiwan Legislative Yuan (http…☆11Jun 8, 2023Updated 3 years ago
- 1-Click AI Models by DigitalOcean Gradient • AdDeploy popular AI models on DigitalOcean Gradient GPU virtual machines with just a single click. Zero configuration with optimized deployments.
- Lyra V2 (SoundStream) running in the browser☆19Sep 20, 2023Updated 2 years ago
- ☆454Oct 15, 2023Updated 2 years ago
- ☆22Jan 5, 2024Updated 2 years ago
- 3rd Place solution for Feedback Prize - Predicting Effective Arguments Kaggle competition☆16Sep 6, 2022Updated 4 years ago
- Examples using 🤗 Hub to share and reload machine learning models☆32Nov 3, 2022Updated 3 years ago
- Large Language Model Text Generation Inference☆10,890Mar 21, 2026Updated 5 months ago
- Use Actions to acquire those precious lambda GPUs☆19Sep 7, 2023Updated 3 years ago