experiments with inference on llama
☆103Jun 6, 2024Updated last year
Alternatives and similar repositories for llama-inference
Users that are interested in llama-inference are comparing it to the libraries listed below
Sorting:
- ☆13May 25, 2023Updated 2 years ago
- ☆20Nov 23, 2022Updated 3 years ago
- Simple (fast) transformer inference in PyTorch with torch.compile + lit-llama code☆10Aug 29, 2023Updated 2 years ago
- Example of applying CUDA graphs to LLaMA-v2☆12Aug 25, 2023Updated 2 years ago
- extensible collectives library in triton☆96Mar 31, 2025Updated 11 months ago
- Track the progress of LLM context utilisation☆55Apr 14, 2025Updated 10 months ago
- This repo lets you run mistral-7b in Google Colab.☆16Oct 1, 2023Updated 2 years ago
- Janus is an opensource IA for Star Citizen☆11Dec 23, 2023Updated 2 years ago
- An automation webcrawler based on Selenium library for retrieving parliamentary questions on The Website of Taiwan Legislative Yuan (http…☆11Jun 8, 2023Updated 2 years ago
- ☆135Nov 24, 2023Updated 2 years ago
- Lyra V2 (SoundStream) running in the browser☆19Sep 20, 2023Updated 2 years ago
- Interface to multicore QR factorization qr_mumps☆18Feb 13, 2026Updated 3 weeks ago
- Microsoft Phi 2 Streamlit App, deployed on HuggingFace Spaces is based on the Microsoft Phi 2 small language model (SLM) for text generat…☆14May 1, 2024Updated last year
- Efficient, scalable and enterprise-grade CPU/GPU inference server for 🤗 Hugging Face transformer models 🚀☆1,688Oct 23, 2024Updated last year
- Examples using 🤗 Hub to share and reload machine learning models☆32Nov 3, 2022Updated 3 years ago
- Smol but mighty language model☆65Apr 4, 2023Updated 2 years ago
- Code for the paper "Cottention: Linear Transformers With Cosine Attention"☆20Nov 15, 2025Updated 3 months ago
- A fork of sqlite-utils with CLI etc removed☆17Jan 29, 2026Updated last month
- LlamaWorksDB is a Retrieval Augmented Generation (RAG) product designed to interact with the documentation of various products such as Ll…☆17May 3, 2024Updated last year
- ☆21Mar 13, 2023Updated 2 years ago
- ☆16Oct 24, 2023Updated 2 years ago
- batched loras☆350Sep 6, 2023Updated 2 years ago
- Various transformers for FSDP research☆38Nov 11, 2022Updated 3 years ago
- S-LoRA: Serving Thousands of Concurrent LoRA Adapters☆1,900Jan 21, 2024Updated 2 years ago
- Serving multiple LoRA finetuned LLM as one☆1,144May 8, 2024Updated last year
- Experiments with multimodal deep learning models based on transformers☆11Oct 9, 2022Updated 3 years ago
- Use Actions to acquire those precious lambda GPUs☆19Sep 7, 2023Updated 2 years ago
- ☆598Aug 23, 2024Updated last year
- A more memory-efficient rewrite of the HF transformers implementation of Llama for use with quantized weights.☆2,913Sep 30, 2023Updated 2 years ago
- MII makes low-latency and high-throughput inference possible, powered by DeepSpeed.☆2,097Jun 30, 2025Updated 8 months ago
- Make triton easier☆50Jun 12, 2024Updated last year
- AutoAWQ implements the AWQ algorithm for 4-bit quantization with a 2x speedup during inference. Documentation:☆2,319May 11, 2025Updated 9 months ago
- Experiments on speculative sampling with Llama models☆128Jun 8, 2023Updated 2 years ago
- ☆20Jan 27, 2024Updated 2 years ago
- ☆18Oct 19, 2024Updated last year
- ☆19May 6, 2023Updated 2 years ago
- Python bindings for the Transformer models implemented in C/C++ using GGML library.☆1,882Jan 28, 2024Updated 2 years ago
- ☆19Sep 26, 2023Updated 2 years ago
- ☆75Jul 2, 2021Updated 4 years ago