yifanlu0227 / LLaMA2-7B-on-laptopLinks
Lab 5 project of MIT-6.5940, deploying LLaMA2-7B-chat on one's laptop with TinyChatEngine.
☆18Updated last year
Alternatives and similar repositories for LLaMA2-7B-on-laptop
Users that are interested in LLaMA2-7B-on-laptop are comparing it to the libraries listed below
Sorting:
- All Homeworks for TinyML and Efficient Deep Learning Computing 6.5940 • Fall • 2023 • https://efficientml.ai☆178Updated last year
- Summary of some awesome work for optimizing LLM inference☆103Updated 3 months ago
- ☆172Updated 2 years ago
- hands on model tuning with TVM and profile it on a Mac M1, x86 CPU, and GTX-1080 GPU.☆49Updated 2 years ago
- 🎓Automatically Update circult-eda-mlsys-tinyml Papers Daily using Github Actions (Update Every 8th hours)☆10Updated this week
- ☆23Updated last year
- Curated collection of papers in MoE model inference☆250Updated last month
- SpInfer: Leveraging Low-Level Sparsity for Efficient Large Language Model Inference on GPUs☆57Updated 5 months ago
- Code release for AdapMoE accepted by ICCAD 2024☆32Updated 4 months ago
- This is a repository of Binary General Matrix Multiply (BGEMM) by customized CUDA kernel. Thank FP6-LLM for the wheels!☆15Updated last year
- Examples of CUDA implementations by Cutlass CuTe☆225Updated 2 months ago
- [ICLR2025]: OSTQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitt…☆72Updated 4 months ago
- InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Management (OSDI'24)☆152Updated last year
- llm theoretical performance analysis tools and support params, flops, memory and latency analysis.☆104Updated last month
- Penn CIS 5650 (GPU Programming and Architecture) Final Project☆38Updated last year
- ☆181Updated last year
- ☆117Updated last month
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆266Updated 5 months ago
- A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores☆53Updated last year
- Large Language Model (LLM) Serving Paper and Resource List☆24Updated 3 months ago
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity☆218Updated last year
- Code for the AAAI 2024 Oral paper "OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Model…☆64Updated last year
- ☆107Updated last year
- A Easy-to-understand TensorOp Matmul Tutorial☆376Updated 11 months ago
- [ICML 2024 Oral] Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMs☆113Updated 2 months ago
- A GPU-optimized system for efficient long-context LLMs decoding with low-bit KV cache.☆58Updated this week
- Solution of Programming Massively Parallel Processors☆47Updated last year
- ☆150Updated last year
- ☆23Updated 5 months ago
- ☆51Updated last year