yifanlu0227 / LLaMA2-7B-on-laptop
Lab 5 project of MIT-6.5940, deploying LLaMA2-7B-chat on one's laptop with TinyChatEngine.
☆10Updated 11 months ago
Related projects ⓘ
Alternatives and complementary repositories for LLaMA2-7B-on-laptop
- List of papers related to Vision Transformers quantization and hardware acceleration in recent AI conferences and journals.☆54Updated 5 months ago
- Summary of some awesome work for optimizing LLM inference☆35Updated this week
- hands on model tuning with TVM and profile it on a Mac M1, x86 CPU, and GTX-1080 GPU.☆39Updated last year
- All Homeworks for TinyML and Efficient Deep Learning Computing 6.5940 • Fall • 2023 • https://efficientml.ai☆135Updated 11 months ago
- ☆80Updated 11 months ago
- Tutorials of Extending and importing TVM with CMAKE Include dependency.☆10Updated last month
- 使用 cutlass 实现 flash-attention 精简版，具有教学意义☆32Updated 3 months ago
- ☆142Updated last year
- This repo contains the Assignments from Cornell Tech's ECE 5545 - Machine Learning Hardware and Systems offered in Spring 2023☆20Updated last year
- ☆121Updated last year
- Flash-LLM: Enabling Cost-Effective and Highly-Efficient Large Generative Model Inference with Unstructured Sparsity☆178Updated last year
- Machine Learning Compiler Road Map☆41Updated last year
- Puzzles for learning Triton, play it with minimal environment configuration!☆61Updated this week
- PyTorch extension for emulating FP8 data formats on standard FP32 Xeon/GPU hardware.☆100Updated 11 months ago
- ☆130Updated 3 months ago
- Examples of CUDA implementations by Cutlass CuTe☆86Updated this week
- ☆21Updated last year
- Since the emergence of chatGPT in 2022, the acceleration of Large Language Model has become increasingly important. Here is a list of pap…☆168Updated last week
- ☆17Updated 8 months ago
- A Easy-to-understand TensorOp Matmul Tutorial☆288Updated last month
- ☆82Updated 5 months ago
- ☆79Updated 8 months ago
- ☆12Updated 7 months ago
- [ACL 2024] A novel QAT with Self-Distillation framework to enhance ultra low-bit LLMs.☆81Updated 5 months ago
- ☆83Updated 4 months ago
- Code for the AAAI 2024 Oral paper "OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Model…☆53Updated 8 months ago
- Code Repository of Evaluating Quantized Large Language Models☆103Updated 2 months ago
- QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.☆76Updated last month
- Penn CIS 5650 (GPU Programming and Architecture) Final Project☆24Updated 11 months ago