Eddie-Wang1120 / Eddie-Wang-Hackathon2023Links
Whisper inference with TensorRT-LLM
☆22Updated 2 years ago
Alternatives and similar repositories for Eddie-Wang-Hackathon2023
Users that are interested in Eddie-Wang-Hackathon2023 are comparing it to the libraries listed below
Sorting:
- ☆72Updated 2 years ago
- export llama to onnx☆135Updated 8 months ago
- Transformer related optimization, including BERT, GPT☆59Updated 2 years ago
- llm-export can export llm model to onnx.☆312Updated 3 weeks ago
- ☆141Updated last year
- Universal cross-platform tokenizers binding to HF and sentencepiece☆389Updated last month
- Transformer related optimization, including BERT, GPT☆17Updated 2 years ago
- LLaMa/RWKV onnx models, quantization and testcase☆365Updated 2 years ago
- A quantization algorithm for LLM☆142Updated last year
- ☆128Updated 9 months ago
- 使用 cutlass 实现 flash-attention 精简版,具有教学意义☆48Updated last year
- Serving Inside Pytorch☆163Updated last month
- Common source, scripts and utilities for creating Triton backends.☆347Updated 2 weeks ago
- optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052☆478Updated last year
- ☆199Updated 4 months ago
- Simple Dynamic Batching Inference☆145Updated 3 years ago
- Transformer related optimization, including BERT, GPT☆39Updated 2 years ago
- ☆59Updated 10 months ago
- ☆32Updated last year
- ☆76Updated 3 years ago
- The Triton TensorRT-LLM Backend☆890Updated last week
- A Toolkit to Help Optimize Large Onnx Model☆159Updated last year
- symmetric int8 gemm☆67Updated 5 years ago
- Run Chinese MobileBert model on SNPE.☆15Updated 2 years ago
- ☆26Updated 2 years ago
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆264Updated last month
- simplify >2GB large onnx model☆63Updated 9 months ago
- ☆125Updated last year
- ☆150Updated 8 months ago
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆40Updated 7 months ago