Eddie-Wang1120 / Eddie-Wang-Hackathon2023
Whisper inference with TensorRT-LLM
☆21Updated last year
Alternatives and similar repositories for Eddie-Wang-Hackathon2023:
Users that are interested in Eddie-Wang-Hackathon2023 are comparing it to the libraries listed below
- ☆71Updated 2 years ago
- ☆74Updated 2 years ago
- A quantization algorithm for LLM☆139Updated 10 months ago
- export llama to onnx☆121Updated 3 months ago
- simplify >2GB large onnx model☆55Updated 4 months ago
- ☆139Updated last year
- 使用 cutlass 实现 flash-attention 精简版，具有教学意义☆39Updated 8 months ago
- Transformer related optimization, including BERT, GPT☆59Updated last year
- Serving Inside Pytorch☆160Updated this week
- ☆127Updated 4 months ago
- ASR client for Triton ASR Service☆28Updated 4 months ago
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆35Updated last month
- Triton Inference Server Web UI☆12Updated last year
- llm deploy project based onnx.☆36Updated 6 months ago
- Transformer related optimization, including BERT, GPT☆17Updated last year
- symmetric int8 gemm☆67Updated 4 years ago
- ☢️ TensorRT 2023复赛——基于TensorRT-LLM的Llama模型推断加速优化☆46Updated last year
- ☆148Updated 3 months ago
- A ctc decoder for both online and offline asr model☆63Updated last year
- ☆124Updated last year
- llm-export can export llm model to onnx.☆282Updated 3 months ago
- Kaldi-compatible online fbank extractor without external dependencies☆94Updated 3 weeks ago
- ☆37Updated 3 years ago
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆243Updated last week
- ☆186Updated 6 months ago
- Simple Dynamic Batching Inference☆145Updated 3 years ago
- ☆58Updated 5 months ago
- LLaMa/RWKV onnx models, quantization and testcase☆361Updated last year
- The Triton backend for TensorRT.☆73Updated this week
- Transformer related optimization, including BERT, GPT☆39Updated 2 years ago