Eddie-Wang1120 / Eddie-Wang-Hackathon2023Links
Whisper inference with TensorRT-LLM
☆22Updated last year
Alternatives and similar repositories for Eddie-Wang-Hackathon2023
Users that are interested in Eddie-Wang-Hackathon2023 are comparing it to the libraries listed below
Sorting:
- ☆72Updated 2 years ago
- export llama to onnx☆131Updated 7 months ago
- Universal cross-platform tokenizers binding to HF and sentencepiece☆369Updated last week
- llm-export can export llm model to onnx.☆301Updated 6 months ago
- Transformer related optimization, including BERT, GPT☆59Updated last year
- ☆139Updated last year
- LLaMa/RWKV onnx models, quantization and testcase☆362Updated 2 years ago
- 使用 cutlass 实现 flash-attention 精简版,具有教学意义☆45Updated 11 months ago
- A quantization algorithm for LLM☆141Updated last year
- Common source, scripts and utilities for creating Triton backends.☆337Updated this week
- ☆128Updated 7 months ago
- Simple Dynamic Batching Inference☆145Updated 3 years ago
- ☆195Updated 3 months ago
- ASR client for Triton ASR Service☆32Updated 7 months ago
- ☆59Updated 8 months ago
- DashInfer is a native LLM inference engine aiming to deliver industry-leading performance atop various hardware architectures, including …☆263Updated last week
- ☆149Updated 6 months ago
- The Triton backend for the ONNX Runtime.☆156Updated this week
- Transformer related optimization, including BERT, GPT☆17Updated 2 years ago
- ☆280Updated last week
- ☆98Updated last week
- simplify >2GB large onnx model☆61Updated 8 months ago
- optimized BERT transformer inference on NVIDIA GPU. https://arxiv.org/abs/2210.03052☆474Updated last year
- The Triton TensorRT-LLM Backend☆872Updated this week
- A parser, editor and profiler tool for ONNX models.☆447Updated this week
- Serving Inside Pytorch☆163Updated last week
- Performance of the C++ interface of flash attention and flash attention v2 in large language model (LLM) inference scenarios.☆39Updated 5 months ago
- ☆32Updated 11 months ago
- ☆15Updated 11 months ago
- A Toolkit to Help Optimize Large Onnx Model☆157Updated last year