Jittor / JittorInferLinks
JittorInfer is a high-performance C++ inference framework designed for large language models on Huawei's Ascend AI processor.
☆78Updated last week
Alternatives and similar repositories for JittorInfer
Users that are interested in JittorInfer are comparing it to the libraries listed below
Sorting:
- FlagCX is a scalable and adaptive cross-chip communication library.☆170Updated this week
- This is a demo how to write a high performance convolution run on apple silicon☆57Updated 3 years ago
- ⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.☆148Updated 8 months ago
- High Performance LLM Inference Operator Library☆603Updated this week
- 🤖FFPA: Extend FlashAttention-2 with Split-D, ~O(1) SRAM complexity for large headdim, 1.8x~3x↑🎉 vs SDPA EA.☆248Updated last week
- Triton adapter for Ascend. Mirror of https://gitee.com/ascend/triton-ascend☆105Updated this week
- We invite you to visit and follow our new repository at https://github.com/microsoft/TileFusion. TiledCUDA is a highly efficient kernel …☆192Updated last year
- Compare different hardware platforms via the Roofline Model for LLM inference tasks.☆120Updated last year
- Tutorials for writing high-performance GPU operators in AI frameworks.☆135Updated 2 years ago
- DLSlime: Flexible & Efficient Heterogeneous Transfer Toolkit☆92Updated last week
- 分层解耦的深度学习推理引擎☆79Updated 11 months ago
- Implement custom operators in PyTorch with cuda/c++☆76Updated 3 years ago
- ☆41Updated last year
- Fast and memory-efficient exact attention☆110Updated last week
- A Triton JIT runtime and ffi provider in C++☆30Updated last week
- ☆96Updated 10 months ago
- ☆130Updated last year
- AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and ver…☆298Updated 2 weeks ago
- A prefill & decode disaggregated LLM serving framework with shared GPU memory and fine-grained compute isolation.☆123Updated last month
- An unofficial cuda assembler, for all generations of SASS, hopefully ：）☆84Updated 2 years ago
- A benchmark suited especially for deep learning operators☆42Updated 2 years ago
- Venus Collective Communication Library, supported by SII and Infrawaves.☆137Updated this week
- KV cache store for distributed LLM inference☆389Updated 2 months ago
- ☆116Updated 3 weeks ago
- ☆144Updated last year
- ☆117Updated 8 months ago
- High performance Transformer implementation in C++.☆148Updated last year
- ☆342Updated this week
- ☆74Updated last week
- ☆152Updated last year