GeeeekExplorer / cupytorchLinks
A small framework mimics PyTorch using CuPy or NumPy
☆47Updated 3 years ago
Alternatives and similar repositories for cupytorch
Users that are interested in cupytorch are comparing it to the libraries listed below
Sorting:
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆41Updated last year
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆22Updated last year
- A Tight-fisted Optimizer☆50Updated 2 years ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆58Updated last year
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆25Updated 2 months ago
- Notes of my introduction about NLP in Fudan University☆37Updated 4 years ago
- The accompanying code for "Memory-efficient Transformers via Top-k Attention" (Ankit Gupta, Guy Dar, Shaya Goodman, David Ciprut, Jonatha…☆70Updated 4 years ago
- Distributed DataLoader For Pytorch Based On Ray☆24Updated 3 years ago
- ☆19Updated last year
- ☆22Updated last year
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year
- Linear Attention Sequence Parallelism (LASP)☆86Updated last year
- This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …☆33Updated 2 years ago
- Lion and Adam optimization comparison☆64Updated 2 years ago
- [EMNLP 2022] Official implementation of Transnormer in our EMNLP 2022 paper - The Devil in Linear Transformer☆62Updated 2 years ago
- Odysseus: Playground of LLM Sequence Parallelism☆77Updated last year
- A *tuned* minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training☆117Updated 4 years ago
- ☆23Updated last month
- ☆11Updated 2 years ago
- Summary of system papers/frameworks/codes/tools on training or serving large model☆57Updated last year
- InsNet Runs Instance-dependent Neural Networks with Padding-free Dynamic Batching.☆67Updated 3 years ago
- ☆106Updated last year
- [KDD'22] Learned Token Pruning for Transformers☆100Updated 2 years ago
- ☆14Updated 2 years ago
- [ICLR 2024] This is the official PyTorch implementation of "QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Mod…☆29Updated last year
- Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI☆56Updated 2 years ago
- Models and examples built with OneFlow☆99Updated 11 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆64Updated last year
- triton ver of gqa flash attn, based on the tutorial☆12Updated last year
- Official PyTorch implementation of CD-MOE☆12Updated 5 months ago