GeeeekExplorer / cupytorchLinks
A small framework mimics PyTorch using CuPy or NumPy
☆48Updated 3 years ago
Alternatives and similar repositories for cupytorch
Users that are interested in cupytorch are comparing it to the libraries listed below
Sorting:
- A Tight-fisted Optimizer☆50Updated 2 years ago
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆41Updated last year
- Implementation of IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs (ICLR 2024).☆25Updated 4 months ago
- ☆79Updated last year
- A *tuned* minimal PyTorch re-implementation of the OpenAI GPT (Generative Pretrained Transformer) training☆118Updated 4 years ago
- Lion and Adam optimization comparison☆64Updated 2 years ago
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆22Updated last year
- Odysseus: Playground of LLM Sequence Parallelism☆78Updated last year
- This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …☆33Updated 2 years ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆58Updated last year
- Distributed DataLoader For Pytorch Based On Ray☆24Updated 4 years ago
- ☆19Updated last year
- [KDD'22] Learned Token Pruning for Transformers☆101Updated 2 years ago
- Inference framework for MoE layers based on TensorRT with Python binding☆41Updated 4 years ago
- ☆22Updated last year
- ☆16Updated last year
- InsNet Runs Instance-dependent Neural Networks with Padding-free Dynamic Batching.☆67Updated 4 years ago
- A tiny, didactical implementation of LLAMA 3☆42Updated 11 months ago
- Manages vllm-nccl dependency☆17Updated last year
- ☆32Updated last year
- 基于Gated Attention Unit的Transformer模型（尝鲜版）☆98Updated 2 years ago
- Summary of system papers/frameworks/codes/tools on training or serving large model☆57Updated last year
- 逻辑回归和单层softmax的解析解☆12Updated 4 years ago
- Notes of my introduction about NLP in Fudan University☆37Updated 4 years ago
- Linear Attention Sequence Parallelism (LASP)☆87Updated last year
- Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI☆56Updated 2 years ago
- An Experiment on Dynamic NTK Scaling RoPE☆64Updated last year
- 📑 Dive into Big Model Training☆115Updated 2 years ago
- differentiable top-k operator☆22Updated 10 months ago
- [ACL 2024] RelayAttention for Efficient Large Language Model Serving with Long System Prompts☆40Updated last year