GeeeekExplorer / cupytorchLinks
A small framework mimics PyTorch using CuPy or NumPy
☆37Updated 3 years ago
Alternatives and similar repositories for cupytorch
Users that are interested in cupytorch are comparing it to the libraries listed below
Sorting:
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆39Updated last year
- patches for huggingface transformers to save memory☆23Updated 3 weeks ago
- Distributed DataLoader For Pytorch Based On Ray☆24Updated 3 years ago
- Contextual Position Encoding but with some custom CUDA Kernels https://arxiv.org/abs/2405.18719☆22Updated last year
- Odysseus: Playground of LLM Sequence Parallelism☆70Updated last year
- ☆22Updated last year
- This repository contains the code for the paper in Findings of EMNLP 2021: "EfficientBERT: Progressively Searching Multilayer Perceptron …☆33Updated 2 years ago
- triton ver of gqa flash attn, based on the tutorial☆11Updated 10 months ago
- OpenLLMDE: An open source data engineering framework for LLMs☆17Updated last year
- A collection of instruction data and scripts for machine translation.☆20Updated last year
- Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI☆57Updated last year
- A Python implementation of Toolformer using Huggingface Transformers☆14Updated 2 years ago
- A Tight-fisted Optimizer☆48Updated 2 years ago
- ☆19Updated last year
- ☆37Updated 2 years ago
- Lion and Adam optimization comparison☆61Updated 2 years ago
- paddle code convert toolkit☆22Updated 2 years ago
- ☆16Updated last year
- 逻辑回归和单层softmax的解析解☆12Updated 3 years ago
- Using FlexAttention to compute attention with different masking patterns☆44Updated 9 months ago
- ☆31Updated last year
- [ICML'24 Oral] The official code of "DiJiang: Efficient Large Language Models through Compact Kernelization", a novel DCT-based linear at…☆101Updated last year
- IntLLaMA: A fast and light quantization solution for LLaMA☆18Updated last year
- code for paper "Accessing higher dimensions for unsupervised word translation"☆21Updated 2 years ago
- ☆20Updated last year
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆57Updated last year
- Implementation of the model: "Reka Core, Flash, and Edge: A Series of Powerful Multimodal Language Models" in PyTorch☆30Updated this week
- differentiable top-k operator☆21Updated 5 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆64Updated last year
- 模型压缩的小白入门教程☆22Updated 11 months ago