mli / transformers-benchmarks
real Transformer TeraFLOPS on various GPUs
☆873Updated 9 months ago
Related projects ⓘ
Alternatives and complementary repositories for transformers-benchmarks
- Several simple examples for popular neural network toolkits calling custom CUDA operators.☆1,332Updated 3 years ago
- ☆588Updated 5 months ago
- LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training☆391Updated this week
- How to use wandb?☆593Updated last year
- an implementation of transformer, bert, gpt, and diffusion models for learning purposes☆144Updated 3 weeks ago
- Rotary Transformer☆811Updated 2 years ago
- Efficient Training (including pre-training and fine-tuning) for Big Models☆560Updated 3 months ago
- SwissArmyTransformer is a flexible and powerful library to develop your own Transformer variants.☆991Updated last week
- A fast MoE impl for PyTorch☆1,558Updated 4 months ago
- personal chatgpt☆315Updated this week
- 该仓库主要记录 大模型（LLMs） 算法工程师相关的面试题☆1,481Updated 7 months ago
- 🎉 Modern CUDA Learn Notes with PyTorch: CUDA Cores, Tensor Cores, fp32/tf32, fp16/bf16, fp8/int8, flash_attn, rope, sgemm, hgemm, sgemv,…☆1,384Updated this week
- LLMs interview notes and answers:该仓库主要记录大模型（LLMs）算法工程师相关的面试题和参考答案☆1,161Updated 10 months ago
- Cool Papers - Immersive Paper Discovery☆396Updated last week
- 利用HuggingFace的官方下载工具从镜像网站进行高速下载。☆810Updated 3 weeks ago
- Ongoing research training transformer language models at scale, including: BERT & GPT-2☆1,884Updated 3 weeks ago
- Personal Project: MPP-Qwen14B & MPP-Qwen-Next(Multimodal Pipeline Parallel based on Qwen-LM). Support [video/image/multi-image] {sft/conv…☆375Updated last month
- how to optimize some algorithm in cuda.☆1,569Updated this week
- A plug-and-play library for parameter-efficient-tuning (Delta Tuning)☆996Updated last month
- PyTorch Project Specification.☆665Updated 3 years ago
- A collection of phenomenons observed during the scaling of big foundation models, which may be developed into consensus, principles, or l…☆275Updated last year
- LLMs interview notes and answers:该仓库主要记录大模型（LLMs）算法工程师相关的面试题和参考答案☆327Updated last year
- Safe RLHF: Constrained Value Alignment via Safe Reinforcement Learning from Human Feedback☆1,336Updated 4 months ago
- Best practice for training LLaMA models in Megatron-LM☆627Updated 10 months ago
- 更纯粹、更高压缩率的Tokenizer☆447Updated 6 months ago
- An easy/swift-to-adapt PyTorch-Lighting template. 套壳模板，简单易用，稍改原来Pytorch代码，即可适配Lightning。You can translate your previous Pytorch code much…☆1,336Updated last year
- The pure and clear PyTorch Distributed Training Framework.☆276Updated 9 months ago
- Tutel MoE: An Optimized Mixture-of-Experts Implementation☆728Updated last week
- The calflops is designed to calculate FLOPs、MACs and Parameters in all various neural networks, such as Linear、 CNN、 RNN、 GCN、Transformer…☆555Updated 4 months ago