void-main / fastertransformer_backend

Related projects ⓘ

Alternatives and complementary repositories for fastertransformer_backend

• void-main / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆60Updated last year
• THUDM / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆39Updated last year
• OpenPPL / ppl.llm.serving
  ☆123Updated 2 weeks ago
• ProjectD-AI / LLaMA-Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆69Updated last year
• CoinCheung / gdGPT
  Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
  ☆90Updated 9 months ago
• OpenPPL / ppl.nn.llm
  ☆140Updated 6 months ago
• MoFHeka / LLaMA-Megatron
  A LLaMA1/LLaMA12 Megatron implement.
  ☆27Updated 11 months ago
• anyscale / llm-continuous-batching-benchmarks
  ☆111Updated 8 months ago
• Rayrtfr / FasterTransformer
  Transformer related optimization, including BERT, GPT
  ☆17Updated last year
• madsys-dev / deepseekv2-profile
  ☆64Updated 3 months ago
• alibaba / ChatLearn
  A flexible and efficient training framework for large-scale alignment tasks
  ☆206Updated this week
• AniZpZ / AutoSmoothQuant
  An easy-to-use package for implementing SmoothQuant for LLMs
  ☆83Updated 6 months ago
• genggui001 / Megatron-DeepSpeed-Llama
  ☆82Updated last year
• neuralmagic / AutoFP8
  ☆157Updated last month
• wejoncy / QLLM
  A general 2-8 bits quantization toolbox with GPTQ/AWQ/HQQ, and export to onnx/onnx-runtime easily.
  ☆149Updated last month
• kwai / Megatron-Kwai
  [USENIX ATC '24] Accelerating the Training of Large Language Models using Efficient Activation Rematerialization and Optimal Hybrid Paral…
  ☆45Updated 3 months ago
• triton-inference-server / vllm_backend
  ☆191Updated this week
• zms1999 / SmartMoE
  A MoE impl for PyTorch, [ATC'23] SmartMoE
  ☆57Updated last year
• cli99 / llm-analysis
  Latency and Memory Analysis of Transformer Models for Training and Inference
  ☆355Updated last week
• feifeibear / LLMRoofline
  Compare different hardware platforms via the Roofline Model for LLM inference tasks.
  ☆75Updated 8 months ago
• pcg-mlp / KsanaLLM
  ☆289Updated this week
• LLMServe / DistServe
  Disaggregated serving system for Large Language Models (LLMs).
  ☆359Updated 3 months ago
• Ascend / AscendSpeed
  ☆74Updated 11 months ago
• sgl-project / sgl-learning-materials
  Materials for learning SGLang
  ☆96Updated this week
• ninehills / llm-inference-benchmark
  LLM Inference benchmark
  ☆350Updated 3 months ago
• FlagOpen / FlagScale
  FlagScale is a large model toolkit based on open-sourced projects.
  ☆169Updated this week
• fanshiqing / grouped_gemm
  PyTorch bindings for CUTLASS grouped GEMM.
  ☆68Updated 4 months ago
• microsoft / sarathi-serve
  A low-latency & high-throughput serving engine for LLMs
  ☆245Updated 2 months ago
• luchangli03 / export_llama_to_onnx
  export llama to onnx
  ☆96Updated 5 months ago
• LowinLi / transformers-stream-generator
  This is a text generation method which returns a generator, streaming out each token in real-time during inference, based on Huggingface/…
  ☆96Updated 8 months ago