Alternatives and similar repositories for transformers-stream-generator

Users that are interested in transformers-stream-generator are comparing it to the libraries listed below

• THUDM / icetk
  A unified tokenization tool for Images, Chinese and English.
  ☆153Updated 2 years ago
• HuangLK / transpeeder
  train llama on a single A100 80G node using 🤗 transformers and 🚀 Deepspeed Pipeline Parallelism
  ☆224Updated 2 years ago
• OpenLMLab / scaling-rope
  code for Scaling Laws of RoPE-based Extrapolation
  ☆73Updated 2 years ago
• chu-tianxiang / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆132Updated last year
• bigscience-workshop / data-preparation
  Code used for sourcing and cleaning the BigScience ROOTS corpus
  ☆318Updated 2 years ago
• DachengLi1 / LongChat
  Official repository for LongChat and LongEval
  ☆533Updated last year
• ProjectD-AI / LLaMA-Megatron-DeepSpeed
  Ongoing research training transformer language models at scale, including: BERT & GPT-2
  ☆69Updated 2 years ago
• zsc / llama_infer
  Inference script for Meta's LLaMA models using Hugging Face wrapper
  ☆110Updated 2 years ago
• IEIT-Yuan / Yuan2.0-M32
  Mixture-of-Experts (MoE) Language Model
  ☆194Updated last year
• the-seeds / imitater
  Imitate OpenAI with Local Models
  ☆89Updated last year
• shjwudp / c4-dataset-script
  Inspired by google c4, here is a series of colossal clean data cleaning scripts focused on CommonCrawl data processing. Including Chinese…
  ☆135Updated 2 years ago
• QwenLM / vllm-gptq
  A high-throughput and memory-efficient inference and serving engine for LLMs
  ☆140Updated last year
• SunLemuria / OpenGPTAndBeyond
  Open efforts to implement ChatGPT-like models and beyond.
  ☆107Updated last year
• OpenBMB / BMCook
  Model Compression for Big Models
  ☆167Updated 2 years ago
• yangjianxin1 / LongQLoRA
  LongQLoRA: Extent Context Length of LLMs Efficiently
  ☆168Updated 2 years ago
• open-compass / BotChat
  Evaluating LLMs' multi-round chatting capability via assessing conversations generated by two LLM instances.
  ☆160Updated 7 months ago
• CoinCheung / gdGPT
  Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.
  ☆98Updated last year
• feizc / MLE-LLaMA
  Multi-language Enhanced LLaMA
  ☆303Updated 2 years ago
• bojone / rerope
  Rectified Rotary Position Embeddings
  ☆386Updated last year
• keezen / ntk_alibi
  NTK scaled version of ALiBi position encoding in Transformer.
  ☆69Updated 2 years ago
• xverse-ai / XVERSE-65B
  XVERSE-65B: A multilingual large language model developed by XVERSE Technology Inc.
  ☆141Updated last year
• jshuadvd / LongRoPE
  Implementation of the LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens Paper
  ☆151Updated last year
• NormXU / Consistent-DynamicNTKRoPE
  An Experiment on Dynamic NTK Scaling RoPE
  ☆64Updated 2 years ago
• liutiedong / goat
  a Fine-tuned LLaMA that is Good at Arithmetic Tasks
  ☆178Updated 2 years ago
• FlagOpen / FlagInstruct
  ☆173Updated 2 years ago
• bojone / NBCE
  Naive Bayes-based Context Extension
  ☆326Updated last year
• onesuper / HuggingFace-Datasets-Text-Quality-Analysis
  Retrieves parquet files from Hugging Face, identifies and quantifies junky data, duplication, contamination, and biased content in datase…
  ☆53Updated 2 years ago
• AtomEcho / AtomBulb
  旨在对当前主流LLM进行一个直观、具体、标准的评测
  ☆95Updated 2 years ago
• TencentARC / LLaMA-Pro
  [ACL 2024] Progressive LLaMA with Block Expansion.
  ☆514Updated last year
• jiahe7ay / infini-mini-transformer
  This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…
  ☆58Updated last year