airaria / GRAIN
GRAIN: Gradient-based Intra-attention Pruning on Pre-trained Language Models
☆17Updated last year
Alternatives and similar repositories for GRAIN:
Users that are interested in GRAIN are comparing it to the libraries listed below
- 大语言模型训练和服务调研☆35Updated last year
- 1.4B sLLM for Chinese and English - HammerLLM🔨☆44Updated 9 months ago
- The complete training code of the open-source high-performance Llama model, including the full process from pre-training to RLHF.☆64Updated last year
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆55Updated 9 months ago
- 基于 LoRA 和 P-Tuning v2 的 ChatGLM-6B 高效参数微调☆54Updated last year
- Imitate OpenAI with Local Models☆85Updated 5 months ago
- the newest version of llama3,source code explained line by line using Chinese☆22Updated 9 months ago
- 百川Dynamic NTK-ALiBi的代码实现:无需微调即可推理更长文本☆47Updated last year
- code for Scaling Laws of RoPE-based Extrapolation☆70Updated last year
- aigc evals☆10Updated last year
- Large-scale exact string matching tool☆15Updated 2 months ago
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆38Updated 10 months ago
- This is a repo for showcasing using MCTS with LLMs to solve gsm8k problems☆46Updated 2 weeks ago
- Its an open source LLM based on MOE Structure.☆57Updated 6 months ago
- A more efficient GLM implementation!☆55Updated last year
- make LLM easier to use☆59Updated last year
- zero零训练llm调参☆31Updated last year
- 本项目用于大模型数学解题能力方面的数据集合成,模型训练及评测,相关文章记录。☆67Updated 4 months ago
- 使用qlora对中文大语言模型进行微调,包含ChatGLM、Chinese-LLaMA-Alpaca、BELLE☆85Updated last year
- moss chat finetuning☆50Updated 9 months ago
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆36Updated last year
- share data, prompt data , pretraining data☆35Updated last year
- 用于微调LLM的中文指令数据集☆27Updated last year
- ☆36Updated 4 months ago
- This repository provides an implementation of the paper "A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Co…☆52Updated 3 weeks ago
- ☆88Updated last month
- ☆45Updated 7 months ago
- ChatGLM2-6B微调, SFT/LoRA, instruction finetune☆105Updated last year
- 纯c++的全平台llm加速库,支持python调用,支持baichuan, glm, llama, moss基座,手机端流畅运行chatglm-6B级模型单卡可达10000+token / s,☆45Updated last year
- ☆24Updated last year