pp1230 / LLMGPUMemEstimator
The GPU RAM Estimator provides a simple tool for estimating GPU memory usage during training and inference.
☆32Updated 10 months ago
Alternatives and similar repositories for LLMGPUMemEstimator:
Users that are interested in LLMGPUMemEstimator are comparing it to the libraries listed below
- ☆45Updated 8 months ago
- Llama-3-SynE: A Significantly Enhanced Version of Llama-3 with Advanced Scientific Reasoning and Chinese Language Capabilities | 继续预训练提升 …☆33Updated 2 months ago
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆56Updated 10 months ago
- Dataset and evaluation script for "Evaluating Hallucinations in Chinese Large Language Models"☆118Updated 8 months ago
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆38Updated 11 months ago
- Counting-Stars (★)☆78Updated 5 months ago
- Code for a New Loss for Mitigating the Bias of Learning Difficulties in Generative Language Models☆58Updated this week
- Clustering and Ranking: Diversity-preserved Instruction Selection through Expert-aligned Quality Estimation☆74Updated 3 months ago
- ☆121Updated last year
- 怎么训练一个LLM分词器☆140Updated last year
- 中文大语言模型评测第二期☆70Updated last year
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆142Updated 5 months ago
- ☆96Updated 11 months ago
- ☆162Updated last year
- ☆139Updated 7 months ago
- ☆57Updated 2 months ago
- MEASURING MASSIVE MULTITASK CHINESE UNDERSTANDING☆88Updated 10 months ago
- ☆80Updated last year
- Benchmarking Complex Instruction-Following with Multiple Constraints Composition (NeurIPS 2024 Datasets and Benchmarks Track)☆64Updated this week
- 百川Dynamic NTK-ALiBi的代码实现:无需微调即可推理更长文本☆47Updated last year
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆153Updated 8 months ago
- A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…☆55Updated last year
- ☆93Updated last year
- NTK scaled version of ALiBi position encoding in Transformer.☆67Updated last year
- 基于DPO算法微调语言大模型,简单好上手。☆30Updated 7 months ago
- ☆84Updated last year
- Implementations of online merging optimizers proposed by Online Merging Optimizers for Boosting Rewards and Mitigating Tax in Alignment☆73Updated 8 months ago
- Train llm (bloom, llama, baichuan2-7b, chatglm3-6b) with deepspeed pipeline mode. Faster than zero/zero++/fsdp.☆92Updated last year
- Official implementation of the paper "From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large L…☆44Updated 7 months ago
- 1.4B sLLM for Chinese and English - HammerLLM🔨☆44Updated 10 months ago