hhnqqq / GemmaLongText
☆16Updated last year
Alternatives and similar repositories for GemmaLongText:
Users that are interested in GemmaLongText are comparing it to the libraries listed below
- Research without Re-search: Maximal Update Parametrization Yields Accurate Loss Prediction across Scales☆32Updated last year
- This is a personal reimplementation of Google's Infini-transformer, utilizing a small 2b model. The project includes both model and train…☆56Updated last year
- ☆46Updated 10 months ago
- code for Scaling Laws of RoPE-based Extrapolation☆73Updated last year
- ☆16Updated 2 weeks ago
- SELF-GUIDE: Better Task-Specific Instruction Following via Self-Synthetic Finetuning. COLM 2024 Accepted Paper☆32Updated 11 months ago
- CLongEval: A Chinese Benchmark for Evaluating Long-Context Large Language Models☆40Updated last year
- [ACL'24] Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning☆148Updated 7 months ago
- [ICLR 2024] CLEX: Continuous Length Extrapolation for Large Language Models☆76Updated last year
- Code for paper "Patch-Level Training for Large Language Models"☆84Updated 5 months ago
- Longitudinal Evaluation of LLMs via Data Compression☆32Updated 11 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆63Updated last year
- A prototype repo for hybrid training of pipeline parallel and distributed data parallel with comments on core code snippets. Feel free to…☆55Updated last year
- ☆27Updated 2 months ago
- Low-bit optimizers for PyTorch☆128Updated last year
- this is an implementation for the paper Improve Mathematical Reasoning in Language Models by Automated Process Supervision from google de…☆28Updated 3 weeks ago
- MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering. A comprehensive evaluation of multimodal large model multilingua…☆56Updated last month
- ☆14Updated last year
- Pretrain、decay、SFT a CodeLLM from scratch 🧙♂️☆35Updated 11 months ago
- ☆98Updated 6 months ago
- We systematically studied the influencing factors when LLM generates benchmarks,By using our code, you can generate high-quality QA datas…☆17Updated 2 months ago
- Source code of paper ''KVSharer: Efficient Inference via Layer-Wise Dissimilar KV Cache Sharing''☆25Updated 6 months ago
- [NeurIPS 2024] Fast Best-of-N Decoding via Speculative Rejection☆42Updated 6 months ago
- ☆37Updated 2 weeks ago
- Efficient Mixture of Experts for LLM Paper List☆61Updated 4 months ago
- [ICLR 2025] MiniPLM: Knowledge Distillation for Pre-Training Language Models☆40Updated 5 months ago
- Unofficial implementations of block/layer-wise pruning methods for LLMs.☆67Updated last year
- Official completion of “Training on the Benchmark Is Not All You Need”.☆31Updated 3 months ago
- [ICML'24] The official implementation of “Rethinking Optimization and Architecture for Tiny Language Models”☆121Updated 3 months ago
- [ACL 2024] Long-Context Language Modeling with Parallel Encodings☆154Updated 10 months ago