normster / llm_rules
RuLES: a benchmark for evaluating rule-following in language models
☆215Updated this week
Alternatives and similar repositories for llm_rules:
Users that are interested in llm_rules are comparing it to the libraries listed below
- ☆93Updated 6 months ago
- Multipack distributed sampler for fast padding-free training of LLMs☆184Updated 5 months ago
- Scaling Data-Constrained Language Models☆330Updated 3 months ago
- Code for the paper "Fishing for Magikarp"☆139Updated this week
- Simple next-token-prediction for RLHF☆222Updated last year
- A curated reading list of research in Adaptive Computation, Inference-Time Computation & Mixture of Experts (MoE).☆136Updated 2 weeks ago
- ☆115Updated 3 months ago
- ☆135Updated 3 months ago
- A simple unified framework for evaluating LLMs☆164Updated 3 weeks ago
- Self-Alignment with Principle-Following Reward Models☆150Updated 10 months ago
- Extract full next-token probabilities via language model APIs☆230Updated 10 months ago
- Manage scalable open LLM inference endpoints in Slurm clusters☆247Updated 6 months ago
- Evaluating LLMs with fewer examples☆141Updated 9 months ago
- Code for the paper "Rethinking Benchmark and Contamination for Language Models with Rephrased Samples"☆297Updated last year
- Code to reproduce "Transformers Can Do Arithmetic with the Right Embeddings", McLeish et al (NeurIPS 2024)☆182Updated 7 months ago
- The official evaluation suite and dynamic data release for MixEval.☆233Updated 2 months ago
- ☆80Updated 3 months ago
- Replicating O1 inference-time scaling laws☆70Updated last month
- Fast bare-bones BPE for modern tokenizer training☆141Updated 2 months ago
- Steering vectors for transformer language models in Pytorch / Huggingface☆78Updated last month
- A puzzle to learn about prompting☆123Updated last year
- Improving Alignment and Robustness with Circuit Breakers☆174Updated 3 months ago
- Language models scale reliably with over-training and on downstream tasks☆96Updated 9 months ago
- ☆135Updated this week
- Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.☆154Updated 2 months ago
- NeurIPS Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day☆253Updated last year
- Understand and test language model architectures on synthetic tasks.☆175Updated this week
- ☆161Updated last year
- Functional Benchmarks and the Reasoning Gap☆82Updated 3 months ago
- ☆150Updated last year