Qurrent-AI / RES-Q
RES-Q: Evaluating the Code-Editing Capability of Large Language Model Systems at the Repository Scale
☆26Updated 10 months ago
Alternatives and similar repositories for RES-Q:
Users that are interested in RES-Q are comparing it to the libraries listed below
- Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions☆41Updated 8 months ago
- ☆73Updated this week
- Just a bunch of benchmark logs for different LLMs☆119Updated 9 months ago
- Scaling is a distributed training library and installable dependency designed to scale up neural networks, with a dedicated module for tr…☆58Updated 5 months ago
- ☆37Updated 3 months ago
- Vivaria is METR's tool for running evaluations and conducting agent elicitation research.☆90Updated this week
- ☆128Updated 3 weeks ago
- ☆54Updated 7 months ago
- Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.☆82Updated last year
- Small, simple agent task environments for training and evaluation☆18Updated 5 months ago
- prime-rl is a codebase for decentralized RL training at scale☆79Updated this week
- Archon provides a modular framework for combining different inference-time techniques and LMs with just a JSON config file.☆171Updated last month
- Functional Benchmarks and the Reasoning Gap☆85Updated 6 months ago
- Repository for the paper Stream of Search: Learning to Search in Language☆145Updated 2 months ago
- ☆114Updated 2 months ago
- Public Inflection Benchmarks☆68Updated last year
- Mixing Language Models with Self-Verification and Meta-Verification☆104Updated 4 months ago
- ☆90Updated last month
- Train your own SOTA deductive reasoning model☆88Updated last month
- Sphynx Hallucination Induction☆53Updated 2 months ago
- ☆91Updated 9 months ago
- ☆72Updated this week
- OpenCoconut implements a latent reasoning paradigm where we generate thoughts before decoding.☆171Updated 3 months ago
- Long context evaluation for large language models☆207Updated last month
- RepoQA: Evaluating Long-Context Code Understanding☆107Updated 5 months ago
- ☆53Updated last week
- ☆21Updated 3 weeks ago
- MiniHF is an inference, human preference data collection, and fine-tuning tool for local language models. It is intended to help the user…☆171Updated this week
- The simplest, fastest repository for training/finetuning medium-sized GPTs.☆105Updated 5 months ago
- Verdict is a library for scaling judge-time compute.☆199Updated last week