huggingface / fineweb-2Links
☆195Updated 3 months ago
Alternatives and similar repositories for fineweb-2
Users that are interested in fineweb-2 are comparing it to the libraries listed below
Sorting:
- Manage scalable open LLM inference endpoints in Slurm clusters☆274Updated last year
- Complex Function Calling Benchmark.☆135Updated 8 months ago
- PyTorch building blocks for the OLMo ecosystem☆305Updated this week
- Official repository for paper "ReasonIR Training Retrievers for Reasoning Tasks".☆203Updated 3 months ago
- EvolKit is an innovative framework designed to automatically enhance the complexity of instructions used for fine-tuning Large Language M…☆240Updated 11 months ago
- Official repository for "Scaling Retrieval-Based Langauge Models with a Trillion-Token Datastore".☆216Updated 2 months ago
- Benchmarking LLMs with Challenging Tasks from Real Users☆241Updated 11 months ago
- ☆155Updated last year
- The official evaluation suite and dynamic data release for MixEval.☆249Updated 11 months ago
- Let's build better datasets, together!☆262Updated 9 months ago
- This is the official repository for Inheritune.☆115Updated 8 months ago
- Reproducible, flexible LLM evaluations☆252Updated 2 months ago
- Accelerating your LLM training to full speed! Made with ❤️ by ServiceNow Research☆237Updated last week
- code for training & evaluating Contextual Document Embedding models☆197Updated 4 months ago
- ☆119Updated last year
- ☆136Updated last month
- BABILong is a benchmark for LLM evaluation using the needle-in-a-haystack approach.☆212Updated last month
- Pre-training code for Amber 7B LLM☆168Updated last year
- LongEmbed: Extending Embedding Models for Long Context Retrieval (EMNLP 2024)☆143Updated 11 months ago
- A pipeline for LLM knowledge distillation☆109Updated 6 months ago
- Code for Zero-Shot Tokenizer Transfer☆138Updated 8 months ago
- Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.☆243Updated 2 months ago
- LongRoPE is a novel method that can extends the context window of pre-trained LLMs to an impressive 2048k tokens.☆260Updated last year
- 🚢 Data Toolkit for Sailor Language Models☆94Updated 7 months ago
- ☆128Updated last year
- LOFT: A 1 Million+ Token Long-Context Benchmark☆212Updated 3 months ago
- Organize the Web: Constructing Domains Enhances Pre-Training Data Curation☆66Updated 5 months ago
- A simplified implementation for experimenting with RLVR on GSM8K, This repository provides a starting point for exploring reasoning.☆129Updated 8 months ago
- Official code for "MAmmoTH2: Scaling Instructions from the Web" [NeurIPS 2024]☆148Updated 11 months ago
- Official repo for "Make Your LLM Fully Utilize the Context"☆254Updated last year