DaertML / context_distillation
Framework to achieve context distillation in LLMs
☆11Updated last year
Alternatives and similar repositories for context_distillation:
Users that are interested in context_distillation are comparing it to the libraries listed below
- This is the oficial repository for "Safer-Instruct: Aligning Language Models with Automated Preference Data"☆17Updated 10 months ago
- In-Context Alignment: Chat with Vanilla Language Models Before Fine-Tuning☆33Updated last year
- The repository contains code for Adaptive Data Optimization☆21Updated last month
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆36Updated last year
- Reference implementation for Reward-Augmented Decoding: Efficient Controlled Text Generation With a Unidirectional Reward Model☆42Updated last year
- Data preparation code for CrystalCoder 7B LLM☆43Updated 8 months ago
- LLMs as Collaboratively Edited Knowledge Bases☆43Updated 10 months ago
- NeurIPS 2024 tutorial on LLM Inference☆37Updated last month
- Code for the arXiv preprint "The Unreasonable Effectiveness of Easy Training Data"☆46Updated last year
- From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu,…☆42Updated 6 months ago
- ☆31Updated 7 months ago
- A new way to generate large quantities of high quality synthetic data (on par with GPT-4), with better controllability, at a fraction of …☆21Updated 3 months ago
- Open sourced backend for Martian's LLM Inference Provider Leaderboard☆17Updated 5 months ago
- ☆64Updated 9 months ago
- Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and min…☆23Updated 2 months ago
- [ICML 24 NGSM workshop] Associative Recurrent Memory Transformer implementation and scripts for training and evaluating☆33Updated this week
- Open Implementations of LLM Analyses☆96Updated 3 months ago
- This is a new metric that can be used to evaluate faithfulness of text generated by LLMs. The work behind this repository can be found he…☆31Updated last year
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆25Updated 9 months ago
- Nexusflow function call, tool use, and agent benchmarks.☆18Updated last month
- ☆25Updated last year
- Pre-training code for CrystalCoder 7B LLM☆55Updated 8 months ago
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated 10 months ago
- ☆30Updated 3 months ago
- Code for the examples presented in the talk "Training a Llama in your backyard: fine-tuning very large models on consumer hardware" given…☆14Updated last year
- Linear Attention Sequence Parallelism (LASP)☆74Updated 7 months ago
- Q-Probe: A Lightweight Approach to Reward Maximization for Language Models☆40Updated 7 months ago
- Repository for CPU Kernel Generation for LLM Inference☆25Updated last year