sdan / selfextend
an implementation of Self-Extend, to expand the context window via grouped attention
☆118Updated 10 months ago
Related projects ⓘ
Alternatives and complementary repositories for selfextend
- inference code for mixtral-8x7b-32kseqlen☆98Updated 11 months ago
- Full finetuning of large language models without large memory requirements☆93Updated 10 months ago
- An easy-to-understand framework for LLM samplers that rewind and revise generated tokens☆113Updated 3 weeks ago
- ☆149Updated 4 months ago
- ☆104Updated 8 months ago
- QLoRA: Efficient Finetuning of Quantized LLMs☆77Updated 7 months ago
- Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.☆81Updated last year
- Low-Rank adapter extraction for fine-tuned transformers model☆162Updated 6 months ago
- Generate Synthetic Data Using OpenAI, MistralAI or AnthropicAI☆221Updated 6 months ago
- Steer LLM outputs towards a certain topic/subject and enhance response capabilities using activation engineering by adding steering vecto…☆203Updated 6 months ago
- Just a bunch of benchmark logs for different LLMs☆114Updated 3 months ago
- ☆48Updated last year
- code for training & evaluating Contextual Document Embedding models☆117Updated this week
- Synthetic Data for LLM Fine-Tuning☆97Updated 11 months ago
- ☆118Updated 3 months ago
- Evaluating LLMs with CommonGen-Lite☆85Updated 8 months ago
- Data preparation code for Amber 7B LLM☆82Updated 6 months ago
- A pipeline for LLM knowledge distillation☆78Updated 3 months ago
- Routing on Random Forest (RoRF)☆84Updated last month
- Fast parallel LLM inference for MLX☆149Updated 4 months ago
- ☆64Updated 5 months ago
- ☆93Updated last month
- Mixing Language Models with Self-Verification and Meta-Verification☆97Updated last year
- ☆72Updated last year
- ☆49Updated 8 months ago
- ☆87Updated 9 months ago
- This is our own implementation of 'Layer Selective Rank Reduction'☆232Updated 5 months ago
- RAFT, or Retrieval-Augmented Fine-Tuning, is a method comprising of a fine-tuning and a RAG-based retrieval phase. It is particularly sui…☆75Updated 2 months ago
- Let's create synthetic textbooks together :)☆70Updated 9 months ago
- Modified Stanford-Alpaca Trainer for Training Replit's Code Model☆40Updated last year