hpcaitech / Titans
A collection of models built with ColossalAI
☆32Updated last year
Related projects: ⓘ
- Ouroboros: Speculative Decoding with Large Model Enhanced Drafting☆60Updated 6 months ago
- Scalable PaLM implementation of PyTorch☆191Updated last year
- [EMNLP 2023 Industry Track] A simple prompting approach that enables the LLMs to run inference in batches.☆65Updated 6 months ago
- Fast LLM Training CodeBase With dynamic strategy choosing [Deepspeed+Megatron+FlashAttention+CudaFusionKernel+Compiler];☆32Updated 8 months ago
- ☆50Updated this week
- A memory efficient DLRM training solution using ColossalAI☆95Updated last year
- ☆52Updated 2 months ago
- A repository sharing the literatures about large language models☆19Updated last month
- Large Scale Distributed Model Training strategy with Colossal AI and Lightning AI☆58Updated last year
- ☆99Updated last year
- Python tools for processing the stackexchange data dumps into a text dataset for Language Models☆74Updated 9 months ago
- An Experiment on Dynamic NTK Scaling RoPE☆59Updated 9 months ago
- ☆73Updated last year
- ☆29Updated last year
- ☆108Updated 6 months ago
- A plug-in of Microsoft DeepSpeed to fix the bug of DeepSpeed pipeline☆26Updated 3 years ago
- ☆105Updated last week
- ☆75Updated 5 months ago
- Official implementation for 'Extending LLMs’ Context Window with 100 Samples'☆72Updated 8 months ago
- Techniques used to run BLOOM at inference in parallel☆37Updated last year
- ☆23Updated last year
- Repository for analysis and experiments in the BigCode project.☆113Updated 6 months ago
- REST: Retrieval-Based Speculative Decoding, NAACL 2024☆158Updated 4 months ago
- Implementation of Speculative Sampling as described in "Accelerating Large Language Model Decoding with Speculative Sampling" by Deepmind☆69Updated 6 months ago
- An experimental implementation of the retrieval-enhanced language model☆75Updated last year
- Implementation of NAACL 2024 Outstanding Paper "LM-Infinite: Simple On-the-Fly Length Generalization for Large Language Models"☆115Updated 2 weeks ago
- LMTuner: Make the LLM Better for Everyone☆33Updated last year
- Evaluating LLMs with Dynamic Data☆66Updated 2 weeks ago
- A dataset of LLM-generated chain-of-thought steps annotated with mistake location.☆65Updated last month
- Implementation of the paper: "Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention" from Google in pyTO…☆48Updated last week