cg123 / bitnet
Modeling code for a BitNet b1.58 Llama-style model.
☆23Updated 10 months ago
Alternatives and similar repositories for bitnet:
Users that are interested in bitnet are comparing it to the libraries listed below
- Parameter-Efficient Sparsity Crafting From Dense to Mixture-of-Experts for Instruction Tuning on General Tasks☆31Updated 10 months ago
- RWKV-7: Surpassing GPT☆81Updated 4 months ago
- ☆27Updated last year
- ☆49Updated last year
- ☆126Updated 7 months ago
- Demonstration that finetuning RoPE model on larger sequences than the pre-trained model adapts the model context limit☆63Updated last year
- QuIP quantization☆52Updated last year
- ☆31Updated 9 months ago
- entropix style sampling + GUI☆25Updated 4 months ago
- GoldFinch and other hybrid transformer components☆45Updated 8 months ago
- My fork os allen AI's OLMo for educational purposes.☆30Updated 3 months ago
- ☆80Updated 2 months ago
- An easy-to-understand framework for LLM samplers that rewind and revise generated tokens☆137Updated last month
- ☆48Updated 4 months ago
- The simplest, fastest repository for training/finetuning medium-sized xLSTMs.☆42Updated 9 months ago
- implementation of https://arxiv.org/pdf/2312.09299☆20Updated 8 months ago
- ☆73Updated 7 months ago
- Entropy Based Sampling and Parallel CoT Decoding☆17Updated 5 months ago
- Model REVOLVER, a human in the loop model mixing system.☆33Updated last year
- Implementation of the Mamba SSM with hf_integration.☆56Updated 6 months ago
- My Implementation of Q-Sparse: All Large Language Models can be Fully Sparsely-Activated☆31Updated 7 months ago
- A public implementation of the ReLoRA pretraining method, built on Lightning-AI's Pytorch Lightning suite.☆33Updated last year
- Testing LLM reasoning abilities with family relationship quizzes.☆62Updated last month