Implementation of the dilated self attention as described in "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
β13Jul 23, 2023Updated 3 years ago
Alternatives and similar repositories for dilated-self-attention
Users that are interested in dilated-self-attention are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- π Fine-tune OpenAI models for text classification, question answering, and moreβ17May 1, 2023Updated 3 years ago
- A RAG that can scale π§π»βπ»β11May 28, 2024Updated 2 years ago
- β10Oct 2, 2024Updated last year
- π€ Trade any tensors over the networkβ31Sep 27, 2023Updated 2 years ago
- β17Jan 5, 2023Updated 3 years ago
- Deploy on Railway without the complexity - Free Credits Offer β’ AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- Knowledgeable Embedding: Injecting dynamically updatable entity knowledge into embeddings to enhance RAGβ15Aug 31, 2025Updated last year
- π€ HuggingFace Inference Toolkit for Google Cloud Vertex AI (similar to SageMaker's Inference Toolkit, but for Vertex AI and unofficial)β17Mar 20, 2024Updated 2 years ago
- A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.β32Sep 19, 2025Updated 11 months ago
- Semantically Search Emojis From the Command Line!β13Nov 26, 2023Updated 2 years ago
- A proposed standard `NOCK` for a Parquet format that supports efficient distributed serialization of multiple kinds of graph technologiesβ21Apr 27, 2026Updated 4 months ago
- Investigation into whether Transformers and self-supervised learning could be used to trade currency marketsβ10Jun 21, 2023Updated 3 years ago
- β14Oct 31, 2023Updated 2 years ago
- β11Dec 26, 2018Updated 7 years ago
- β15Sep 1, 2025Updated last year
- GPUs on demand by Runpod - Special Offer Available β’ AdRun AI, ML, and HPC workloads on powerful cloud GPUsβwithout limits or wasted spend. Deploy GPUs in under a minute and pay by the second.
- [CoLM 24] Official Repository of MambaByte: Token-free Selective State Space Modelβ27Oct 12, 2024Updated last year
- Alternate LLM Architecture Experiment with O(n) time complexityβ11Jul 18, 2026Updated last month
- LongAttn οΌSelecting Long-context Training Data via Token-level Attentionβ15Jul 16, 2025Updated last year
- [ICLR 2023] Eva: Practical Second-order Optimization with Kronecker-vectorized Approximationβ12Jul 31, 2023Updated 3 years ago
- Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.β13Jun 5, 2024Updated 2 years ago
- β14May 6, 2024Updated 2 years ago
- SPRINT Toolkit helps you evaluate diverse neural sparse models easily using a single click on any IR dataset.β48Jul 25, 2023Updated 3 years ago
- Parsing and serialization support for PSSH boxes used in DRM systemsβ16Jul 24, 2026Updated last month
- Life before `main()`β19Feb 2, 2021Updated 5 years ago
- GPU virtual machines on DigitalOcean Gradient AI β’ AdGet to production fast with high-performance AMD and NVIDIA GPUs you can spin up in seconds. The definition of operational simplicity.
- RWKV model implementationβ37Jul 15, 2023Updated 3 years ago
- Library for evaluating RAG using Nuclia's modelsβ19Jul 31, 2024Updated 2 years ago
- DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domainsβ22Feb 7, 2024Updated 2 years ago
- Smart commit messagesβ18Oct 25, 2024Updated last year
- Model implementation for the contextual embeddings projectβ48Jun 2, 2025Updated last year
- [NeurIPS 2024] πΈ GlotCC Dataset and Piplineβ21Apr 6, 2025Updated last year
- Implementation for Revela: Dense Retriever Learning via Language Modeling - ICLR 2026 Oralβ22Mar 26, 2026Updated 5 months ago
- The code of Self-Training Multi-Sequence Learning with Transformer for Weakly Supervised Video Anomaly Detectionβ24Nov 23, 2022Updated 3 years ago
- Log-structured merge-tree implementation in Rustβ19Nov 6, 2018Updated 7 years ago
- Managed Database hosting by DigitalOcean β’ AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- possibly useful materials for learning RWKV language model.β27Jun 8, 2023Updated 3 years ago
- π Modular retrievers for zero-shot multilingual IR.β30Mar 6, 2024Updated 2 years ago
- Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Rankingβ25Apr 4, 2025Updated last year
- Official Repository for Efficient Linear-Time Attention Transformers.β17Jun 2, 2024Updated 2 years ago
- Starbucks: Improved Training for 2D Matryoshka Embeddingsβ25Jun 30, 2025Updated last year
- Official repository for the paper "SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention"β102Sep 30, 2024Updated last year
- β15Oct 28, 2020Updated 5 years ago