Implementation of the dilated self attention as described in "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
☆13Jul 23, 2023Updated 3 years ago
Alternatives and similar repositories for dilated-self-attention
Users that are interested in dilated-self-attention are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- (Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens" (https://arxiv.org/abs/2307…☆52Aug 7, 2023Updated 3 years ago
- Implementation of the BERT model using LongNet☆10Sep 29, 2026Updated 3 weeks ago
- 🚂 Fine-tune OpenAI models for text classification, question answering, and more☆17May 1, 2023Updated 3 years ago
- Code for the paper Adversarial Robustness via Adversarial Label-Smoothing☆12Feb 5, 2020Updated 6 years ago
- Training SLMs for reasoning-intensive re-ranking☆11Jul 18, 2025Updated last year
- End-to-end encrypted cloud storage - Proton Drive • AdSpecial offer: 40% Off Yearly / 80% Off First Month. Protect your most important files, photos, and documents from prying eyes.
- [NAACL 2018] Robust Sequence Labeling with Adversarial Training☆10Sep 30, 2019Updated 7 years ago
- 🤗 HuggingFace Inference Toolkit for Google Cloud Vertex AI (similar to SageMaker's Inference Toolkit, but for Vertex AI and unofficial)☆17Mar 20, 2024Updated 2 years ago
- Investigation into whether Transformers and self-supervised learning could be used to trade currency markets☆10Jun 21, 2023Updated 3 years ago
- A proposed standard `NOCK` for a Parquet format that supports efficient distributed serialization of multiple kinds of graph technologies☆21Apr 27, 2026Updated 5 months ago
- Emacs Lisp color manipulation utilities.☆13Jul 16, 2024Updated 2 years ago
- ☆14Oct 31, 2023Updated 2 years ago
- ☆11Dec 26, 2018Updated 7 years ago
- Rust derive macros for automating the boring stuff.☆14Aug 3, 2025Updated last year
- A Python module for retrieving script types of writing systems including alphabets, abjads, abugidas, syllabaries, logographs, featurals …☆15Aug 21, 2026Updated 2 months ago
- Proton VPN Special Offer - Get 70% off • AdSpecial partner offer. Trusted by over 100 million users worldwide. Tested, Approved and Recommended by Experts.
- recreation of the classic drug trading game "dope wars"☆11May 9, 2019Updated 7 years ago
- Alternate LLM Architecture Experiment with O(n) time complexity☆11Jul 18, 2026Updated 3 months ago
- A small spreadsheet demo in Rust, Yew, and WASM☆11Jun 16, 2023Updated 3 years ago
- 🧬 Typescript Genetic Algorithm Framework built using deno☆13Jun 7, 2022Updated 4 years ago
- Compressing token embeddings for transformer-based language models using a novel orientation-based loss objective (accepted to RepL4NLP w…☆10Sep 21, 2022Updated 4 years ago
- Life before `main()`☆19Feb 2, 2021Updated 5 years ago
- Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"☆727Jan 7, 2024Updated 2 years ago
- Bleeding edge low level Rust binding for GGML☆18Jun 26, 2024Updated 2 years ago
- Developing, training, and assessing the performance of a Proximal Policy Optimization (PPO) Stock Trading Agent.☆14Aug 20, 2025Updated last year
- Deploy on Railway without the complexity - Free Credits Offer • AdConnect your repo and Railway handles the rest with instant previews. Quickly provision container image services, databases, and storage volumes.
- RWKV model implementation☆37Jul 15, 2023Updated 3 years ago
- Keyphrase Extraction Prototypes☆15Nov 24, 2016Updated 9 years ago
- Library for evaluating RAG using Nuclia's models☆19Jul 31, 2024Updated 2 years ago
- Official Repository for Efficient Linear-Time Attention Transformers.☆17Jun 2, 2024Updated 2 years ago
- Compute the variation of dice coefficient loss for real-value regression task.☆19Aug 8, 2019Updated 7 years ago
- Rhythm analysis toolkit in Python☆13Jul 5, 2026Updated 3 months ago
- The code of Self-Training Multi-Sequence Learning with Transformer for Weakly Supervised Video Anomaly Detection☆24Nov 23, 2022Updated 3 years ago
- 🌏 Modular retrievers for zero-shot multilingual IR.☆30Mar 6, 2024Updated 2 years ago
- Implementation for Revela: Dense Retriever Learning via Language Modeling - ICLR 2026 Oral☆24Mar 26, 2026Updated 6 months ago
- Managed hosting for WordPress and PHP on Cloudways • AdManaged hosting for WordPress, Magento, Laravel, or PHP apps, on multiple cloud providers. Deploy in minutes on Cloudways by DigitalOcean.
- ☆24Jul 10, 2026Updated 3 months ago
- Rank-DistiLLM: Closing the Effectiveness Gap Between Cross-Encoders and LLMs for Passage Re-Ranking☆26Apr 4, 2025Updated last year
- Babel plugin for Regex+☆15Dec 16, 2025Updated 10 months ago
- Make SharedWorker works like Remote Procedure Call easily☆10Updated this week
- SMS gateway for sending text messages☆13Apr 9, 2023Updated 3 years ago
- Free Cloud JSON Storage Written in Python☆14Feb 2, 2023Updated 3 years ago
- ☆23Jan 14, 2014Updated 12 years ago