fkodom / dilated-attention-pytorch

(Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens" (https://arxiv.org/abs/2307.02486)
51Updated last year

Related projects

Alternatives and complementary repositories for dilated-attention-pytorch