Use the tokenizer in parallel to achieve superior acceleration
☆20Mar 21, 2024Updated last year
Alternatives and similar repositories for ParallelTokenizer
Users that are interested in ParallelTokenizer are comparing it to the libraries listed below
Sorting:
- ☆20Jul 5, 2024Updated last year
- ☆18Mar 10, 2023Updated 3 years ago
- Fine-Tuning Pre-trained Transformers into Decaying Fast Weights☆19Oct 9, 2022Updated 3 years ago
- Official Implementation of ACL2023: Don't Parse, Choose Spans! Continuous and Discontinuous Constituency Parsing via Autoregressive Span …☆14Aug 25, 2023Updated 2 years ago
- Recursive Bayesian Networks☆11May 11, 2025Updated 10 months ago
- Code for M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models☆23Jul 27, 2024Updated last year
- source code for NAACL2022 main conference "Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs"☆10Sep 26, 2022Updated 3 years ago
- ☆31Dec 31, 2025Updated 2 months ago
- ☆13Feb 7, 2023Updated 3 years ago
- [NeurIPS 2024] Can Language Models Learn to Skip Steps?☆22Jan 25, 2025Updated last year
- code for Scaling Laws of RoPE-based Extrapolation☆73Oct 16, 2023Updated 2 years ago
- Advanced Formal Language Theory (263-5352-00L; Frühjahr 2023)☆10Feb 21, 2023Updated 3 years ago
- ☆132Jun 6, 2025Updated 9 months ago
- Implementation of fast algorithms for Maximum Spanning Tree (MST) parsing that includes fast ArcMax+Reweighting+Tarjan algorithm for sing…☆16Aug 9, 2023Updated 2 years ago
- Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spaces, NeurIPS 2021☆14Dec 11, 2021Updated 4 years ago
- Simple and efficient pytorch-native transformer training and inference (batched)☆79Apr 2, 2024Updated last year
- Implementation and experiments for Partially Supervised NER via Expected Entity Ratio in TACL 2022☆14Nov 7, 2022Updated 3 years ago
- Expanding linear RNN state-transition matrix eigenvalues to include negatives improves state-tracking tasks and language modeling without…☆21Mar 15, 2025Updated last year
- Code for the ACL 2021 paper "Structural Guidance for Transformer Language Models"☆13Sep 17, 2025Updated 6 months ago
- ☆18May 28, 2021Updated 4 years ago
- 基于树形条件随机场的高阶句法分析☆16Apr 28, 2022Updated 3 years ago
- Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"☆27Apr 17, 2024Updated last year
- ☆15Mar 22, 2023Updated 2 years ago
- Official repository of paper "LOVE-R1: Advancing Long Video Understanding with Adaptive Zoom-in Mechanism via Multi-Step Reasoning"☆22Nov 1, 2025Updated 4 months ago
- ☆14Nov 20, 2022Updated 3 years ago
- Learning to Model Editing Processes☆26Aug 3, 2025Updated 7 months ago
- Official Repository for Efficient Linear-Time Attention Transformers.☆18Jun 2, 2024Updated last year
- A reference implementation of algorithms for distributions over spanning trees.☆21Mar 10, 2020Updated 6 years ago
- Efficient PScan implementation in PyTorch☆17Jan 2, 2024Updated 2 years ago
- ☆16Nov 5, 2018Updated 7 years ago
- A Structured Span Selector (NAACL 2022). A structured span selector with a WCFG for span selection tasks (coreference resolution, semanti…☆21Jul 11, 2022Updated 3 years ago
- The code for “PromptNER: A Prompting Method for Few-shot Named Entity Recognition via k Nearest Neighbor Search”☆19Mar 13, 2024Updated 2 years ago
- Python package for serving a local search engine. One command to download and serve a datastore---that's it 😎.☆25Jun 6, 2025Updated 9 months ago
- [ICLR 2025 Spotlight] Weak-to-strong preference optimization: stealing reward from weak aligned model☆17Feb 24, 2025Updated last year
- This is the code repo for the paper <UTC-IE: A Unified Token-pair Classification Architecture for Information Extraction>☆15Aug 10, 2023Updated 2 years ago
- u-MPS implementation and experimentation code used in the paper Tensor Networks for Probabilistic Sequence Modeling (https://arxiv.org/ab…☆19Jul 2, 2020Updated 5 years ago
- Researching the forward-backward algorithm☆11Aug 3, 2018Updated 7 years ago
- Basic exercises of chinese information processing☆34Sep 1, 2021Updated 4 years ago
- Reproducing R1 for Code with Reliable Rewards☆12Apr 9, 2025Updated 11 months ago