Use the tokenizer in parallel to achieve superior acceleration
☆20Mar 21, 2024Updated last year
Alternatives and similar repositories for ParallelTokenizer
Users that are interested in ParallelTokenizer are comparing it to the libraries listed below
Sorting:
- ☆18Mar 10, 2023Updated 2 years ago
- Advanced Formal Language Theory (263-5352-00L; Frühjahr 2023)☆10Feb 21, 2023Updated 3 years ago
- source code for NAACL2022 main conference "Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs"☆10Sep 26, 2022Updated 3 years ago
- Official Implementation of ACL2023: Don't Parse, Choose Spans! Continuous and Discontinuous Constituency Parsing via Autoregressive Span …☆14Aug 25, 2023Updated 2 years ago
- Recursive Bayesian Networks☆11May 11, 2025Updated 9 months ago
- Fine-Tuning Pre-trained Transformers into Decaying Fast Weights☆19Oct 9, 2022Updated 3 years ago
- Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spaces, NeurIPS 2021☆14Dec 11, 2021Updated 4 years ago
- ☆13Feb 7, 2023Updated 3 years ago
- Code for the ACL 2021 paper "Structural Guidance for Transformer Language Models"☆13Sep 17, 2025Updated 5 months ago
- ☆16Nov 5, 2018Updated 7 years ago
- This is the code repo for the paper <UTC-IE: A Unified Token-pair Classification Architecture for Information Extraction>☆15Aug 10, 2023Updated 2 years ago
- Implementation and experiments for Partially Supervised NER via Expected Entity Ratio in TACL 2022☆14Nov 7, 2022Updated 3 years ago
- Expanding linear RNN state-transition matrix eigenvalues to include negatives improves state-tracking tasks and language modeling without…☆20Mar 15, 2025Updated 11 months ago
- ☆129Jun 6, 2025Updated 8 months ago
- Implementation of fast algorithms for Maximum Spanning Tree (MST) parsing that includes fast ArcMax+Reweighting+Tarjan algorithm for sing…☆16Aug 9, 2023Updated 2 years ago
- ☆15Mar 22, 2023Updated 2 years ago
- ☆14Nov 20, 2022Updated 3 years ago
- 基于树形条件随机场的高阶句法分析☆16Apr 28, 2022Updated 3 years ago
- Basic exercises of chinese information processing☆34Sep 1, 2021Updated 4 years ago
- Source code of paper “A Novel Three-Stage Learning Framework for Low-Resource Knowledge-Grounded Dialogue Generation”☆16Nov 25, 2021Updated 4 years ago
- Learning to Model Editing Processes☆26Aug 3, 2025Updated 6 months ago
- Using conversational games to evaluate powerful LLMs☆18Sep 3, 2023Updated 2 years ago
- 🍼 Official implementation of Dynamic Data Mixing Maximizes Instruction Tuning for Mixture-of-Experts☆41Sep 29, 2024Updated last year
- Efficient PScan implementation in PyTorch☆17Jan 2, 2024Updated 2 years ago
- ☆18May 28, 2021Updated 4 years ago
- ☆29Dec 31, 2025Updated last month
- Simple and efficient pytorch-native transformer training and inference (batched)☆79Apr 2, 2024Updated last year
- Official Repository for Efficient Linear-Time Attention Transformers.☆18Jun 2, 2024Updated last year
- code for Scaling Laws of RoPE-based Extrapolation☆73Oct 16, 2023Updated 2 years ago
- [NeurIPS 2024] Can Language Models Learn to Skip Steps?☆22Jan 25, 2025Updated last year
- The official code repository for the FullFront benchmark☆26May 16, 2025Updated 9 months ago
- u-MPS implementation and experimentation code used in the paper Tensor Networks for Probabilistic Sequence Modeling (https://arxiv.org/ab…☆19Jul 2, 2020Updated 5 years ago
- Python package for serving a local search engine. One command to download and serve a datastore---that's it 😎.☆25Jun 6, 2025Updated 8 months ago
- A probabilitic model for contextual word representation. Accepted to ACL2023 Findings.☆25Oct 22, 2023Updated 2 years ago
- ☆23Dec 8, 2022Updated 3 years ago
- A Structured Span Selector (NAACL 2022). A structured span selector with a WCFG for span selection tasks (coreference resolution, semanti…☆21Jul 11, 2022Updated 3 years ago
- ☆20Jul 5, 2024Updated last year
- 一种面向中文复杂问句的查询图生成方法，以及一份含有多种复杂句的中文知识图谱问答数据集☆18Mar 16, 2023Updated 2 years ago
- A reference implementation of algorithms for distributions over spanning trees.☆21Mar 10, 2020Updated 5 years ago