eole-nlp / eole
Open language modeling toolkit based on PyTorch
☆100Updated this week
Alternatives and similar repositories for eole:
Users that are interested in eole are comparing it to the libraries listed below
- The FLORES+ Machine Translation Benchmark☆101Updated 4 months ago
- Bicleaner fork that uses neural networks☆39Updated 7 months ago
- A library for preparing data for machine translation research (monolingual preprocessing, bitext mining, etc.) built by the FAIR NLLB te…☆270Updated 2 months ago
- OpusFilter - Parallel corpus processing toolkit☆104Updated this week
- OpusCleaner is a web interface that helps you select, clean and schedule your data for training machine translation models.☆50Updated 2 months ago
- NTREX -- News Test References for MT Evaluation☆81Updated 9 months ago
- ☆242Updated 9 months ago
- A tool that locates, downloads, and extracts machine translation corpora☆154Updated this week
- A neural word aligner based on multilingual BERT☆342Updated 3 years ago
- Fine-tuning Open-Source LLMs for Adaptive Machine Translation☆76Updated 2 weeks ago
- Glot500: Scaling Multilingual Corpora and Language Models to 500 Languages -- ACL 2023☆100Updated 11 months ago
- ☆57Updated 2 years ago
- Multilingual sentence alignment using sentence embeddings☆113Updated 4 months ago
- Neural Machine Translation (NMT) tutorial. Data preprocessing, model training, evaluation, and deployment.☆160Updated 11 months ago
- Bicleaner is a parallel corpus classifier/cleaner that aims at detecting noisy sentence pairs in a parallel corpus.☆155Updated 9 months ago
- Curriculum training☆17Updated 2 weeks ago
- Efficient Low-Memory Aligner☆142Updated 2 months ago
- Improved Sentence Alignment in Linear Time and Space☆169Updated 2 years ago
- 💬 Language Identification with Support for More Than 2000 Labels -- EMNLP 2023☆122Updated 3 months ago
- Enhancing Translation with RAG-Powered Large Language Models☆76Updated 2 months ago
- GEMBA — GPT Estimation Metric Based Assessment☆113Updated 7 months ago
- Open information and community for machine translation☆74Updated last month
- ☆85Updated 3 months ago
- Repository accompanying "An Open Dataset and Model for Language Identification" (Burchell et al., 2023)☆70Updated 11 months ago
- cLang-8 is a dataset for grammatical error correction.☆103Updated 2 years ago
- The Open Parallel Corpus☆67Updated 3 weeks ago
- ☆25Updated last year
- BigTranslate: Augmenting Large Language Models with Multilingual Translation Capability over 100 Languages☆221Updated last year
- Machine Translation (MT) Preparation Scripts☆31Updated last month
- ☆72Updated 3 weeks ago