Alternatives and similar repositories for code_tokenize

Users that are interested in code_tokenize are comparing it to the libraries listed below

• giganticode / codeprep
  A toolkit for pre-processing large source code corpora
  ☆47Updated 3 years ago
• cedricrupb / TSSB3M
  Mining tool and large-scale datasets of single statement bug fixes in Python
  ☆18Updated last year
• IBM / tree-sitter-codeviews
  Extract and combine multiple source code views using tree-sitter
  ☆153Updated last month
• saltudelft / many-types-4-py-dataset
  ManyTypes4Py: A benchmark Python dataset for machine learning-based type inference
  ☆24Updated 3 years ago
• utopia-group / TypeT5
  Seq2seq Type Inference using Static Analysis and CodeT5
  ☆32Updated 2 years ago
• google-research-datasets / great
  The dataset for the variable-misuse task, used in the ICLR 2020 paper 'Global Relational Models of Source Code' [https://openreview.net/f…
  ☆22Updated 5 years ago
• eth-sri / TFix
  ☆68Updated 3 years ago
• VHellendoorn / ICLR20-Great
  Data and Code for Reproducing "Global Relational Models of Source Code"
  ☆84Updated 4 years ago
• google-research / plur
  PLUR (Programming-Language Understanding and Repair) is a collection of source code datasets suitable for graph-based machine learning. W…
  ☆87Updated 3 years ago
• saltudelft / type4py
  Type4Py: Deep Similarity Learning-Based Type Inference for Python
  ☆65Updated 2 years ago
• danhper / bigcode-tools
  Set of tools to help working with "Big Code"
  ☆43Updated 3 years ago
• zkcpku / HiT-hierarchy-transformer
  code for "Implant Global and Local Hierarchy Information to Sequence based Code Representation Models"
  ☆12Updated 10 months ago
• cedricrupb / code_diff
  Fast AST based code differencing in Python
  ☆34Updated 9 months ago
• cedricrupb / code_ast
  Fast and robust AST parsing of any language
  ☆55Updated 9 months ago
• saltudelft / codefill
  Contains the code and data for our #ICSE2022 paper titled as "CodeFill: Multi-token Code Completion by Jointly Learning from Structure an…
  ☆15Updated 3 years ago
• mast-group / OpenVocabCodeNLM
  Contains the code for our ICSE 2020 paper: Big Code != Big Vocabulary: Open-Vocabulary Language Models for Source Code and for its earlie…
  ☆84Updated 2 years ago
• bdqnghi / awesome-ai4code
  A collection of recent papers, benchmarks and datasets of AI4Code domain.
  ☆58Updated last year
• microsoft / codamosa
  ☆146Updated 5 months ago
• danielzuegner / code-transformer
  Implementation of the paper "Language-agnostic representation learning of source code from structure and context".
  ☆171Updated 3 years ago
• giganticode / run_bug_run
  The RunBugRun dataset of executable bugs
  ☆23Updated last month
• mahimanzum / FixEval
  We introduce FixEval , a dataset for competitive programming bug fixing along with a comprehensive test suite and show the necessity of e…
  ☆24Updated 3 years ago
• panthap2 / LearningToUpdateNLComments
  Learning to Update Natural Language Comments Based on Code Changes: Artifact
  ☆33Updated 5 years ago
• EngineeringSoftware / CoditT5
  CoditT5: Pretraining for Source Code and Natural Language Editing
  ☆28Updated 9 months ago
• saikat107 / Codit
  ☆13Updated 2 years ago
• microsoft / neurips21-self-supervised-bug-detection-and-repair
  Replication Code for "Self-Supervised Bug Detection and Repair" NeurIPS 2021
  ☆111Updated 3 years ago
• tech-srl / slm-code-generation
  TensorFlow code for the neural network presented in the paper: "Structural Language Models of Code" (ICML'2020)
  ☆90Updated 3 years ago
• yuewang-cuhk / awesome-programming-language-pretraining-papers
  Recent Advances in Programming Language Pre-Trained Models (PL-PTMs)
  ☆59Updated 3 years ago
• tech-srl / c3po
  Code for the paper "A Structural Model for Contextual Code Changes"
  ☆32Updated 2 years ago
• AI-nstein / hoppity
  Hoppity
  ☆60Updated 4 years ago
• wasiahmad / AVATAR
  Official code of our work, AVATAR: A Parallel Corpus for Java-Python Program Translation.
  ☆58Updated last year