dspinellis / tokenizer
Convert source code into numerical tokens
☆65Updated last year
Alternatives and similar repositories for tokenizer:
Users that are interested in tokenizer are comparing it to the libraries listed below
- A toolkit for pre-processing large source code corpora☆47Updated 2 years ago
- Contains the code for our ICSE 2020 paper: Big Code != Big Vocabulary: Open-Vocabulary Language Models for Source Code and for its earlie…☆83Updated 2 years ago
- srcML Toolkit☆130Updated this week
- Sequence-to-Sequence Learning for End-to-End Program Repair (IEEE TSE 2019). Open-science repo. http://arxiv.org/pdf/1901.01808☆85Updated last year
- Implementation of 'A Convolutional Attention Network for Extreme Summarization of Source Code' in PyTorch using TorchText☆13Updated 6 years ago
- Repository for Deep API Learning (DeepAPI)☆54Updated 3 years ago
- A tool to convert nodes in an Abstract Syntax Tree into vector embeddings☆75Updated 2 years ago
- Implementation of "Automatic Source Code Summarization with Extended Tree-LSTM"☆36Updated 2 years ago
- Dataset and code corresponding to Associating Natural Language Comment and Source Code Entities (AAAI 2020)☆20Updated 4 years ago
- Learning to Spot and Refactor Inconsistent Method Names☆16Updated 2 years ago
- AutoenCODE is a Deep Learning infrastructure that allows to encode source code fragments into vector representations, which can be used t…☆60Updated 7 years ago
- Your library for dynamic language modeling☆67Updated 6 years ago
- ☆30Updated 6 years ago
- ☆26Updated 4 years ago
- ☆112Updated 2 years ago
- ☆66Updated 2 years ago
- Tensorflow implementation of Gated Graph Neural Network for Source Code Classification☆42Updated 4 years ago
- Set of tools to help working with "Big Code"☆43Updated 2 years ago
- Reproduce the results of Tree-based Convolutional Neural Network (TBCNN)☆39Updated 2 years ago
- Functional clone detection(currently maintained version)☆33Updated 2 years ago
- PyTorch's implementation of the code2seq model.☆62Updated 8 months ago
- A tool for identifying patch correctness in test-based program repair☆18Updated 5 years ago
- A javac plugin for extracting a feature graph for plugging in to machine learning models☆27Updated 4 years ago
- The source code and data for paper "Identifying Self-admitted Technical Debt in Open Source Projects Using Text Mining"☆9Updated 8 years ago
- Official repository for PraPR source code☆14Updated 3 years ago
- ☆49Updated 2 years ago
- Code and dataset for paper C4: Contrastive Cross-Language Code Clone Detection☆29Updated 2 years ago
- Deep learning code semantic similarity☆62Updated 5 years ago
- Description: We want to create a deep Neural Network that can automatically generate comments for code snippets passed to it. The motiva…☆44Updated 2 years ago
- A library for mining of path-based representations of code (and more)☆287Updated last year