Convert source code into numerical tokens
☆67Jul 27, 2023Updated 3 years ago
Alternatives and similar repositories for tokenizer
Users that are interested in tokenizer are comparing it to the libraries listed below. We may earn a commission when you buy through links labeled 'Ad' on this page.
Sorting:
- C# Data Extraction for "Learning to Represent Edits"☆28Nov 3, 2018Updated 8 years ago
- Smelling smells using Deep Learning☆47Mar 2, 2021Updated 5 years ago
- Unit testing for SQL queries☆26Aug 16, 2024Updated 2 years ago
- This project collects methods that enhance the comparison between AMR graphs.☆11Jun 15, 2023Updated 3 years ago
- Artifacts and other data for "Code Vectors: Understanding Programs Through Embedded Abstraced Symbolic Traces"☆22Jun 5, 2020Updated 6 years ago
- Managed Kubernetes at scale on DigitalOcean • AdDigitalOcean Kubernetes includes the control plane, bandwidth allowance, container registry, automatic updates, and more for free.
- CD4Py: Code De-Duplication for Python☆24Dec 13, 2020Updated 5 years ago
- ☆20Nov 6, 2019Updated 7 years ago
- Babelfish Python client☆17Nov 6, 2019Updated 7 years ago
- Your library for dynamic language modeling☆70Oct 23, 2018Updated 8 years ago
- ☆15May 27, 2022Updated 4 years ago
- Improving Code Readability Classification using Convolutional Neural Networks☆11Apr 18, 2018Updated 8 years ago
- A multi-language tokenizer for extracting identifiers from source code.☆24Apr 27, 2022Updated 4 years ago
- A C++17 library for parser generation for LALR(1) languages☆13Jan 9, 2023Updated 3 years ago
- DeepCS: Deep Code Search☆285May 26, 2022Updated 4 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- fastText pretrained models for semantic representations of source code in Java, Python, PHP, C, C++ and C#.☆17Nov 11, 2020Updated 6 years ago
- Calculate the score of a repository based on best engineering practices.☆118Sep 27, 2020Updated 6 years ago
- Contains the code for our ICSE 2020 paper: Big Code != Big Vocabulary: Open-Vocabulary Language Models for Source Code and for its earlie…☆84Mar 24, 2023Updated 3 years ago
- ☆19Apr 15, 2024Updated 2 years ago
- FaCoY Code-to-Code Search Engine☆34Jan 18, 2019Updated 7 years ago
- A Source Code Tokenizer☆13Oct 30, 2024Updated 2 years ago
- Assessing Source Code Semantic Similarity with Unsupervised Learning☆39Feb 27, 2018Updated 8 years ago
- ProgQuery is a system to extract useful syntactic and semantic information from source code programs and store it in a graph database for…☆17Jan 22, 2025Updated last year
- Discovery of Rhyme Schemes in Poetry☆17Nov 22, 2011Updated 15 years ago
- Managed Database hosting by DigitalOcean • AdPostgreSQL, MySQL, MongoDB, Kafka, Valkey, and OpenSearch available. Automatically scale up storage and focus on building your apps.
- TensorFlow code for the neural network presented in the paper: "code2vec: Learning Distributed Representations of Code"☆1,149Sep 20, 2023Updated 3 years ago
- Maven plugin to create HTML report to show dependecies in DSM view.☆14Nov 2, 2026Updated 2 weeks ago
- OCaml library to transform an Llvm control flow graph in an SMT formula.☆13Apr 20, 2018Updated 8 years ago
- differential☆12Oct 30, 2019Updated 7 years ago
- Accmut is a framework for acclerating mutation testing, which is based on LLVM-IR.☆10Jan 25, 2018Updated 8 years ago
- ☆17Dec 9, 2022Updated 3 years ago
- A dynamic method for detecting faults in incremental and parallel builds.☆18Jul 27, 2022Updated 4 years ago
- ☆22Jun 3, 2019Updated 7 years ago
- ☆15Oct 17, 2023Updated 3 years ago
- Deploy to Railway using AI coding agents - Free Credits Offer • AdUse Claude Code, Codex, OpenCode, and more. Autonomous software development now has the infrastructure to match with Railway.
- A tool to analyze and compare software repositories:☆26Jul 6, 2023Updated 3 years ago
- An Android application for logging Crossfit workouts☆11Jun 1, 2011Updated 15 years ago
- ☆24Jun 17, 2021Updated 5 years ago
- Manipulate C-family ASTs with Clang☆70Oct 22, 2018Updated 8 years ago
- ☆11Nov 11, 2025Updated last year
- source{d} datasets ("big code") for source code analysis and machine learning on source code☆347Nov 27, 2019Updated 6 years ago
- 2019 年开源年度报告☆11Jan 7, 2020Updated 6 years ago
Are these results useful?