mithril-security / tokenizers-wasm

Related projects ⓘ

Alternatives and complementary repositories for tokenizers-wasm

• FL33TW00D / embd
  GPU accelerated client-side embeddings for vector search, RAG etc.
  ☆63Updated 11 months ago
• iamlemec / bert.cpp
  GGML implementation of BERT model with Python bindings and quantization.
  ☆51Updated 9 months ago
• ashvardanian / usearch-binary
  Binary vector search example using Unum's USearch engine and pre-computed Wikipedia embeddings from Co:here and MixedBread
  ☆19Updated 7 months ago
• rahuldshetty / starcoder.js
  Web browser version of StarCoder.cpp
  ☆43Updated last year
• abetlen / program-constrained-language-model-sampling
  ☆34Updated last year
• octoml / triton-client-rs
  A client library in Rust for Nvidia Triton.
  ☆24Updated last year
• mixedbread-ai / batched
  ☆108Updated this week
• FL33TW00D / laserbeak
  Add local LLMs to your Web or Electron apps! Powered by Rust + WebGPU
  ☆102Updated last year
• Narsil / hf-chat
  ☆25Updated 10 months ago
• Vaibhavs10 / fast-llm.rs
  ☆136Updated 9 months ago
• argilla-io / distilabel-spin-dibt
  Repository containing the SPIN experiments on the DIBT 10k ranked prompts
  ☆23Updated 8 months ago
• dottxt-ai / outlines-core
  Structured generation in Rust
  ☆128Updated this week
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆45Updated last year
• raphaelsty / LeNLP
  NLP with Rust for Python 🦀🐍
  ☆59Updated 5 months ago
• Narsil / bloomserver
  ☆36Updated 2 years ago
• euclaise / SlimTrainer
  Full finetuning of large language models without large memory requirements
  ☆93Updated 10 months ago
• wozeparrot / tinyrwkv
  tinygrad port of the RWKV large language model.
  ☆43Updated 5 months ago
• UmerHA / triton_util
  Make triton easier
  ☆41Updated 5 months ago
• davanstrien / haiku-dpo
  Using open source LLMs to build synthetic datasets for direct preference optimization
  ☆40Updated 8 months ago
• eth-easl / deltazip
  Compression for Foundation Models
  ☆19Updated 3 weeks ago
• DeployQL / LintDB
  Vector Database with support for late interaction and token level embeddings.
  ☆54Updated last month
• teknium1 / stanford_alpaca-replit
  Modified Stanford-Alpaca Trainer for Training Replit's Code Model
  ☆40Updated last year
• whitphx / transformers.js.py
  ☆72Updated this week
• CarperAI / decontamination
  This repository contains code for cleaning your training data of benchmark data to help combat data snooping.
  ☆25Updated last year
• taylorai / onnx_embedding_models
  utilities for loading and running text embeddings with onnx
  ☆39Updated 3 months ago
• ConiferLabsWA / flan-ul2-alpaca
  ☆32Updated last year
• LLukas22 / llm-rs-python
  Unofficial python bindings for the rust llm library. 🐍❤️🦀
  ☆73Updated last year
• ggerganov / bert.cpp
  GGML implementation of BERT model with Python bindings and quantization.
  ☆26Updated 9 months ago