Alternatives and similar repositories for nucleus:

Users that are interested in nucleus are comparing it to the libraries listed below

• trisongz / pylines
  Simplifying parsing of large jsonline files in NLP Workflows
  ☆12Updated 3 years ago
• CarperAI / squeakily
  A library for squeakily cleaning and filtering language datasets.
  ☆46Updated last year
• bentoml / sentence-embedding-bento
  Sentence Embedding as a Service
  ☆15Updated last year
• mixedbread-ai / binary-embeddings
  Showcase how mxbai-embed-large-v1 can be used to produce binary embedding. Binary embeddings enabled 32x storage savings and 40x faster r…
  ☆16Updated last year
• yiyixuxu / n-grammer-flax
  Implementation of N-Grammer in Flax
  ☆17Updated 2 years ago
• BlackHC / llmtracer
  Trace LLM calls (and others) and visualize them in WandB, as interactive SVG or using a streaming local webapp
  ☆14Updated last month
• chainyo / tensorshare
  🤝 Trade any tensors over the network
  ☆30Updated last year
• modal-labs / ci-on-modal
  A sample pattern for running CI tests on Modal
  ☆17Updated this week
• axiomic-ai / axiomic
  Creating Generative AI Apps which work
  ☆17Updated 9 months ago
• Systemcluster / kitoken
  Fast and versatile tokenizer for language models, compatible with SentencePiece, Tokenizers, Tiktoken and more. Supports BPE, Unigram and…
  ☆19Updated 3 weeks ago
• stephantul / unitoken
  Tokenization across languages. Useful as preprocessing for subword tokenization.
  ☆22Updated 2 years ago
• philschmid / optimum-static-quantization
  ☆28Updated last year
• criteo / fromconfig
  A library to instantiate any Python object from configuration files.
  ☆24Updated 2 years ago
• Narsil / bloomserver
  ☆39Updated 2 years ago
• Zyphra / zcookbook
  Training hybrid models for dummies.
  ☆20Updated 2 months ago
• replicate / cog-vllm
  Run LLMs on Replicate with vLLM
  ☆17Updated 6 months ago
• tmbdev-archive / tarproc
  Utilities for sequential processing of tar files.
  ☆24Updated 3 years ago
• alvarobartt / vertex-ai-huggingface-inference-toolkit
  🤗 HuggingFace Inference Toolkit for Google Cloud Vertex AI (similar to SageMaker's Inference Toolkit, but for Vertex AI and unofficial)
  ☆17Updated last year
• hamelsmu / ft-drift
  Check for data drift between two OpenAI multi-turn chat jsonl files.
  ☆37Updated last year
• nbroad1881 / strideformer
  Using short models to classify long texts
  ☆21Updated 2 years ago
• Alignment-Lab-AI / datagen
  a pipeline for using api calls to agnostically convert unstructured data into structured training data
  ☆30Updated 6 months ago
• maxdotio / mighty-batch
  Highly concurrent and fast content processing for Mighty Inference Server
  ☆10Updated 2 years ago
• jquesnelle / ctranslate2-rs
  Rust bindings for CTranslate2
  ☆14Updated last year
• sileod / language-model-recommendation
  Resources accompanying the "Zero-Shot Recommendation as Language Modeling" paper (ECIR2022)
  ☆13Updated last year
• fastai / fastrelease
  DEPRECATED--all functionality moved to nbdev
  ☆15Updated 2 years ago
• withmartian / leaderboard-backend
  Open sourced backend for Martian's LLM Inference Provider Leaderboard
  ☆17Updated 7 months ago
• allenai / cached_path
  A file utility for accessing both local and remote files through a unified interface.
  ☆40Updated last month
• amzn / extremely-efficient-query-encoder
  efficient query encoding for dense retrieval
  ☆11Updated 8 months ago
• cloudera / cmlextensions
  Added functionality to the cml python package
  ☆14Updated 6 months ago
• crypdick / timm-lr-scheduler-explorer
  A dashboard for exploring timm learning rate schedulers
  ☆19Updated 4 months ago