google-research-datasets / indic-gen-bench

Related projects: ⓘ

• AI4Bharat / IndicInstruct
  Code repository for "Introducing Airavata: Hindi Instruction-tuned LLM"
  ☆52Updated last month
• AI4Bharat / Shoonya
  Shoonya - Platform to Annotate and label data at scale.
  ☆48Updated 2 weeks ago
• google-research-datasets / Hinglish-TOP-Dataset
  Consists of the largest (10K) human annotated code-switched semantic parsing dataset & 170K generated utterance using the CST5 augmentati…
  ☆33Updated last year
• AI4Bharat / FBI
  FBI: Finding Blindspots in LLM Evaluations with Interpretable Checklists
  ☆18Updated last month
• pacman100 / openhathi_instruct
  This repository contains the code for dataset curation and finetuning of instruct variant of the Bilingual OpenHathi model. The resultin…
  ☆23Updated 8 months ago
• adithya-s-k / indic_eval
  A lightweight evaluation suite tailored specifically for assessing Indic LLMs across a diverse range of tasks, aiding in performance asse…
  ☆31Updated 3 months ago
• AI4Bharat / webcorpus
  Generate large textual corpora for almost any language by crawling the web
  ☆11Updated 7 months ago
• huggingface / olm-training
  Repo for training MLMs, CLMs, or T5-type models on the OLM pretraining data, but it should work with any hugging face text dataset.
  ☆91Updated last year
• google-research / metricx
  ☆56Updated 7 months ago
• nbroad1881 / strideformer
  Using short models to classify long texts
  ☆20Updated last year
• IlyasMoutawwakil / py-txi
  A Python wrapper around HuggingFace's TGI (text-generation-inference) and TEI (text-embedding-inference) servers.
  ☆31Updated 2 weeks ago
• AI4Bharat / IndicBERT
  Pretraining, fine-tuning and evaluation scripts for IndicBERT-v2 and IndicXTREME
  ☆67Updated 2 weeks ago
• nateraw / huggingface-datasets-converter
  Scripts to convert datasets from various sources to Hugging Face Datasets.
  ☆57Updated last year
• Rallio67 / language-model-agents
  Experiments with generating opensource language model assistants
  ☆97Updated last year
• bminixhofer / zett
  Code for Zero-Shot Tokenizer Transfer
  ☆109Updated 2 months ago
• EleutherAI / semantic-memorization
  ☆44Updated 2 months ago
• VarunGumma / IndicTransToolkit
  A simple, consistent and extendable toolkit for IndicTrans2
  ☆16Updated 3 weeks ago
• ChrisHayduk / QLoRA-for-MLM
  QLoRA for Masked Language Modeling
  ☆20Updated last year
• ShareChatAI / MACD
  ☆16Updated 6 months ago
• Upaya07 / NeurIPS-llm-efficiency-challenge
  Code for NeurIPS LLM Efficiency Challenge
  ☆52Updated 5 months ago
• AI4Bharat / IndicLID
  Language Identification for Indian languages
  ☆10Updated 7 months ago
• allenai / bff
  ☆38Updated 5 months ago
• AI4Bharat / IndicLLMSuite
  A blueprint for creating Pretraining and Fine-Tuning datasets for Indic languages
  ☆88Updated last month
• akjindal53244 / Arithmo
  Small and Efficient Mathematical Reasoning LLMs
  ☆69Updated 7 months ago
• TeluguLLMLabs / Indic-gemma-7b-Navarasa
  Repository for fine-tuning gemma models using unsloth for indic languages
  ☆80Updated 6 months ago
• Pleias / Various-Finetuning
  Set of scripts to finetune LLMs
  ☆36Updated 5 months ago
• illuin-tech / vidore-benchmark
  Vision Document Retrieval (ViDoRe): Benchmark. Evaluation code for the ColPali paper.
  ☆101Updated last week
• EleutherAI / lm-eval2
  ☆13Updated this week
• bloomberg / minilmv2.bb
  Our open source implementation of MiniLMv2 (https://aclanthology.org/2021.findings-acl.188)
  ☆59Updated last year
• tcapelle / llm_recipes
  A set of scripts and notebooks on LLM finetunning and dataset creation
  ☆89Updated this week