cipher982 / llm-benchmarks
Benchmarking LLM Inference Speeds
☆12Updated this week
Alternatives and similar repositories for llm-benchmarks:
Users that are interested in llm-benchmarks are comparing it to the libraries listed below
- Simple (fast) transformer inference in PyTorch with torch.compile + lit-llama code☆10Updated last year
- ☆38Updated 10 months ago
- ☆31Updated last week
- Large language models (LLMs) made easy, EasyLM is a one stop solution for pre-training, finetuning, evaluating and serving LLMs in JAX/Fl…☆65Updated 6 months ago
- A TinyStories LM with SAEs and transcoders☆11Updated last month
- Experiments with representation engineering☆11Updated 11 months ago
- Experiments with generating opensource language model assistants☆97Updated last year
- Code and Data Repo for the CoNLL Paper -- Future Lens: Anticipating Subsequent Tokens from a Single Hidden State☆18Updated last year
- This repository contains code for cleaning your training data of benchmark data to help combat data snooping.☆25Updated last year
- An unofficial implementation of the Infini-gram model proposed by Liu et al. (2024)☆30Updated 8 months ago
- A collection of different ways to implement accessing and modifying internal model activations for LLMs☆11Updated 4 months ago
- Code to reproduce key results accompanying "SAEs (usually) Transfer Between Base and Chat Models"☆11Updated 7 months ago
- ☆31Updated 8 months ago
- A repository for transformer critique learning and generation☆88Updated last year
- Repository containing the SPIN experiments on the DIBT 10k ranked prompts☆24Updated 11 months ago
- Implementation of Influence Function approximations for differently sized ML models, using PyTorch☆15Updated last year
- Investigating the generalization behavior of LM probes trained to predict truth labels: (1) from one annotator to another, and (2) from e…☆26Updated 8 months ago
- Code for the ICLR 2024 paper "How to catch an AI liar: Lie detection in black-box LLMs by asking unrelated questions"☆64Updated 8 months ago
- A library for efficient patching and automatic circuit discovery.☆53Updated this week
- Open source replication of Anthropic's Crosscoders for Model Diffing☆39Updated 3 months ago
- A library for squeakily cleaning and filtering language datasets.☆46Updated last year
- Algebraic value editing in pretrained language models☆62Updated last year
- Mechanistic Interpretability for Transformer Models☆49Updated 2 years ago
- ☆12Updated last week
- Experiments for efforts to train a new and improved t5☆77Updated 10 months ago
- Embedding Recycling for Language models☆38Updated last year
- Official code repo for paper "Great Memory, Shallow Reasoning: Limits of kNN-LMs"☆21Updated 5 months ago
- ☆27Updated this week