hundredblocks / large-model-parallelismLinks
Functional local implementations of main model parallelism approaches
☆95Updated 2 years ago
Alternatives and similar repositories for large-model-parallelism
Users that are interested in large-model-parallelism are comparing it to the libraries listed below
Sorting:
- JAX implementation of the Llama 2 model☆219Updated last year
- Comprehensive analysis of difference in performance of QLora, Lora, and Full Finetunes.☆82Updated last year
- Inference code for LLaMA models in JAX☆118Updated last year
- ☆92Updated last year
- Resources from the EleutherAI Math Reading Group☆53Updated 3 months ago
- HomebrewNLP in JAX flavour for maintable TPU-Training☆50Updated last year
- Train very large language models in Jax.☆205Updated last year
- An interactive exploration of Transformer programming.☆264Updated last year
- Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*☆84Updated last year
- A puzzle to learn about prompting☆128Updated 2 years ago
- ☆20Updated last year
- Training and Inference Notebooks for the RedPajama (OpenLlama) models☆18Updated 2 years ago
- Automatically take good care of your preemptible TPUs☆36Updated 2 years ago
- ☆46Updated 2 years ago
- ☆78Updated 11 months ago
- Proof-of-concept of global switching between numpy/jax/pytorch in a library.☆18Updated last year
- Highly commented implementations of Transformers in PyTorch☆136Updated last year
- a pipeline for using api calls to agnostically convert unstructured data into structured training data☆30Updated 9 months ago
- Latent Diffusion Language Models☆68Updated last year
- ☆22Updated last year
- Various transformers for FSDP research☆37Updated 2 years ago
- Serialize JAX, Flax, Haiku, or Objax model params with 🤗`safetensors`☆45Updated last year
- A place to store reusable transformer components of my own creation or found on the interwebs☆56Updated last week
- Automatic gradient descent☆208Updated 2 years ago
- ☆20Updated last year
- ☆67Updated 2 years ago
- gzip Predicts Data-dependent Scaling Laws☆35Updated last year
- git extension for {collaborative, communal, continual} model development☆213Updated 7 months ago
- Experiments for efforts to train a new and improved t5☆77Updated last year
- ☆143Updated 2 years ago