roudimit / whisper-flamingo
[Interspeech 2024] Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation
☆151Updated 2 months ago
Alternatives and similar repositories for whisper-flamingo:
Users that are interested in whisper-flamingo are comparing it to the libraries listed below
- AudioBench: A Universal Benchmark for Audio Large Language Models☆201Updated last month
- VoiceBench: Benchmarking LLM-Based Voice Assistants☆184Updated last week
- Code for the paper: GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities☆124Updated 5 months ago
- VoiceLDM: Text-to-Speech with Environmental Context☆175Updated 8 months ago
- Promting Whisper for Audio-Visual Speech Recognition, Code-Switched Speech Recognition, and Zero-Shot Speech Translation☆143Updated last year
- FACodec: Speech Codec with Attribute Factorization used for NaturalSpeech 3☆196Updated last year
- ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations☆156Updated last year
- Codec for paper: LLaSA: Scaling Train-time and Inference-time Compute for LLaMA-based Speech Synthesis☆262Updated last month
- [Interspeech 2023] Intelligible Lip-to-Speech Synthesis with Speech Units☆34Updated 6 months ago
- Official Implementation of EnCLAP (ICASSP 2024)☆91Updated 11 months ago
- An Audio Language model for Audio Tasks☆304Updated last year
- Official implementation of the paper "BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec"☆158Updated 7 months ago
- Ultra-low bitrate neural audio codec (0.31~1.40 kbps) with a better semantic in the latent space.☆198Updated last month
- Official repository for the "Powerset multi-class cross entropy loss for neural speaker diarization" paper published in Interspeech 2023.☆83Updated last year
- This is an evolving repo for the paper "Towards Controllable Speech Synthesis in the Era of Large Language Models: A Survey".☆138Updated 2 weeks ago
- [ICASSP 2024] This is the official code for "VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching"☆343Updated 8 months ago
- ☆65Updated 7 months ago
- Audio Codec Speech processing Universal PERformance Benchmark☆252Updated 3 weeks ago
- It's a repository for implementations of neural speech editing algorithms.☆196Updated last year
- Versatile Evaluation of Speech and Audio☆239Updated last week
- Models and code for RepCodec: A Speech Representation Codec for Speech Tokenization☆175Updated 9 months ago
- AAAI 2025: Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model☆205Updated 2 weeks ago
- Companion repo for the paper "PixIT: Joint Training of Speaker Diarization and Speech Separation from Real-world Multi-speaker Recordings…☆87Updated 3 months ago
- Training code for FAcodec presented in NaturalSpeech3☆204Updated 8 months ago
- Libriheavy: a 50,000 hours ASR corpus with punctuation casing and context☆194Updated 7 months ago
- Official repository of the paper "MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization".☆180Updated 3 months ago
- Real-time Speech-Text Foundation Model Toolkit (wip)☆226Updated last month
- Scripts for computing the Intelligibility and CLVP scores for evaluating TTS models☆154Updated last year
- Unified Speech Language Model for paper "SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models"(ICLR 2024)☆142Updated last year
- [INTERSPEECH 2022] This dataset is designed for multi-modal speaker diarization and lip-speech synchronization in the wild.☆52Updated last year