roudimit / whisper-flamingo
[Interspeech 2024] Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation
☆127Updated this week
Alternatives and similar repositories for whisper-flamingo:
Users that are interested in whisper-flamingo are comparing it to the libraries listed below
- AudioBench: A Universal Benchmark for Audio Large Language Models☆124Updated 3 weeks ago
- Code for the paper: GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities☆107Updated 2 months ago
- PyTorch implementation of Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities.☆225Updated 4 months ago
- [Interspeech 2023] Intelligible Lip-to-Speech Synthesis with Speech Units☆27Updated 3 months ago
- Codec for paper: LLaSA: Scaling Train-time and Inference-time Compute for LLaMA-based Speech Synthesis☆163Updated this week
- Official Implementation of the work "Audio Mamba: Bidirectional State Space Model for Audio Representation Learning"☆127Updated 2 months ago
- Official Implementation of EnCLAP (ICASSP 2024)☆90Updated 8 months ago
- VoiceLDM: Text-to-Speech with Environmental Context☆169Updated 6 months ago
- VoiceBench: Benchmarking LLM-Based Voice Assistants☆108Updated last week
- This is an evolving repo for the paper "Towards Controllable Speech Synthesis in the Era of Large Language Models: A Survey".☆118Updated last month
- SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformer.☆76Updated last month
- FACodec: Speech Codec with Attribute Factorization used for NaturalSpeech 3☆186Updated 9 months ago
- A 6-million Audio-Caption Paired Dataset Built with a LLMs and ALMs-based Automatic Pipeline☆116Updated 2 months ago
- Official repository for the "Powerset multi-class cross entropy loss for neural speaker diarization" paper published in Interspeech 2023.☆79Updated last year
- ☆63Updated 5 months ago
- A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models (ICASSP 2024)☆51Updated 9 months ago
- Official implementation of the paper "BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec"☆130Updated 4 months ago
- AAAI 2025: Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model☆148Updated last month
- ☆149Updated 2 months ago
- [ICASSP 2024] This is the official code for "VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching"☆328Updated 5 months ago
- SSR-Speech: Towards Stable, Safe and Robust Zero-shot Speech Editing and Synthesis☆120Updated last month
- This repository contains the code and data for the paper EmoKnob: Enhance Voice Cloning with Fine-Grained Emotion Control by Haozhe Chen,…☆66Updated 4 months ago
- Audio Captioning datasets for PyTorch.☆113Updated 3 months ago
- Update ASR paper everyday☆127Updated this week
- Official implementation of USR (NeurIPS 2024)☆28Updated last month
- Companion repo for the paper "PixIT: Joint Training of Speaker Diarization and Speech Separation from Real-world Multi-speaker Recordings…☆78Updated last month
- Real-time Speech-Text Foundation Model Toolkit (wip)☆128Updated 4 months ago
- [INTERSPEECH 2022] This dataset is designed for multi-modal speaker diarization and lip-speech synchronization in the wild.☆47Updated last year
- UTokyo-SaruLab MOS Prediction System☆144Updated 2 months ago
- Models and code for RepCodec: A Speech Representation Codec for Speech Tokenization☆168Updated 7 months ago