roudimit / whisper-flamingo
[Interspeech 2024] Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation
☆145Updated last month
Alternatives and similar repositories for whisper-flamingo:
Users that are interested in whisper-flamingo are comparing it to the libraries listed below
- AudioBench: A Universal Benchmark for Audio Large Language Models☆161Updated this week
- Code for the paper: GAMA: A Large Audio-Language Model with Advanced Audio Understanding and Complex Reasoning Abilities☆114Updated 3 months ago
- Codec for paper: LLaSA: Scaling Train-time and Inference-time Compute for LLaMA-based Speech Synthesis☆227Updated last week
- VoiceBench: Benchmarking LLM-Based Voice Assistants☆141Updated last week
- VoiceLDM: Text-to-Speech with Environmental Context☆172Updated 7 months ago
- [Interspeech 2023] Intelligible Lip-to-Speech Synthesis with Speech Units☆29Updated 4 months ago
- FACodec: Speech Codec with Attribute Factorization used for NaturalSpeech 3☆193Updated 11 months ago
- This is an evolving repo for the paper "Towards Controllable Speech Synthesis in the Era of Large Language Models: A Survey".☆122Updated 2 months ago
- ☆64Updated 6 months ago
- Ultra-low bitrate neural audio codec (0.31~1.40 kbps) with a better semantic in the latent space.☆193Updated last week
- UTokyo-SaruLab MOS Prediction System☆155Updated 3 weeks ago
- Official implementation of the paper "BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec"☆137Updated 6 months ago
- [ICASSP 2024] This is the official code for "VoiceFlow: Efficient Text-to-Speech with Rectified Flow Matching"☆336Updated 6 months ago
- Official repository for the "Powerset multi-class cross entropy loss for neural speaker diarization" paper published in Interspeech 2023.☆82Updated last year
- AAAI 2025: Codec Does Matter: Exploring the Semantic Shortcoming of Codec for Audio Language Model☆182Updated 2 months ago
- [INTERSPEECH 2024] EmoBox: Multilingual Multi-corpus Speech Emotion Recognition Toolkit and Benchmark☆210Updated 9 months ago
- This repository contains the code and data for the paper EmoKnob: Enhance Voice Cloning with Fine-Grained Emotion Control by Haozhe Chen,…☆67Updated 5 months ago
- Official Implementation of EnCLAP (ICASSP 2024)☆90Updated 9 months ago
- Unified Speech Language Model for paper "SpeechTokenizer: Unified Speech Tokenizer for Speech Large Language Models"(ICLR 2024)☆138Updated last year
- SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformer.☆82Updated 2 months ago
- ACM MM 2023 CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model☆205Updated 10 months ago
- The official repository of SpeechCraft dataset, a large-scale expressive bilingual speech dataset with natural language descriptions.☆111Updated 2 months ago
- ☆140Updated last year
- The official Pytorch implementation of "Frame-wise streaming end-to-end speaker diarization with non-autoregressive self-attention-based …☆120Updated 2 weeks ago
- The official implementation of EmoSphere++☆74Updated 2 months ago
- ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations☆147Updated last year
- Update ASR paper everyday☆168Updated this week
- [INTERSPEECH 2022] This dataset is designed for multi-modal speaker diarization and lip-speech synchronization in the wild.☆49Updated last year
- ACM MM 2024 FlashSpeech: Efficient Zero-Shot Speech Synthesis☆129Updated 6 months ago
- a text-conditional diffusion probabilistic model capable of generating high fidelity audio.☆156Updated 9 months ago