Hugging Face's logo

dkounadis
/
artificial-styletts2

Text-to-Audio
Audiocraft
English
audiogen
styletts2
shift
audeering
sound
audio-generation
text-to-speech
mimic3
Model card Files Community
4
artificial-styletts2 / README.md
Dionyssos's picture
Dionyssos
fx lnk
b073bdf
preview code
|
raw
history blame contribute delete
3.6 kB
metadata
license: mit
language:
  - en
pipeline_tag: text-to-audio
tags:
  - audiocraft
  - audiogen
  - styletts2
  - shift
  - audeering
  - sound
  - audio-generation
  - text-to-speech
  - mimic3

Affective TTS / Soundscape

This repo is an expansion of the SHIFT TTS tool with foreign langs and audio sound generation via AudioGen.

  • Analysis of emotions of TTS #1
  • landscape2soundscape.py generates soundscape, i.e. trees, water, .. & overlays TTS & creates video from image.

Available Voices

Listen to available voices!

Flask API

Create virtualenv

Clone

git clone https://huggingface.co/dkounadis/artificial-styletts2

Install

virtualenv --python=python3 ~/.envs/.my_env
source ~/.envs/.my_env/bin/activate
cd artificial-styletts2/
pip install -r requirements.txt

Flask API

CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=./hf_home CUDA_VISIBLE_DEVICES=2 python api.py

Landscape 2 Soundscape

The following needs api.py to be already running on a tmux session. 

# TTS & soundscape - overlay to .mp4
python landscape2soundscape.py

For SHIFT demo / Collaboration with SMB

  • YouTube Videos

01

02

03

05

06

07

08

10

11

12

SoundScape Live (iterative) Demo - Paplay

Special Flask API for playing sounds live

CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=/data/dkounadis/.hf7/ CUDA_VISIBLE_DEVICES=4 python live_api.py

Client - Describe any sound with words and it will be played back to you.

python live_demo.py  # will ask text input & play soundscape

SoundScape (basic) Demo 

CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=/data/dkounadis/.hf7/ CUDA_VISIBLE_DEVICES=4 python demo.py

Audiobook

Convert .docx to audio .wav & .mp4. Via multiple voices. Listen to it in YouTube male voice / female voice

# uses Flask api.py
# download shift/assets/INCLUSION_IN_MUSEUMS_audiobook.docx
python audiobook.py