link
Browse files
README.md
CHANGED
@@ -23,14 +23,21 @@ tags:
|
|
23 |
|
24 |
- [SHIFT TTS tool](https://github.com/audeering/shift)
|
25 |
- Analysis of emotionality [#1](https://huggingface.co/dkounadis/artificial-styletts2/discussions/2)
|
26 |
-
- Soundscape `e.g. trees, water
|
27 |
- `landscape2soundscape.py` shows how to overlay TTS & sound to image and create video
|
28 |
|
29 |
## Available Voices
|
30 |
|
31 |
<a href="https://audeering.github.io/shift/">Listen to available voices!</a>
|
32 |
|
33 |
-
## API
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
34 |
|
35 |
```
|
36 |
virtualenv --python=python3 ~/.envs/.my_env
|
@@ -39,7 +46,7 @@ cd shift/
|
|
39 |
pip install -r requirements.txt
|
40 |
```
|
41 |
|
42 |
-
|
43 |
|
44 |
```
|
45 |
CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=./hf_home CUDA_VISIBLE_DEVICES=2 python api.py
|
@@ -103,7 +110,6 @@ python tts.py --text sample.txt --image assets/image_from_T31.jpg
|
|
103 |
```
|
104 |
|
105 |
## Landscape 2 Soundscape
|
106 |
-
- YouTube Videos For SHIFT demo / Collaboration with [SMB](https://www.smb.museum/home/)
|
107 |
|
108 |
|
109 |
|
@@ -114,9 +120,15 @@ python tts.py --text sample.txt --image assets/image_from_T31.jpg
|
|
114 |
python landscape2soundscape.py
|
115 |
```
|
116 |
|
|
|
|
|
|
|
|
|
117 |
[![01](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____01_Schick_AII840_001.jpg)](https://youtu.be/SSi3gUO4GtY)
|
118 |
|
119 |
-
[![02](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____02_Constable_AI555_001.jpg)(https://youtu.be/2YjxAPkdXIc)
|
|
|
|
|
120 |
|
121 |
|
122 |
# Live Demo - Paplay
|
|
|
23 |
|
24 |
- [SHIFT TTS tool](https://github.com/audeering/shift)
|
25 |
- Analysis of emotionality [#1](https://huggingface.co/dkounadis/artificial-styletts2/discussions/2)
|
26 |
+
- Soundscape `e.g. trees, water` via [AudioGen](https://huggingface.co/dkounadis/artificial-styletts2/discussions/3)
|
27 |
- `landscape2soundscape.py` shows how to overlay TTS & sound to image and create video
|
28 |
|
29 |
## Available Voices
|
30 |
|
31 |
<a href="https://audeering.github.io/shift/">Listen to available voices!</a>
|
32 |
|
33 |
+
## Flask API
|
34 |
+
|
35 |
+
Clone this repo
|
36 |
+
|
37 |
+
```
|
38 |
+
git clone https://huggingface.co/dkounadis/artificial-styletts2
|
39 |
+
```
|
40 |
+
Build env
|
41 |
|
42 |
```
|
43 |
virtualenv --python=python3 ~/.envs/.my_env
|
|
|
46 |
pip install -r requirements.txt
|
47 |
```
|
48 |
|
49 |
+
Start API
|
50 |
|
51 |
```
|
52 |
CUDA_DEVICE_ORDER=PCI_BUS_ID HF_HOME=./hf_home CUDA_VISIBLE_DEVICES=2 python api.py
|
|
|
110 |
```
|
111 |
|
112 |
## Landscape 2 Soundscape
|
|
|
113 |
|
114 |
|
115 |
|
|
|
120 |
python landscape2soundscape.py
|
121 |
```
|
122 |
|
123 |
+
For SHIFT demo / Collaboration with [SMB](https://www.smb.museum/home/)
|
124 |
+
- YouTube Videos
|
125 |
+
|
126 |
+
|
127 |
[![01](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____01_Schick_AII840_001.jpg)](https://youtu.be/SSi3gUO4GtY)
|
128 |
|
129 |
+
[![02](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____02_Constable_AI555_001.jpg)](https://youtu.be/2YjxAPkdXIc)
|
130 |
+
|
131 |
+
[![10](uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____10_Boecklin_967648_NG2-80_001_rsz.jpg)](https://www.youtube.com/watch?v=Y8QyYUgLaCg)
|
132 |
|
133 |
|
134 |
# Live Demo - Paplay
|
landscape2soundscape.py
CHANGED
@@ -61,8 +61,8 @@ DESCRIPTIONS = [
|
|
61 |
[
|
62 |
'03_Schinkel_WS200-002.jpg',
|
63 |
'03_Schinkel_WS200-002.txt',
|
64 |
-
'Arriving at the
|
65 |
-
'
|
66 |
'en_US/hifi-tts_low#6097',
|
67 |
],
|
68 |
#
|
@@ -85,7 +85,7 @@ DESCRIPTIONS = [
|
|
85 |
[
|
86 |
'06_Menzel_AI900_001.jpg',
|
87 |
'06_Menzel_AI900_001.txt',
|
88 |
-
'Olive
|
89 |
'A. Menzel, Bauplatz mit Weiden, 1846',
|
90 |
'en_US/cmu-arctic_low#aup',
|
91 |
],
|
@@ -146,7 +146,7 @@ SILENT_VIDEO = '_silent_video.mp4'
|
|
146 |
# SILENT CLIP
|
147 |
|
148 |
|
149 |
-
for img, text, scene, title, voice in DESCRIPTIONS[:20]:
|
150 |
|
151 |
|
152 |
|
|
|
61 |
[
|
62 |
'03_Schinkel_WS200-002.jpg',
|
63 |
'03_Schinkel_WS200-002.txt',
|
64 |
+
'Arriving at the valley on galloping horses',
|
65 |
+
'KF. Schinkel Gotische Kirche Auf Einem Felsen 1815',
|
66 |
'en_US/hifi-tts_low#6097',
|
67 |
],
|
68 |
#
|
|
|
85 |
[
|
86 |
'06_Menzel_AI900_001.jpg',
|
87 |
'06_Menzel_AI900_001.txt',
|
88 |
+
'Olive scenery roman Colloseum',
|
89 |
'A. Menzel, Bauplatz mit Weiden, 1846',
|
90 |
'en_US/cmu-arctic_low#aup',
|
91 |
],
|
|
|
146 |
# SILENT CLIP
|
147 |
|
148 |
|
149 |
+
for img, text, scene, title, voice in DESCRIPTIONS[2:20]:
|
150 |
|
151 |
|
152 |
|
uc_spk_Landscape2Soundscape_Masterpieces_pics/04_Friedrich_FV317_001.txt
CHANGED
@@ -1,9 +1,9 @@
|
|
1 |
-
In 1825, the
|
2 |
|
3 |
-
Caspar David Friedrich had never actually visited the Alps, yet he created several paintings of these mountains. For his iconic work
|
4 |
|
5 |
-
Friedrich had already exhibited a large Alpine landscape in Dresden in 1824, a view of Mont Blanc
|
6 |
|
7 |
-
In Friedrich’s vision, the glacier-clad Watzmann, radiant in bright light, embodies the vastness
|
8 |
|
9 |
-
Eberhard Hanfstaengl, who acquired
|
|
|
1 |
+
In 1825, the Literary Conversation newspaper wrote about the Dresden Academy Exhibition, mentioning a beautiful large landscape by Professor Friedrich: a solitary mountain region... splendidly depicted with varying tones across the upper mountain ranges. In the foreground, grasses and small trees cling to the towering basalt rocks. Higher up, mist curls around the desolate ridges, and at the top, the shimmering rock faces, crowned with eternal snow, glow in untarnished clarity. The solitude here is eerie; one longs to see at least an eagle or chamois—yet in vain, no life dwells here except for the air and light. Every pulse of emotion seems to falter at this height.
|
2 |
|
3 |
+
Caspar David Friedrich had never actually visited the Alps, yet he created several paintings of these mountains. For his iconic work The Watzmann, he used a variety of sources. A key reference was a watercolor study of the mountain’s summit captured by his pupil Johann August Heinrich, now in the National Museum of Oslo. Friedrich also drew his own sketches from travels in the Harz Mountain range. The rocky formation in the foreground was based on his drawings of the Trudenstein at Hohnekopf, near Brocken summit in June 28 1811.
|
4 |
|
5 |
+
Friedrich had already exhibited a large Alpine landscape in Dresden in 1824, a view of Mont Blanc, formerly at the National Gallery in Berlin, lost during the war. As The Watzmann show he borrowed styles from another artist’s drawing, of a sketch by Carl Gustav Carus.
|
6 |
|
7 |
+
In Friedrich’s vision, the glacier-clad Watzmann, radiant in bright light, embodies the vastness of nature, serving as a distant symbol of divine majesty. Gentle mists hover over the ridges of the Archenkopf and Grünstein mountains below. Friedrich renders the shifting tones between green and violet. In this isolated alpine landscape, no living creatures are present. The pyramid-like composition culminates in the brilliant white of eternal ice—a concept Friedrich had explored two years earlier in his famous painting The Sea of Ice now in Hamburg Kunsthalle.
|
8 |
|
9 |
+
Eberhard Hanfstaengl, who acquired The Watzmann for the Berlin National Gallery, praised Friedrich’s "artistic and poetic power... in which he blends his own and others’ depictions of nature into a heightened form, transcending mere landscape painting and becoming the epitome of mountain portrayal."
|
uc_spk_Landscape2Soundscape_Masterpieces_pics/thumb____10_Boecklin_967648_NG2-80_001_rsz.jpg
ADDED