3v324v23 commited on
Commit
4326fd1
1 Parent(s): 71d658a
Files changed (1) hide show
  1. app.py +24 -24
app.py CHANGED
@@ -43,7 +43,6 @@ def BeatSwap(audiofile, pattern: str = 'test', scale:float = 1, shift:float = 0,
43
  y=min(len(image), len(image[0]), 2048)
44
  y=max(y, 2048)
45
  image = np.rot90(np.clip(cv2.resize(image, (y,y), interpolation=cv2.INTER_NEAREST), -1, 1))
46
- print(image)
47
  #print(image)
48
  except Exception as e:
49
  print(f'Image generation failed: {e}')
@@ -55,7 +54,7 @@ def BeatSwap(audiofile, pattern: str = 'test', scale:float = 1, shift:float = 0,
55
  return ((song.sr, song.audio), image)
56
 
57
  audiofile=Audio(source='upload', type='filepath')
58
- patternbox = Textbox(label="Pattern:", placeholder="1, 3, 2, 4!", value="1, 2!", lines=1)
59
  scalebox = Textbox(value=1, label="Beatmap scale. At 2, every two beat positions will be merged, at 0.5 - a beat position added between every two existing ones.", placeholder=1, lines=1)
60
  shiftbox = Textbox(value=0, label="Beatmap shift, in beats (applies before scaling):", placeholder=0, lines=1)
61
  cachebox = Checkbox(value=True, label="Enable caching generated beatmaps for faster loading. Saves a file with beat positions and loads it when you open same audio again.")
@@ -65,40 +64,35 @@ gr.Interface (fn=BeatSwap,inputs=[audiofile,patternbox,scalebox,shiftbox, cacheb
65
  title = "Stunlocked's Beat Manipulator"
66
  ,description = """Remix music using AI-powered beat detection and advanced beat swapping. Make \"every other beat is missing\" remixes, or completely change beat of the song.
67
 
68
- Github - https://github.com/stunlocked1/BeatManipulator.
69
 
70
  Colab version - https://colab.research.google.com/drive/1gEsZCCh2zMKqLmaGH5BPPLrImhEGVhv3?usp=sharing"""
71
- ,article="""# <h1><p style='text-align: center'><a href='https://github.com/stunlocked1/BeatManipulator' target='_blank'>Github</a></p></h1>
72
-
73
- # Basic usage
74
-
75
  Upload your audio, enter the beat swapping pattern, change scale and shift if needed, and run it.
76
 
77
- # pattern syntax
78
- The pattern syntax is quite powerful and you can do a whole bunch of stuff with it.
79
-
80
- Basic syntax is - `1, 3, 2, 4` means every 4 beats, swap 2nd and 3rd beats, but you can do much more, like applying audio effects, shuffling beats, slicing them, mixing two songs, adding samples, sidechain.
81
 
82
  You can use spaces freely in patterns for formatting. Most other symbols have some use though. Here is the full syntax:
83
- #### beats
84
  * `1` - 1st beat;
85
  * `1>0.5` - first half of first beat
86
  * `1<0.5` - second half of first beat
87
  * `0:0.5` - range of beats, this also means first half of first beat, but with this you can do complex stuff like `1.25:1.5`. However this one is a bit more confusing because indexing starts from 0, so 1:2 is second beat, not first.
88
  * Also sometimes it is more convenient to use smaller `scale`, like 0.5 or 0.25, instead of slicing beats.
89
- #### basic patterns
90
  - `1, 3, 2, 4` - 3rd and 2nd beats will be swapped every 4 beats. Happens every 4 beats because 4 is the biggest number in the pattern.
91
  - `1, 2, 3, 4!` - every 4 beats, it will remove the 4th beat. `!` allows you to skip a beat but it still counts for pattern size.
92
- - Specifying pattern length: `pattern = '1, 2, 3', length = 4` is another way to remove 4th beat every 4 beats.
93
  - `1, 4` skips 2nd and 3rd beat
94
  - `1; 2` plays 1st and second beat at the same time.
95
- #### joining operators
96
  `,` and `;` are beat joining operators. Here are all available joiners:
97
  - `,` puts the beat next to previous beat.
98
  - `;` puts the beat on top of previous beat. Normalizes the volume to avoid clipping. If previous beat is shorter, your beat will be shortened to match it.
99
  - `^` multiplies previous beat by your beat. This can be used for fake sidechain.
100
  - `$` adds the beat on top of previous beat + sidechains previous beat by your beat (I haven't tested this one)
101
- #### effects
102
  beats can be followed by effects. For example `1s0.75` means take first beat and play it at 0.75x speed. Here are all available effects:
103
  - `s` - speed. `1s2` means first beat will be played at 2x speed.
104
  - `r` - reverse. `1r` means first beat will have reversed audio.
@@ -108,37 +102,43 @@ beats can be followed by effects. For example `1s0.75` means take first beat and
108
  - `g` - gradient, sounds like highpass. `1g1` is the recommended value
109
  - `c` - channel. If not followed by number, swaps channels. If followed by 0, plays only left channel. If 1, only right channel
110
  - mixing effects - `1s2rd8` - take first beat, play at 2x speed, reversed, and downsampled.
111
- #### math
112
  mathematical expressions with `+`, `-`, `*`, `/`, and `**` are supported. For example, if you write `1/3` anywhere in the pattern, to slice beats or as effect value, it will be replaced by `0.33333333`
113
- #### using samples, mixing two songs
114
  - WIP (you can do that if you run locally, I am just figuring out gradio UI because that requires a bunch of new input interface)
115
- #### other stuff
116
  - `i` will be replaced by current position, e.g. `i, i, i, i+1` is equvalent to `1, 2, 3, 4 + 1`, or `1, 2, 3, 5`.
117
  - `#` will add shuffle all beats with the same number after it. `1#1, 2#2, 3#1, 4#2, 5#1, 6#2, 7#1, 8#2` will shuffle 1st, 3rd, 5th and 7th beats (the are in 1st group), and 2nd, 4th, 6th and 8th beats - from 2nd shuffle group.
118
  - `!` skips that beat. If you want to remove every 4th beat, you can't just do `1, 2, 3`, because that would simply play every 3 beats. So to play 3 beats every 4 beats, you can write `1, 2, 3, 4!`
119
  - `?` makes that beat not count for pattern size. For example, `1, 2, 3, 8` will normally repeat every 8 beats because 8 is the highest number, but `1, 2, 3, 8?` will repeat every 3 beats.
120
  - `@` allows you to take a random beat with the following syntax: @start_stop_step. For example, `@1_4_0.5` means it will take a random beat out of 1st, 1.5, 2nd, 2.5, 3rd, 3.5, and 4th. It will take whole beat, so you can also add `>0.5` to take only first half.
121
  - `%` - for very advanced patterns you can create variables from various metrics. For example, `%v` will create a variable with average volume of that beat, and all following `%` will be replaced by that variable until you create a new one. Useful for applying different effects based on different song metrics. All metrics are in `beat_manipulator/metrics.py`.
122
- #### special patterns
123
  You can write special commands into the `pattern` argument instead of actual patterns.
124
  - `reverse` - plays all beats in reverse chronological order
125
  - `shuffle` - shuffles all beats
126
  - `test` - puts different pitched cowbells on each beat, useful for testing beat detection and adjusting it using scale and shift. Each cowbell is 1 beat, highest pitched cowbell is the 1st beat, lowest pitched - 4th.
127
  #### complex patterns
128
  You should be able to use all of the above operators in any combination, as complex as you want. Very low scales should also be fine, up to 0.01.
129
- ### scale
130
  `scale = 0.5` will insert a new beat position between every existing beat position in the beatmap. That allows you to make patterns on smaller intervals.
131
 
132
  `scale = 2`, on the other hand, will merge every two beat positions in the beatmap. Useful, for example, when beat map detection puts sees BPM as two times faster than it actually is, and puts beats in between every actual beat.
133
- ### shift
134
  Shifts the beatmap, in beats. For example, if you want to remove 4th beat every four beats, you can do it by writing `1, 2, 3, 4!`. However sometimes it doesn't properly detect which beat is first, and for example remove 2nd beat every 4 beats instead. In that case, if you want 4th beat, use `shift = 2`. Also sometimes beats are detected right in between actual beats, so shift = 0.5 or -0.5 will fix it.
135
  ## creating images
136
  You can create cool images based on beat positions. Each song produces its own unique image. This gradio app creates a 2048x2048 image from each song.
137
- ## presets
138
  A bunch of example patterns: https://github.com/stunlocked1/beat_manipulator/blob/main/beat_manipulator/presets.yaml
139
 
140
  Those are supposed to be used on normalized beat maps, where kick + snare is two beats, so make sure to adjust beatmaps using `scale` and `shift`.
141
 
142
- # My soundcloud https://soundcloud.com/stunlocked
 
 
 
 
 
 
143
  """
144
  ).launch(share=False)
 
43
  y=min(len(image), len(image[0]), 2048)
44
  y=max(y, 2048)
45
  image = np.rot90(np.clip(cv2.resize(image, (y,y), interpolation=cv2.INTER_NEAREST), -1, 1))
 
46
  #print(image)
47
  except Exception as e:
48
  print(f'Image generation failed: {e}')
 
54
  return ((song.sr, song.audio), image)
55
 
56
  audiofile=Audio(source='upload', type='filepath')
57
+ patternbox = Textbox(label="Pattern:", placeholder="1, 3, 2, 4!", value="1, 2>0.5, 3, 4>0.5, 5, 6>0.5, 3, 4>0.5, 7, 8", lines=1)
58
  scalebox = Textbox(value=1, label="Beatmap scale. At 2, every two beat positions will be merged, at 0.5 - a beat position added between every two existing ones.", placeholder=1, lines=1)
59
  shiftbox = Textbox(value=0, label="Beatmap shift, in beats (applies before scaling):", placeholder=0, lines=1)
60
  cachebox = Checkbox(value=True, label="Enable caching generated beatmaps for faster loading. Saves a file with beat positions and loads it when you open same audio again.")
 
64
  title = "Stunlocked's Beat Manipulator"
65
  ,description = """Remix music using AI-powered beat detection and advanced beat swapping. Make \"every other beat is missing\" remixes, or completely change beat of the song.
66
 
67
+ Github - https://github.com/stunlocked1/beat_manipulator.
68
 
69
  Colab version - https://colab.research.google.com/drive/1gEsZCCh2zMKqLmaGH5BPPLrImhEGVhv3?usp=sharing"""
70
+ ,article="""# <h1><p style='text-align: center'><a href='https://github.com/stunlocked1/beat_manipulator' target='_blank'>Github</a></p></h1>
71
+ ### Basic usage
 
 
72
  Upload your audio, enter the beat swapping pattern, change scale and shift if needed, and run it.
73
 
74
+ ### pattern syntax
75
+ `1, 3, 2, 4` means every 4 beats, swap 2nd and 3rd beats. You can do much more, like applying audio effects, shuffling beats, slicing them, mixing two songs, adding samples, sidechain.
 
 
76
 
77
  You can use spaces freely in patterns for formatting. Most other symbols have some use though. Here is the full syntax:
78
+ ##### beats
79
  * `1` - 1st beat;
80
  * `1>0.5` - first half of first beat
81
  * `1<0.5` - second half of first beat
82
  * `0:0.5` - range of beats, this also means first half of first beat, but with this you can do complex stuff like `1.25:1.5`. However this one is a bit more confusing because indexing starts from 0, so 1:2 is second beat, not first.
83
  * Also sometimes it is more convenient to use smaller `scale`, like 0.5 or 0.25, instead of slicing beats.
84
+ ##### basic patterns
85
  - `1, 3, 2, 4` - 3rd and 2nd beats will be swapped every 4 beats. Happens every 4 beats because 4 is the biggest number in the pattern.
86
  - `1, 2, 3, 4!` - every 4 beats, it will remove the 4th beat. `!` allows you to skip a beat but it still counts for pattern size.
 
87
  - `1, 4` skips 2nd and 3rd beat
88
  - `1; 2` plays 1st and second beat at the same time.
89
+ ##### joining operators
90
  `,` and `;` are beat joining operators. Here are all available joiners:
91
  - `,` puts the beat next to previous beat.
92
  - `;` puts the beat on top of previous beat. Normalizes the volume to avoid clipping. If previous beat is shorter, your beat will be shortened to match it.
93
  - `^` multiplies previous beat by your beat. This can be used for fake sidechain.
94
  - `$` adds the beat on top of previous beat + sidechains previous beat by your beat (I haven't tested this one)
95
+ ##### effects
96
  beats can be followed by effects. For example `1s0.75` means take first beat and play it at 0.75x speed. Here are all available effects:
97
  - `s` - speed. `1s2` means first beat will be played at 2x speed.
98
  - `r` - reverse. `1r` means first beat will have reversed audio.
 
102
  - `g` - gradient, sounds like highpass. `1g1` is the recommended value
103
  - `c` - channel. If not followed by number, swaps channels. If followed by 0, plays only left channel. If 1, only right channel
104
  - mixing effects - `1s2rd8` - take first beat, play at 2x speed, reversed, and downsampled.
105
+ ##### math
106
  mathematical expressions with `+`, `-`, `*`, `/`, and `**` are supported. For example, if you write `1/3` anywhere in the pattern, to slice beats or as effect value, it will be replaced by `0.33333333`
107
+ ##### using samples, mixing two songs
108
  - WIP (you can do that if you run locally, I am just figuring out gradio UI because that requires a bunch of new input interface)
109
+ ##### other stuff
110
  - `i` will be replaced by current position, e.g. `i, i, i, i+1` is equvalent to `1, 2, 3, 4 + 1`, or `1, 2, 3, 5`.
111
  - `#` will add shuffle all beats with the same number after it. `1#1, 2#2, 3#1, 4#2, 5#1, 6#2, 7#1, 8#2` will shuffle 1st, 3rd, 5th and 7th beats (the are in 1st group), and 2nd, 4th, 6th and 8th beats - from 2nd shuffle group.
112
  - `!` skips that beat. If you want to remove every 4th beat, you can't just do `1, 2, 3`, because that would simply play every 3 beats. So to play 3 beats every 4 beats, you can write `1, 2, 3, 4!`
113
  - `?` makes that beat not count for pattern size. For example, `1, 2, 3, 8` will normally repeat every 8 beats because 8 is the highest number, but `1, 2, 3, 8?` will repeat every 3 beats.
114
  - `@` allows you to take a random beat with the following syntax: @start_stop_step. For example, `@1_4_0.5` means it will take a random beat out of 1st, 1.5, 2nd, 2.5, 3rd, 3.5, and 4th. It will take whole beat, so you can also add `>0.5` to take only first half.
115
  - `%` - for very advanced patterns you can create variables from various metrics. For example, `%v` will create a variable with average volume of that beat, and all following `%` will be replaced by that variable until you create a new one. Useful for applying different effects based on different song metrics. All metrics are in `beat_manipulator/metrics.py`.
116
+ ##### special patterns
117
  You can write special commands into the `pattern` argument instead of actual patterns.
118
  - `reverse` - plays all beats in reverse chronological order
119
  - `shuffle` - shuffles all beats
120
  - `test` - puts different pitched cowbells on each beat, useful for testing beat detection and adjusting it using scale and shift. Each cowbell is 1 beat, highest pitched cowbell is the 1st beat, lowest pitched - 4th.
121
  #### complex patterns
122
  You should be able to use all of the above operators in any combination, as complex as you want. Very low scales should also be fine, up to 0.01.
123
+ #### scale
124
  `scale = 0.5` will insert a new beat position between every existing beat position in the beatmap. That allows you to make patterns on smaller intervals.
125
 
126
  `scale = 2`, on the other hand, will merge every two beat positions in the beatmap. Useful, for example, when beat map detection puts sees BPM as two times faster than it actually is, and puts beats in between every actual beat.
127
+ #### shift
128
  Shifts the beatmap, in beats. For example, if you want to remove 4th beat every four beats, you can do it by writing `1, 2, 3, 4!`. However sometimes it doesn't properly detect which beat is first, and for example remove 2nd beat every 4 beats instead. In that case, if you want 4th beat, use `shift = 2`. Also sometimes beats are detected right in between actual beats, so shift = 0.5 or -0.5 will fix it.
129
  ## creating images
130
  You can create cool images based on beat positions. Each song produces its own unique image. This gradio app creates a 2048x2048 image from each song.
131
+ ### presets
132
  A bunch of example patterns: https://github.com/stunlocked1/beat_manipulator/blob/main/beat_manipulator/presets.yaml
133
 
134
  Those are supposed to be used on normalized beat maps, where kick + snare is two beats, so make sure to adjust beatmaps using `scale` and `shift`.
135
 
136
+ ### Changelog:
137
+ - play two beats at the same time by using `;` instead of `,`
138
+ - shuffle and randomize beats
139
+ - gradient effect, similar to high pass
140
+ - significantly reduced clicking
141
+
142
+ ### My soundcloud https://soundcloud.com/stunlocked
143
  """
144
  ).launch(share=False)