Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2021 | OriginalPaper | Chapter

SerumRNN: Step by Step Audio VST Effect Programming

Authors: Christopher Mitcheltree, Hideki Koike

Published in: Artificial Intelligence in Music, Sound, Art and Design

Publisher: Springer International Publishing

share
SHARE

Abstract

Learning to program an audio production VST synthesizer is a time consuming process, usually obtained through inefficient trial and error and only mastered after years of experience. As an educational and creative tool for sound designers, we propose SerumRNN: a system that provides step-by-step instructions for applying audio effects to change a user’s input audio towards a desired sound. We apply our system to Xfer Records Serum: currently one of the most popular and complex VST synthesizers used by the audio production community. Our results indicate that SerumRNN is consistently able to provide useful feedback for a variety of different audio effects and synthesizer presets. We demonstrate the benefits of using an iterative system and show that SerumRNN learns to prioritize effects and can discover more efficient effect order sequences than a variety of baselines.
Appendix
Available only for authorised users
Literature
3.
go back to reference Cáceres, J.P.: Sound design learning for frequency modulation synthesis parameters (2007) Cáceres, J.P.: Sound design learning for frequency modulation synthesis parameters (2007)
4.
go back to reference Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus). CoRR abs/1511.07289 (2016) Clevert, D.A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus). CoRR abs/1511.07289 (2016)
5.
go back to reference Damskägg, E.P., Juvela, L., Thuillier, E., Välimäki, V.: Deep learning for tube amplifier emulation. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 471–475 (2019) Damskägg, E.P., Juvela, L., Thuillier, E., Välimäki, V.: Deep learning for tube amplifier emulation. In: ICASSP 2019–2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 471–475 (2019)
10.
go back to reference Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2015) Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs/1412.6980 (2015)
11.
go back to reference Ly, E., Villegas, J.: Genetic reverb: synthesizing artificial reverberant fields via genetic algorithms. In: EvoMUSART (2020) Ly, E., Villegas, J.: Genetic reverb: synthesizing artificial reverberant fields via genetic algorithms. In: EvoMUSART (2020)
12.
go back to reference Macret, M., Pasquier, P.: Automatic design of sound synthesizers as pure data patches using coevolutionary mixed-typed cartesian genetic programming. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 309–316 (2014). https://​doi.​org/​10.​1145/​2576768.​2598303 Macret, M., Pasquier, P.: Automatic design of sound synthesizers as pure data patches using coevolutionary mixed-typed cartesian genetic programming. In: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, GECCO 2014, pp. 309–316 (2014). https://​doi.​org/​10.​1145/​2576768.​2598303
13.
go back to reference Ramírez, M.A.M., Reiss, J.: End-to-end equalization with convolutional neural networks. In: Proceedings of the 21st International Conference on Digital Audio Effects (DAFx-18) (2018) Ramírez, M.A.M., Reiss, J.: End-to-end equalization with convolutional neural networks. In: Proceedings of the 21st International Conference on Digital Audio Effects (DAFx-18) (2018)
14.
go back to reference Sheng, D., Fazekas, G.: A feature learning siamese model for intelligent control of the dynamic range compressor. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2019) Sheng, D., Fazekas, G.: A feature learning siamese model for intelligent control of the dynamic range compressor. In: 2019 International Joint Conference on Neural Networks (IJCNN), pp. 1–8 (2019)
15.
go back to reference Sommer, N., Ralescu, A.: Developing a machine learning approach to controlling musical synthesizer parameters in real-time live performance. In: MAICS (2014) Sommer, N., Ralescu, A.: Developing a machine learning approach to controlling musical synthesizer parameters in real-time live performance. In: MAICS (2014)
16.
go back to reference Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014) MathSciNetMATH Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014) MathSciNetMATH
17.
go back to reference Tatar, K., Macret, M., Pasquier, P.: Automatic synthesizer preset generation with presetgen. J. New Music Res. 45, 124–144 (2016) CrossRef Tatar, K., Macret, M., Pasquier, P.: Automatic synthesizer preset generation with presetgen. J. New Music Res. 45, 124–144 (2016) CrossRef
18.
go back to reference Thio, V., Donahue, C.: Neural loops. In: NeurIPS Workshop on Machine Learning for Creativity and Design (2019) Thio, V., Donahue, C.: Neural loops. In: NeurIPS Workshop on Machine Learning for Creativity and Design (2019)
Metadata
Title
SerumRNN: Step by Step Audio VST Effect Programming
Authors
Christopher Mitcheltree
Hideki Koike
Copyright Year
2021
DOI
https://doi.org/10.1007/978-3-030-72914-1_15