Computational Intelligence in Music, Sound, Art and Design

Frontmatter

Algorithmic Songwriting with ALYSIA

Abstract

This paper introduces ALYSIA: Automated LYrical SongwrIting Application. ALYSIA is based on a machine learning model using Random Forests, and we discuss its success at pitch and rhythm prediction. Next, we show how ALYSIA was used to create original pop songs that were subsequently recorded and produced. Finally, we discuss our vision for the future of Automated Songwriting for both co-creative and autonomous systems.

Margareta Ackerman, David Loker

On Symmetry, Aesthetics and Quantifying Symmetrical Complexity

Abstract

The concepts of order and complexity and their quantitative evaluation have been at the core of computational notion of aesthetics. One of the major challenges is conforming human intuitive perception and what we perceive as aesthetically pleasing with the output of a computational model. Informational theories of aesthetics have taken advantage of entropy in measuring order and complexity of stimuli in relation to their aesthetic value. However entropy fails to discriminate structurally different patterns in a 2D plane. In this work, following an overview on symmetry and its significance in the domain of aesthetics, a nature-inspired, swarm intelligence technique (Dispersive Flies Optimisation or DFO) is introduced and then adapted to detect symmetries and quantify symmetrical complexities in images. The 252 Jacobsen & Höfel’s images used in this paper are created by researchers in the psychology and visual domain as part of an experimental study on human aesthetic perception. Some of the images are symmetrical and some are asymmetrical, all varying in terms of their aesthetics, which are ranked by humans. The results of the presented nature-inspired algorithm is then compared to what humans in the study aesthetically appreciated and ranked. Whilst the authors believe there is still a long way to have a strong correlation between a computational model of complexity and human appreciation, the results of the comparison are promising.

Mohammad Majid al-Rifaie, Anna Ursyn, Robert Zimmer, Mohammad Ali Javaheri Javid

Towards Polyphony Reconstruction Using Multidimensional Multiple Sequence Alignment

Abstract

The digitization of printed music scores through the process of optical music recognition is imperfect. In polyphonic scores, with two or more simultaneous voices, errors of duration or position can lead to badly aligned and inharmonious digital transcriptions. We adapt biological sequence analysis tools as a post-processing step to correct the alignment of voices. Our multiple sequence alignment approach works on multiple musical dimensions and we investigate the contribution of each dimension to the correct alignment. Structural information, such musical phrase boundaries, is of major importance; therefore, we propose the use of the popular bioinformatics aligner Mafft which can incorporate such information while being robust to temporal noise. Our experiments show that a harmony-aware Mafft outperforms sophisticated, multidimensional alignment approaches and can achieve near-perfect polyphony reconstruction.

Dimitrios Bountouridis, Frans Wiering, Dan Brown, Remco C. Veltkamp

Melody Retrieval and Classification Using Biologically-Inspired Techniques

Abstract

Retrieval and classification are at the center of Music Information Retrieval research. Both tasks rely on a method to assess the similarity between two music documents. In the context of symbolically encoded melodies, pairwise alignment via dynamic programming has been the most widely used method. However, this approach fails to scale-up well in terms of time complexity and insufficiently models the variance between melodies of the same class. Compact representations and indexing techniques that capture the salient and robust properties of music content, are increasingly important. We adapt two existing bioinformatics tools to improve the melody retrieval and classification tasks. On two datasets of folk tunes and cover song melodies, we apply the extremely fast indexing method of the Basic Local Alignment Search Tool (BLAST) and achieve comparable classification performance to exhaustive approaches. We increase retrieval performance and efficiency by using multiple sequence alignment algorithms for locating variation patterns and profile hidden Markov models for incorporating those patterns into a similarity model.

Dimitrios Bountouridis, Dan Brown, Hendrik Vincent Koops, Frans Wiering, Remco C. Veltkamp

Evolved Aesthetic Analogies to Improve Artistic Experience

Abstract

It has been demonstrated that computational evolution can be utilised in the creation of aesthetic analogies between two artistic domains by the use of mapping expressions. When given an artistic input these mapping expressions can be used to guide the generation of content in a separate domain. For example, a piece of music can be used to create an analogous visual display. In this paper we examine the implementation and performance of such a system. We explore the practical implementation of real-time evaluation of evolved mapping expressions, possible musical input and visual output approaches, and the challenges faced therein. We also present the results of an exploratory study testing the hypothesis that an evolved mapping expression between the measurable attributes of musical and visual harmony will produce an improved aesthetic experience compared to a random mapping expression. Expressions of various fitness values were used and the participants were surveyed on their enjoyment, interest, and fatigue. The results of this study indicate that further work is necessary to produce a strong aesthetic response. Finally, we present possible approaches to improve the performance and artistic merit of the system.

Aidan Breen, Colm O’Riordan, Jerome Sheahan

Deep Artificial Composer: A Creative Neural Network Model for Automated Melody Generation

Abstract

The inherent complexity and structure on long timescales make the automated composition of music a challenging problem. Here we present the Deep Artificial Composer (DAC), a recurrent neural network model of note transitions for the automated composition of melodies. Our model can be trained to produce melodies with compositional structures extracted from large datasets of diverse styles of music, which we exemplify here on a corpus of Irish folk and Klezmer melodies. We assess the creativity of DAC-generated melodies by a new measure, the novelty of musical sequences, showing that melodies imagined by the DAC are as novel as melodies produced by human composers. We further use the novelty measure to show that the DAC creates melodies musically consistent with either of the musical styles it was trained on. This makes the DAC a promising candidate for the automated composition of convincing musical pieces of any provided style.

Florian Colombo, Alexander Seeholzer, Wulfram Gerstner

A Kind of Bio-inspired Learning of mUsic stylE

Abstract

In the field of Computer Music, computational intelligence approaches are very relevant for music information retrieval applications. A challenging task in this area is the automatic recognition of musical styles. The style of a music performer is the result of the combination of several factors such as experience, personality, preferences, especially in music genres where the improvisation plays an important role.

In this paper we propose a new approach for both recognition and automatic composition of music of a specific performer’s style. Such a system exploits: (1) a one-class machine learning classifier to learn a specific music performer’s style, (2) a music splicing system to compose melodic lines in the learned style, and (3) a LSTM network to predict patterns coherent with the learned style and used to guide the splicing system during the composition.

To assess the effectiveness of our system we performed several tests using transcriptions of solos of popular Jazz musicians. Specifically, with regard to the recognition process, tests were performed to analyze the capability of the system to recognize a style. Also, we show that performances of our classifier are comparable to that of traditional two-class SVM, and that it is able to achieve an accuracy of \(97\%\). With regard to the composition process, tests were performed to verify whether the produced melodies were able to catch the most significant music aspects of the learned style.

Roberto De Prisco, Delfina Malandrino, Gianluca Zaccagnino, Rocco Zaccagnino, Rosalba Zizza

Using Autonomous Agents to Improvise Music Compositions in Real-Time

Abstract

This paper outlines an approach to real-time music generation using melody and harmony focused agents in a process inspired by jazz improvisation. A harmony agent employs a Long Short-Term Memory (LSTM) artificial neural network trained on the chord progressions of 2986 jazz ‘standard’ compositions using a network structure novel to chord sequence analysis. The melody agent uses a rule-based system of manipulating provided, pre-composed melodies to improvise new themes and variations. The agents take turns in leading the direction of the composition based on a rating system that rewards harmonic consistency and melodic flow. In developing the multi-agent system it was found that implementing embedded spaces in the LSTM encoding process resulted in significant improvements to chord sequence learning.

Patrick Hutchings, Jon McCormack

Generating Polyphonic Music Using Tied Parallel Networks

Abstract

We describe a neural network architecture which enables prediction and composition of polyphonic music in a manner that preserves translation-invariance of the dataset. Specifically, we demonstrate training a probabilistic model of polyphonic music using a set of parallel, tied-weight recurrent networks, inspired by the structure of convolutional neural networks. This model is designed to be invariant to transpositions, but otherwise is intentionally given minimal information about the musical domain, and tasked with discovering patterns present in the source dataset. We present two versions of the model, denoted TP-LSTM-NADE and BALSTM, and also give methods for training the network and for generating novel music. This approach attains high performance at a musical prediction task and successfully creates note sequences which possess measure-level musical structure.

Daniel D. Johnson

Mixed-Initiative Creative Drawing with webIconoscope

Abstract

This paper presents the webIcononscope tool for creative drawing, which allows users to draw simple icons composed of basic shapes and colors in order to represent abstract semantic concepts. The goal of this creative exercise is to create icons that are ambiguous enough to confuse other people attempting to guess which concept they represent. webIcononscope is available online and all creations can be browsed, rated and voted on by anyone; this democratizes the creative process and increases the motivation for creating both appealing and ambiguous icons. To complement the creativity of the human users attempting to create novel icons, several computational assistants provide suggestions which alter what the user is currently drawing based on certain criteria such as typicality and novelty. This paper reports trends in the creations of webIcononscope users, based also on feedback from an online audience.

Antonios Liapis

Clustering Agents for the Evolution of Autonomous Musical Fitness

Abstract

This paper presents a cyclical system that generates autonomous fitness functions or Agents for evolving short melodies. A grammar is employed to create a corpus of melodies, each of which is composed of a number of segments. A population of Agents are evolved to give numerical judgements on the melodies based on the spacing of these segments. The fitness of an individual Agent is calculated in relation to its clustering of the melodies and how much this clustering correlates with the clustering of the entire Agent population. A preparatory run is used to evolve Agents using 30 melodies of known ‘clustering’. The full run uses these Agents as the initial population in evolving a new best Agent on a separate corpus of melodies of random distance measures. This evolved Agent is then used in combination with the original melody grammar to create a new melody which replaces one of those from the initial random corpus. This results in a complex adaptive system creating new melodies without any human input after initialisation. This paper describes the behaviour of each phase in the system and presents a number of melodies created by the system.

Róisín Loughran, Michael O’Neill

EvoFashion: Customising Fashion Through Evolution

Abstract

In today’s society, where everyone desires unique and fashionable products, the ability to customise products is almost mandatory in every online store. Despite of many stores allowing the users to personalize their products, they do not always do it in the most efficient and user-friendly manner. In order to have products that reflect the user’s design preferences, they have to go through a laborious process of picking the components that they want to customise. In this paper we propose a framework that aims to relieve the design burden from the user side, by automating the design process through the use of Interactive Evolutionary Computation (IEC). The framework is based on a web-interface that facilitates the interaction between the user and the evolutionary process. The user can select between two types of evolution: (i) automatic; and (ii) partially-automatic. The results show the ability of the framework to promote evolution towards solutions that reflect the user aesthetic preferences.

Nuno Lourenço, Filipe Assunção, Catarina Maçãs, Penousal Machado

A Swarm Environment for Experimental Performance and Improvisation

Abstract

This paper describes Swarm Performance and Improvisation (Swarm-PI), a real-time computer environment for music improvisation that uses swarm algorithms to control sound synthesis and to mediate interactions with a human performer. Swarm models are artificial, multi-agent systems where the organized movements of large groups are the result of simple, local rules between individuals. Swarms typically exhibit self-organization and emergent behavior. In Swarm-PI, multiple acoustic descriptors from a live audio feed generate parameters for an independent swarm among multiple swarms in the same space, and each swarm is used to synthesize a stream of sound using granular sampling. This environment demonstrates the effectiveness of using swarms to model human interactions typical to group improvisation and to generate organized patterns of synthesized sound.

Frank Mauceri, Stephen M. Majercik

Niche Constructing Drawing Robots

Abstract

This paper describes a series of experiments in creating autonomous drawing robots that generate aesthetically interesting and engaging drawings. Based on a previous method for multiple software agents that mimic the biological process of niche construction, the challenge in this project was to re-interpret the implementation of a set of evolving software agents into a physical robotic system. In this new robotic system, individual robots try to reinforce a particular niche defined by the density of the lines drawn underneath them. The paper also outlines the role of environmental interactions in determining the style of drawing produced.

Jon McCormack

Automated Shape Design by Grammatical Evolution

Abstract

This paper proposes a automated shape generation methodology based on grammatical genetic programming for specific design cases. Two cases of the shape generation are presented: architectural envelope design and facade design. Through the described experiments, the applicability of this evolutionary method for design applications is showcased. Through this study it can be seen that automated shape generation by grammatical evolution offers a huge potential for the development of performance-based creative systems.

Manuel Muehlbauer, Jane Burry, Andy Song

Evolutionary Image Transition Using Random Walks

Abstract

We present a study demonstrating how random walk algorithms can be used for evolutionary image transition. We design different mutation operators based on uniform and biased random walks and study how their combination with a baseline mutation operator can lead to interesting image transition processes in terms of visual effects and artistic features. Using feature-based analysis we investigate the evolutionary image transition behaviour with respect to different features and evaluate the images constructed during the image transition process.

Aneta Neumann, Bradley Alexander, Frank Neumann

Evaluation Rules for Evolutionary Generation of Drum Patterns in Jazz Solos

Abstract

The learning of improvisation in jazz and other music styles requires years of practice. For music scholars which do not play in a band, technical solutions for automatic generation of accompaniment on home computers are very helpful. They may support the learning process and significantly improve the experience to play with other musicians. However, many up-to-date approaches can not interact with a solo player, generating static or random patterns without a direct musical dialogue between a soloist and accompanying instruments. In this paper, we present a novel system for the generation of drum patterns based on an evolutionary algorithm. As the main extension to existing solutions, we propose a set of musically meaningful jazz-related rules for the real-time validation and adjustment of generated drum patterns. In the evaluation study, musicians agreed that the system can be successfully used for learning of jazz improvisation and that the wide range of parameters helps to adapt the response of the virtual drummer to the needs of individual scholars (Examples of generated music are available at http://​sig-ma.​de/​wp-content/​uploads/​2017/​01/​JazzDrumPatterns​.​zip.).

Fabian Ostermann, Igor Vatolkin, Günter Rudolph

Assessing Augmented Creativity: Putting a Lovelace Machine for Interactive Title Generation Through a Human Creativity Test

Abstract

The aim of this study is to find to what extent computers can assist humans in the creative process of writing titles, using psychological tests for creativity that are typically used for humans only. To this end, a computer tool was designed that recommends new titles to users, based on knowledge generated from a pre-built corpus. This paper gives a description of both the development of the system as well as tests applied to the participants, derived from classical psychological tests for human creativity. A total of 89 participants divided in two groups completed two tasks which consisted of generating titles for paintings. One group was allowed to use a template-based system for generating titles, the other group did not use any tools. The results of the experiments show higher creativity scores for the combination of participants augmented by a computational creativity tool.

Yasser S. Arenas Rebolledo, Peter van der Putten, Maarten H. Lamers

Play it Again: Evolved Audio Effects and Synthesizer Programming

Abstract

Automatic programming of sound synthesizers and audio devices to match a given, desired sound is examined and a Genetic Algorithm (GA) that functions independent of specific synthesis techniques is proposed. Most work in this area has focused on one synthesis model or synthesizer, designing the GA and tuning the operator parameters to obtain optimal results. The scope of such inquiries has been limited by available computing power, however current software (Ableton Live, herein) and commercially available hardware is shown to quickly find accurate solutions, promising a practical application for music creators. Both software synthesizers and audio effects processors are examined, showing a wide range of performance times (from seconds to hours) and solution accuracy, based on particularities of the target devices. Random oscillators, phase synchronizing, and filters over empty frequency ranges are identified as primary challenges for GA based optimization.

Benjamin D. Smith

Fashion Design Aid System with Application of Interactive Genetic Algorithms

Abstract

These days, consumers can make their choice from a wide variety of clothes provided in the market; however, some prefer to have their clothes custom-made. Since most of these consumers are not professional designers, they contact a designer to help them with the process. This approach, however, is not efficient in terms of time and cost and it does not reflect the consumer’s personal taste as much as desired. This study proposes a design system using Interactive Genetic Algorithm (IGA) to overcome these problems. IGA differs from traditional Genetic Algorithm (GA) by leaving the fitness function to the personal preference of the user. The proposed system uses user’s taste as a fitness value to create a large number of design options, and it is based on an encoding scheme either describing a dress as a whole or as a two-part piece of clothing. The system is designed in the Rhinoceros 3D software, using python, which provides good speed and interface options. The assessment experiments with several subjects indicated that the proposed system is effective.

Nazanin Alsadat Tabatabaei Anaraki

Generalisation Performance of Western Instrument Recognition Models in Polyphonic Mixtures with Ethnic Samples

Abstract

Instrument recognition in polyphonic audio recordings is a very complex task. Most research studies until now were focussed on the recognition of Western instruments in Western classical and popular music, but also an increasing number of recent works addressed the classification of ethnic/world recordings. However, such studies are typically restricted to one kind of music and do not measure the bias of “Western” effect, i.e., the danger of overfitting towards Western music when the classification models are optimised only for such tracks. In this paper, we analyse the performance of several instrument classification models which are trained and optimised on polyphonic mixtures of Western instruments, but independently validated on mixtures created with randomly added ethnic samples. The conducted experiments include evolutionary multi-objective feature selection from a large set of audio signal descriptors and the estimation of individual feature relevance.

Igor Vatolkin

Exploring the Exactitudes Portrait Series with Restricted Boltzmann Machines

Abstract

In this paper we explore the use of deep neural networks to analyze semi-structured series of artworks. We train stacked Restricted Boltzmann Machines on the Exactitudes collection of photo series, and use this to understand the relationship between works and series, uncover underlying features and dimensions, and generate new images. The projection of the series on the two major decorrelated features (PCA on top of Boltzmann features) results in a visualization that clearly reflects the semi structured nature of the photos series, although the original features provide better classification results when assigning photographs to series. This work provides a useful case example of understanding structure that is uncovered by deep neural networks, as well as a tool to analyze the underlying structure of a collection of visual artworks, as a very first step towards a robot curator.

Sam D. Verkoelen, Maarten H. Lamers, Peter van der Putten

Evolving Mondrian-Style Artworks

Abstract

This paper describes a Genetic Algorithm (GA) software system for automatically generating Mondrian-style symmetries and abstract artwork. The research examines Mondrian’s paintings from 1922 through 1932 and analyses the balances, color symmetries and composition in these paintings. We used a set of eleven criteria to define the automated system. We then translated and formulized these criteria into heuristics and criteria that can be measured and used in the GA algorithm. The software includes a module that provides a range of GA parameter values for interactive selection. Despite a number of limitations, the method yielded high quality results with colors close to those of Mondrian and rectangles that did not overlap and fit the canvas.

Miri Weiss Cohen, Leticia Cherchiglia, Rachel Costa

Predicting Expressive Bow Controls for Violin and Viola

Abstract

Though computational systems can simulate notes on a staff of sheet music, capturing the artistic liberties professional musicians take to communicate their interpretation of those notes is a much more difficult task. In this paper, we demonstrate that machine learning methods can be used to learn models of expressivity, focusing on bow articulation for violin and viola. First we describe a new data set of annotated sheet music with information about specific aspects of bow control. We then present experiments for building and testing predictive models for these bow controls, as well as analysis that includes both general metrics and manual examination.

Lauren Jane Yu, Andrea Pohoreckyj Danyluk

Backmatter