“It’s cleaner, definitely”: Collaborative Process in Audio Production

Fieldwork was conducted at the workplace of Commands (Kyle & Keir), a versatile and much in demand production duo based in a professional music studio in central London. Commands work regularly with international pop artists and record companies. The duo has extensive, individual and shared, professional production experience in electronic dance and pop music. Both are trained musicians and producers, and have experience of performing in bands and as solo artists. The fieldwork took place over 3 months, with five sessions worth of video, along with a series of other visits on non-consecutive days. The musical content being worked on each visit was different. The basement studio is situated with six other studios belonging to their publisher, a studio manager’s office, a shared kitchen and toilets, and a stairwell up to the outside world. The studio is a room cut off from the outside world by a heavy, soundproofed door. The room contains a vast array of audio production equipment and musical instruments; from simple toys through to expensive professional ‘gear’. The room is fitted with a mixture of acoustical padding material and various LED lights. Views of their workspaces can be seen in Figure 1. While being a technical space with specific perceptual features for music production, it is also a social one, people drop by from other studios in the building to chat, collaborate, and introduce clients. This structure provides access to new collaborations with artists or record companies. Figure 2 highlights a structural pillar within their studio, that acts as a situated history of this social interaction with numerous markings from visitors. The workspace and collaboration style of Commands is common for their industry, with writing teams often being co-located geographically (Bennett, 2014).

Commands’ work covers a range of activities: (i) recording sessions with vocalists; (ii) writing, production and mixing of music for other artists, both ghostwriting¹ and production as the Commands brand; (iii) remixing of songs; (iv) production of samples and tracks for library music; (v) production of music and sound design for moving image; (vi) making of music for individual artistic projects with other collaborators. Work activities not currently undertaken by Commands include (i) full band or instrumental recording sessions; (ii) mastering of songs or albums; (iii) preparation of music for live performance.

The study was carried out following ethical review by the Queen Mary Ethics Committee and all participants provided informed consent and agreed for data samples to be shared (Approval ID QMREC1619).

For the analysis in this paper, phases of work are articulated by their interactional features. Detailing interactional features focuses analysis on the real-time performance of activities, situated in a shared social and material space, using video-based interaction analysis (IA) (Jordan and Austin Henderson, 1995). Aspects of the IA were discussed in collaborative data sessions (Albert, 2017). The sessions focused on short segments of video data where the Commands duo works together on a song. The data format used in the presentation did not include any inferences collected during the analysis and coding, in order to have themes surface in the collaborative data sessions themselves. The findings and discussions of data sessions have been integrated into this work.

For the presentation of data, transcriptions include simple turns of talk, gesture, gaze, and posture. Adapting a heuristic device from previous research, transcriptions include references to audio playback and interface interaction during phases of action (Brooker and Sharrock, 2016). The combined transcription of interaction aims to address how sense-making is socially distributed and structured around the playback of audio and interface use. Fragment I is an example of the transcription method.

For those unfamiliar with audio production, this section provides a short review of key workflows in audio production using DAWs. Tracking is the live recording of audio into a DAW. Sampling is the use of pre-made audio files to produce layers of instrumentation. Synthesis is the use of sound generators to produce instrument layers for a song, synthesis is often MIDI² controlled. Editing is commonly associated with temporal selection, cropping, and arrangement of audio content in a DAW timeline, see Figure 3 for example. Mixing is the balancing the levels and frequencies of different parts of a song. In practice, a mixture of simple and advanced signal processing strategies are undertaken to mix a song. Within a DAW, whole workflows with dedicated UI views are used for mixing. Rendering/bouncing audio is the term used for the exporting audio files from a project, often for archival or distribution. Associated with bouncing out audio is the term Stems, these are often stereo recordings of parts of a song combining multiple individual tracks. For example, a drum stem will typically be a stereo audio file that sounds like all of the drum parts mixed together.

Some other practical terminology includes: Master track: is the main audio output for all audio tracks in a DAW session. Solo/Mute: Soloing a track means listening to it by itself, in DAWs multiple tracks can be soloed simultaneously, muting is silencing a track. Bussing: mixer routing abstraction process where the outputs of a series of tracks are routed together to apply group-level effects processing and volume control. Equalisation (EQ): amplifying or attenuating the levels of different frequencies in a signal. dB: stands for decibel, the unit of sound intensity; in audio production, it is used for mixer meters and volume control amounts. Side-chaining: a form of audio signal cross-modulation used to change the volume (or dynamic range) of a track based on another; commonly used in mixing to balance elements creating ‘space’ in the mix. Clipping: is when an audio signal is amplified past the maximum allowed limit, it leads to distortion and a lowering of audio quality. Stereo Imaging: is the positioning of sounds using two outputs (speakers or headphones), it is also used to create a sense of space for the listener.

This section presents an overview of how Commands structured their work processes for music production. Audio production work requires high levels of detailed editing in complex project hierarchies. This makes project and asset management a key concern from the outset of new commissions. DAWs have abstractions for managing sessions and file variations within them, but in their workflow, Commands manage numerous file and folder structures explicitly; this is similar to previous research findings (Duignan et al. 2010). The following wordings are based on Commands’ language and organisation process, rather than software or industry terms.

Project: An individual song within a commission. In the case of remixes, the projects are kept separately. As data, the project level is exemplified by folder structures for each project part. These link together with email communications and a whiteboard in the room that indicates different projects’ priority (Figure 4).

Tracks: Each session is made up of tracks e.g. vocal, guitar, drums, and sub-mixes. This term is used contextually to mean different things by the pair. For instance, a track can mean versions or project. More concretely in the production process, a track is a discrete component of audio, it can be either mono or stereo recorded audio or a MIDI instrument. Hereafter, the usage of the term Track refers to the concrete production process meaning. Individual sessions (DAW files) count between 10–300 tracks. The way Commands work means final versions typically have between 100-200 individual tracks, often with complex sonic interdependencies in the structure of each version.

In the following fragments, a dancehall/afrobeat song, composed and produced by Commands, has been returned to them with notes on ‘things to fix’ based on artist comments and the commissioner’s recommendations. In that context, the following phases of work are analysed:

In both features, Commands work on the bass sound of the song using the session views shown in Figure 5. The bass sound is made up by seven individual tracks and a bass buss, these can be seen in Figure 5d, the track number, names, routing, and functions are as follows: The ‘sub’ is a bass sound, it refers to frequency components < 100Hz. Each mixer track of the bass sound has an EQ, used to sculpt the sonic profile of that sound; these panels can be seen in Figure 5c.

DAW-based collaborative audio production is an interactionally-dependent process, where aesthetic decisions are embedded in social practice. In our data, content assessments were often marshalled using embodied behaviour (F. II-IV). Commands utilised an array of multimodal resources to move forward with work and support mutual understanding of problems. Multimodal resources include:

The contribution of gesture to the co-ordination of audio production cannot be underestimated. The availability of information about a collaborators’ gestural and postural relationships concerning specific references, timed precisely, is of importance. This is because embodied conduct in the studio structures how people problem-solve sonic issues. To diagnose problems, Commands: isolated sources and made available their assessment (F. II), coordinated media (F. IV), and directed action (F. VI). Gesture, posture, gaze, and speech, each offer ways to separate features within the shared problem context. For instance, Commands guided each other’s attention to facets of the sound and previous assessments while downgrading or upgrading specific assessments made in the process (F. V-VI). Each participant had to highlight the merits of individual solutions, often after trialling failed strategies. Here, interactional work supported the refocusing of efforts on other candidate solutions. To consider a solution, requests for reasoning create a requirement for an individual to explicate their current strategy. Similar to the appraisal of static visual art (Heath and Vom Lehn, 2004), a function of embodied conduct in making assessments is to continually establish co-orientation towards particular features of sonic objects and encourage each other to appreciate them in specific ways. Precisely what aspects of the song are targeted for assessment vary, but they are reflexively tied to the strategies and perspectives of the speakers assessing them.

By extending previous analytic categories (Brooker and Sharrock, 2016), we integrate our analyses in the idea of collaborative audio production interactional practices (CAPIP), categories include:

We acknowledge that these features provide an ordered gloss not present in reality, meaning that within any instant of interaction, multiple aspects of these features could be unfolding simultaneously. We discuss these practices further through three features of social interaction: Attunement, Experimentation, and Making Assessments.

Attunement is the uncovering of processes required to progress tasks. It is closely related to the critical listening and reviewing musical choices. In our data, attunement includes the shared analysis of the audio and negotiation of possible solutions (F. II), where Commands utilise audio, visual, and distributed memory resources (human and machine). Interaction modalities help to ground information in a shared context (Clark and Brennan, 1991), building levels of attunement to the group’s goals. Key to the attunement phase is that the work of problem solving is mutually resolved. We can see this in transition of problems from high level instructions down to smaller problems, solvable by content actions. Here, collaborators resolve different perspectives on the state of affairs, changing the navigation of the problem space from what might occur in an individual’s process. For example in Fragment II, Kyle interjected on Keir’s process of content action to suggest a simpler course of action to fix the bass instrument.

Experimentation links closely with Talking about musical ideas and Enacting musical ideas. It is the coordination of participation woven into audio production processes. When making content, iterative reflections are based on changes made to sound and other information. This makes experimentation in CAP is form of joint stimulus evaluation. Participants need to resolve different assessments of issues, using multiple information sources such as the audio, the DAW, and shared history of discussion. For instance, in Fragment IV there is an iterative process of collaborative listening, content review, and basic mixing actions (solo, mute, EQ changes). In that process, making clear assessments is complicated by the fluctuating sonic characteristics and multiple control opportunities present in DAWs (e.g. the mixer and EQ in F. VI). This is because it is hard to focus, critically listen, and specifically reference issues when content can change so quickly. This idea is discussed further in the next section.

Making Assessments highlights the exchange of information territory in Commands’ interactions. The territory comprises: parameter shifts in assessments of technical and aesthetic issues (F. IV, V, VI), self-repetition (F. III), clarification requests (F. IV), and confirmation checks (F. IV). The function of territory exchange is to anchor aesthetic or technical assessments to proceed with work. Making assessments is a key behaviour that weaves through all aspects of CAPIP.

Simple, situated, spatial practices allow Commands to structure the complexity of the workflow. For instance:

The iterative process of contextualisation using the audio and visual resources, mediated by interface control is a key characteristic of working with complex song structures in modern music production. The assembly of on-screen resources (panels for MIDI notes, audio, corresponding mixer channels, and signal processing panel views) are used for focusing attention and directing the action. But temporal variability highlights a problem in the song as a collaborative object, certain features have a level of indeterminacy. The visual representation of music objects exists at multiple levels, some temporally fluctuating (signal meters) and others ostensibly static (timeline representations, effects processing lists). Closely coupled with the audio-visual representations are the interactive features of music objects (equalisation, mixer controls, effects parameters), these elements of sound control can be utilised to continually modulate the other two forms of perceptual availability (audio, vision). What this interaction of representations and control opportunities creates is a level of indeterminacy, as soon as a feature of perception may be grasped in the auditory domain, it changes due to actions in the DAW. Also, this lack of stability is not isolated to one member in a collaboration, each person in a situation may be operating at different levels of critical listening or analytic focus. This social indeterminacy makes referential activity potentially ambiguous and open to interpretation in the process of making assessments.

So, the nature of working with audio can be viewed in a perceptually dependent cognitive loop, that exhibits levels of indeterminacy. But, changing perspective on a sonic problem allows alternate ideas and appraisals to be formed. This is conceptually similar to drawing, writing or other representation-based design problems; collaborators think through, and about, the different assemblages of resources individually and collaboratively. This malleability of what is happening, what could be done, and what should be done to content, provides an important concept for understanding the design of collaborative features for complex media workflows. Following this idea, an alternative way to think about studio work, and potential remote needs, is to consider how information sources are made available. Features would need to allow users to make their understanding available and contribute to their collaborators’ analysis of the materials.

Commands usually work on different deadlines, checking in with each other at key points before sending a project back to the client. Also, while the workspace and collaboration style of Commands is common for their industry, the solo producer is the most common form of employment in the production chain. What is shared by solo producers and collaborative groups like Commands is that content passes through a series of stages (a chain of production). Given this, there are important aspects of work not covered in this study. This study does not address the needs of the workplace that are coordinated at a larger scale, with distributed, temporally asynchronous tools. For instance, developing musical concepts at a very early stage in remote artist collaborations, the transition of projects between multiple parties, and chasing up of payment through a variety of social technologies. The incremental work done in these stages also requires significant collaboration over shared materials. To situate design implications for collaborative systems in the wider context of CAP would also be worthwhile, but doing so would require configuring goals and data collection differently to this study.

The musical work conducted by Commands during the ethnography was almost completely done within the DAW or relating to it. Periods of instrumental input (guitar, keyboards, synthesizer) iterated existing content or were used for problem-solving musical issues (harmony finding on a piano). Also the music style covered in our data (DAW-based electronic dance) does not represent all the aesthetic needs of CAP, such as instrumental artists performing, recording, and producing content over a series of days (Nabavian and Nick Bryan-Kinns, 2006; McGarry et al. 2017; Lefford and Thompson, 2018). Also, in their daily tasks Commands would switch in and out equipment (e.g. keyboards, guitars, microphones) based on the needs of the job. This is primarily a function of the size of the studio, and the variety of work they must complete on irregular schedules. Also, external hardware for effects processing, ‘out-board’, was used frequently to augment sonic characteristics of tracks in a session. Using the wording of previous research, the use of non-computer based systems introduces contextual meta-data about choices made in the production process (McGarry et al. 2017). But, the study presented in this paper does not extend understanding about how to capture or represent this meta-data in digital objects.

As described in the Research Approach, this paper describes the process of music making through interactional competencies with the lens of problem-solving. The paper’s findings analyse decisions made to progress ideas together, which resolve to periods of negotiation and subtle convergent work on bounded tasks. More generally, the approach taken by Commands was largely top-down, rather than generative and bottom-up. They divide tasks and progress through the workloads, making decisions and not backtracking unless necessary. When directly asked about their process, they highlighted that volume of work requires them to rationalise, make decisions, and progress quickly. But it is important to note that other forms of more generative work occurred in Commands’ music making. Within the other data collected, numerous instances of embodied imagination, musical virtuosity and creative collaborative process were observed. Also, despite having a collection of external hardware drum machines, all the drum programming took place within DAW interfaces. When asked why they did not use external drum machines, Kyle responded that those devices were only really used for personal ‘artist’ projects, where the boundaries of creation are perceived as different. Given this, we wish to acknowledge that technologies designed for audio production should undoubtedly accelerate core aspects of the craft, making sounds and being creative! Further studies should assess CAP in more generative creative contexts, using a broader array of musical tools, to build a complete picture of design needs.

A fundamental feature observed was that Commands shared the same sonic and visual environment, with this they had a form of collaborative perceptual space. With a shared environment they were able to make precise decisions about audio production issues. This relied on a level of perceptual consistency so that suggestions were mutually understood. While a somewhat obvious feature of the face-to-face situation observed, it does have important implications for CAP system design.

Perceptual consistency is problematic as music unfolds in time. The complexity of sonic features and visual representation form a tension for design, as we must balance the needs of critical listening and the visual monitoring of information. Visual interface design for audio is challenging given the sheer amount of sources used in projects. But high levels of detail are required to allow producers to see and act on problems in different ways. This level of detail makes referencing difficult. Also, because musical objects unfold in time as sound, aspects of them are only available temporarily. This is similar to other temporal interaction resources like speech, where referencing and assessment is specific, layered, and temporally located. These factors complicate the cognitive processes involved in joint attention and mutual understanding.

Looking forward to design, sharing perceptual space and it’s consistency, brings forward technical questions about temporality in audio workspaces. Namely, are collaborators working in the same moment of audio or at different points. Something as simple as the timeline play-head presents complex issues for collaboration. Imagine two users, working in headphones, at different points in a song’s timeline. At some point, one of them requires input on a decision or issue, at that moment the other user would have sampled entirely different information before that point. This would mean common ground would not be well set out, and a phase of attunement would probably be needed. This simple example, does not even factor in more advanced groupware implementation possibilities where the entire DAW could have individual states per user, such as different looping points, individual solo/mute arrangements, and unique bypass settings on effects plugins. For synchronous work, sharing precise assessments about audio content is a central concern that groupware feature design needs to focus on and experiment with. From our findings we suggest that something like single display groupware⁵ is a logical starting point for synchronous-time CAP work, given situated awareness needs (the specific timing of gesture, shared perceptual space, and trading of temporally precise assessments).

We position the following design ideas within the context of the collaborative audio production interactional processes, set out in Section 5.2. While our findings are based on purely co-located synchronous interaction, we suggest that aspects of situated practices in the studio can generalise to certain remote system design issues. However we do not discuss asynchronous features here.