Controls and Art

Flock Logic was developed as an art and engineering project to explore how the feedback laws used to model flocking translate when applied by dancers. The artistic goal was to create choreographic tools that leverage multiagent system dynamics with designed feedback and interaction. The engineering goal was to provide insights and design principles for multiagent systems, such as human crowds, animal groups, and robotic networks, by examining what individual dancers do and what emerges at the group level. We describe our methods to create dance and investigate collective motion. We analyze video of an experiment in which dancers moved according to simple rules of cohesion and repulsion with their neighbors. Using the prescribed interaction protocol and tracked trajectories, we estimate the time-varying graph that defines who is responding to whom. We compute status of nodes in the graph and show the emergence of leaders. We discuss results and further directions.

This chapter summarizes work to understand various aspects of the communication that occurs through the movements of partners in a dance. The first part of the paper adopts the terminology of motion description languages and deconstructs an elementary form of the well-known popular dance, salsa, in terms of four motion primitives (dance steps). These motion primitives can be specified entirely by the motions of the dancer’s feet, and hence the motions can be effectively carried out by simple unicycle-like mobile robots. We describe an experiment in which ten performances by an actual pair of dancers are evaluated by judges and then compared in terms of several complexity metrics. An energy metric is also defined. Values of this metric are obtained by summing up the lengths of motion segments executed by wheeled robots replicating the movements of the human dancers in each of ten dance performances. Of all the metrics that are considered in this experiment, energy is the most closely correlated with the human judges assessments of performance quality. The second part of the paper discusses an enhanced form of (intermediate level) salsa in which upper body motions play a role in the steps of the dance. In this version, it is stipulated that the dancers must remain in physical contact by holding hands. The number of motion primitives is increased to eleven, and the requirement that the dancers’ hands must remain linked imposes constraints on the structure of the dance sequences. These constraints are discussed using simple ideas from topological knot theory. Using a natural Markov model to generate possible dance sequences, it is shown that because the topological constraints enforce syntactic constraints on the motion transitions, the entropy rate of the intermediate level salsa is smaller than that of the beginner level salsa. The chapter concludes with a brief discussion of the challenges and possible approaches to creating robotic movement that will be perceived as having artistic merit.

This chapter reviews an approach for generating rhythmic flight motions that are executed by quadrocopters and timed to music. It represents a research and artistic experiment, which explores for the first time the potential of using flying vehicles in rhythmic, musical performances. We introduce periodic movements as the basic motion elements of such a performance, and derive control algorithms for guiding the vehicles along the desired motion paths and synchronizing their motion to the music. The vehicle dynamics and constraints are taken into account to determine, prior to flight, which motions are feasible. We demonstrate the resulting multivehicle flight performances at the ETH Zurich Flying Machine Arena.

This chapter outlines the design of software for embedded control of robotic marionettes using choreography. In traditional marionette puppetry, the puppets often possess dynamics that are quite different from the creatures they imitate. Puppeteers must therefore understand and leverage the inherent dynamics of the puppets to create believable and expressive characters. Because marionettes are actuated by strings, the mechanical description of the marionettes either creates a multiscale or degenerate system—making simulation of the constrained dynamics challenging. Moreover, marionettes have 40–50 degrees of freedom with closed kinematic chains. Generating puppet choreography that is mimetic (that is, recognizably human) results in a high-dimensional nonlinear optimal control problem that must be solved for each motion. In performance, these motion primitives must be combined in a way that preserves stability, resulting in an optimal timing control problem. Our software accounts for the efficient computation of the (1) discrete time dynamics that preserve the constraints and other integrals of motion, (2) nonlinear optimal control policies (including optimal control of LTV systems), and (3) optimal timing of choreography, all within a single framework. We discuss our current results and the potential application of our findings across disciplines, including the development of entertainment robots and autonomous theater.

A large class of control problems in multi-agent systems use the so-called consensus protocol to achieve coordinated motion among a team of agents. Inspired by the “standard” consensus protocol ẋ = -Lx, in this paper we propose a decentralized control law for multi-agent formations in two dimensions that allows the participating vehicles to display intricate periodic and quasi-periodic geometric patterns. Similarly to the standard consensus protocol, these controls rely only on the relative position between the networked agents which are neighbors in the underlying communication graph. Several examples are presented, resulting in nontrivial geometric patterns described by trochoidal curves, similar to those generated using a spirograph. These paths can be useful for coordinated, distributed surveillance, and monitoring applications, as well as for the sake of their own esthetic beauty.

For a responsive audio art installation in a skylit atrium, we developed a single-camera statistical segmentation and tracking algorithm. The algorithm combines statistical background image estimation, per-pixel Bayesian classification, and an approximate solution to the multi-target tracking problem using a bank of Kalman filters and Gale-Shapley matching. A heuristic confidence model enables selective filtering of tracks based on dynamic data. Experiments suggest that our algorithm improves recall and \(F_{2}\)-score over existing methods in OpenCV 2.1. We also find that feedback between the tracking and the segmentation systems improves recall and \(F_{2}\)-score. The system operated effectively for 5–8 h per day for 4 months. Source code and sample data is open source and available in OpenCV.

This chapter reviews a framework for generating robotic motion and describes its application in a performance at Georgia Institute of Technology. In particular, the movement model stems from the view that cannons of basic warm-up exercises in formalized movement genres, such as classical ballet, seed more complex phrases. This model employs a separation of basic movement ordering and execution. Basic movements are sequenced, and their individual execution modulated via a notion of quality from dance theory. The sequencing framework is then employed in performance both on a humanoid robot and real dancers. Results from a human study, questionnaires given to audience members after the show, are also presented. The generation framework also lends itself to movement interpretation and that extension will be briefly presented as well.