Can you describe Spirits Melted Into Air?
Spirits Melted Into Air is a project which I produced for the Royal Shakespeare Company (and programmed by Caper) that explores and visualises the motion of actors on stage. Visualisations are generated through custom software: recording actors’ motions, and turning it into both large-scale prints and laser-cut wood. I did this for two fragments of scenes from this year’s productions of Richard III and The Comedy Of Errors.
As well as being a technology prototype, it’s also a data visualisation, and an artwork – all at once. It may be made of software and graphic design, but I really wanted the output to be beautiful in its own right: parametric art, as it were.
Shakespeare’s works are so often thought of as text on the page, but really, they’re plays to be performed. That’s what the RSC does: it turns text into productions. I wanted to highlight that in this commission, and so to do so, I wanted to find a way to strip the text away from dramatic performance. By removing the playwright, it would highlight the production elements – the actor, the director, choreographer, audience: everyone else who influences a performance.
As for its slightly wordy title: it comes from a line of Prospero’s in The Tempest:
“These our actors / As I foretold you, were all spirits and / Are melted into air, into thin air”.
It seemed apt – the actors dissolved into nothing but data.
How did the idea come about?
It began in a workshop with Caper to produce three commissions for myShakespeare, from Nat Buckley, myself, and Matthew Somerville. We explored lots of ideas of what we could do at the RSC with technology; everything from discussions of textual variance to installing sensors on stages. Eventually, we whittled down the ideas, and began to seize them as our own. I ended up picking the visualisation of movement idea, trusting in my own ability to work out how to write the code! Once the commissions had been established and agreed on, I dealt directly with the RSC, and they produced the project (excellently, I might add).
How long did it take to get to this stage?
So this is about two weeks’ work – though spread out over a longer period of time; I first committed code on it in August! There was around a solid week of technical work: building the code to record mouse movements, implementing the skewed projection, and overlaying that on top of video. There was also a lot of video transcoding in this period.
The second week of work was mainly about output: turning the data into attractive objects. That entitled designing graphic formats that could be re-used, laying out pages, designing laser-cutting templates and then getting prints and cuts made. In this second week, I also produced the explanatory materials – the film, photography, and website that is the public face of the work.
The work is part technology prototype, part data visualisation, and part artwork. For you personally, has it been chiefly a techy or arty project – or is the point perhaps that these distinctions are less meaningful now?
In the course of the project, I flip-flopped between the two: yes, it’s a technology project, and I was exploring what stories I could tell through technology about the RSC. At the same time, once I realised there was going to be output, I was very interested in making it beautiful, tactile. When I was up to my eyes in documentation and code, it didn’t feel much like art – but when I was laying out posters and designing type, it didn’t feel much like a software project!
I don’t think it’s art because the output is aesthetically attractive, though – it runs a bit deeper than that, for me. Whenever you create something, you put a bit of yourself into it: your ideas, your opinions, your politics. To that end, once I began creating my own output, it couldn’t really not be a kind of art. In this case, I was exploring a little bit of my own belief about the stage, about what theatre is.
Labels are still useful – but they’re rarely mutually exclusive.
Did anything about the actors’ movements around the stage surprise you, when you examined the visualisations?
The visualisations themselves were, in the end, roughly what I’d hope they’d be. But choosing the scenes to visualise was hard!
In Richard III, I picked Richard’s opening speech, “Now is the winter of our discontent”. I was hoping to show that what looks like long, static text on the page, is often quite lively. In fact, Jonjo O’Neill delivers this speech quite directly to the audience. There’s movement in there, but he’s really selling the words, and was somewhat still, so it wasn’t quite as physical as I was hoping. Still, there was enough to make the point I was trying to in the text, especially when you include his dramatic entrance.
As a result, I deliberately looked for something to counter this in my selection from Comedy of Errors, hence the focus on a scene with some physicality, some physical comedy and action, and two actors interacting.
Stripping away the text completely feels like a bold move – have you attempted to relate the motion to the stories or are you examining the pure production aspect?
Not really: whilst a lot of motion serves the story being told, much of it serves the drama: how the story needs to feel. Chaotic scenes feature chaotic movement. That’d be an interesting avenue to explore, though, especially across the length of a whole play – how velocity changes according to what’s going on in the script.
I didn’t want to remove the text entirely, this is why the posters has “footnotes”, anchoring individual lines to points on the path, giving a sense of time and duration to the image, what happened when.
What do you/audiences/performers/the RSC hope to learn from these visualisations (if anything)?
I’m not sure there’s necessarily a vast amount to be gained from them. I hope they raise a kind of awareness in audiences of the physicality of theatre, of the fact that a staged work is more than lines in a book. Future variants of this might be more useful as a tool. If you had automated capture, and recorded a whole run of a play (perhaps even the rehearsals), you could begin to see how a performance evolves, how it’s shaped by an audience and by time.
For now, though, I think it’s valuable just to encourage people to look sideways at the world from time to time: to understand the RSC’s work in a new way.
There are a lot of different skills at play here; did you face any challenges while you were working?
Technically, the most complex work was just about integrating the video into the mouse-recording application without making things too complex. I went into the project with a smattering of knowledge of ffmpeg; I now know quite a bit more!
But really, I think the biggest challenge was in the design of the artifacts: balancing the need for them to be beautiful with the need for them to be somewhat self-explanatory and clear. I’m not much of a graphic designer, but I’m very pleased with how things turned out.
9. Were you influenced by any other projects when you were designing this work?
No other projects, directly. I think I was more influenced by other artifacts: as I mention in the film, dance notation, blocking diagrams; soccer play-by-play illustrations in the newspaper, or American football playbooks. The influences were on the visuals, the desire to visualise movement, I guess.
What’s next for Spirits Melted into Air?
I don’t know! I’m really interested to see how people respond to it. There are very direct things you could do – repeating it for more work – but I’d probably do things slightly differently a second time around and the software needs a bit of an overhaul! The more interesting places to take it, which I discuss on the project website, are explorations of new materials and output formats, and automated capture. I think the shape of the work changes dramatically when the data’s no longer captured through tracing.
More broadly, though, I hope stands as an example for how technology collaborations with the arts can function. This was two weeks, but it led to an interesting exploration and outcome; shorter, sharper, tighter projects can be just as rewarding and relevant (if not more so) than sprawling ones. In that sense, I hope it’ll lead to future collborations, not just in the field of motion tracking, but much more generally. And I hope it’ll inspire other organisations to see what they could do alongside technologists.