Upperquad - Projection Mapping On Canvas For An Open House Event

This project was done by Anya Tran, who is a developer for Upperquad, an independent creative agency. Back in June 2018, they held a happy hour at the UQ office for SF Design Week 2018. After a year of remodeling and slowly making their new office home, they were excited to invite their friends and community to come celebrate their new space. It also happened to be the perfect occasion for some creative expression in the form of an interactive projection piece.

The shape and pattern artwork they've been developing over the last few months was a key starting point for this project’s visuals. A matrix of shapes and cells quickly emerged, giving them a platform to show off their colors and setting the stage for all kinds of animation opportunities.

Randomizing their colors and shapes was an awesome way to give it an always-fresh, never-quite-the-same feeling. Scale changes and explosions added another layer of personality to create a more dynamic and engaging experience.

The physical, interactive part of the experiment took a lot of research and logistical planning. First, they had to figure out where to place their installation. Inspired by their conference room’s existing artwork, they decided on a triptych approach, which allowed them to place canvases where their paintings once hung.

Once they had their placement, they started construction. The first question was what to use as a base. While they initially thought photo frames might work, they were worried about how sturdy they’d be with all the touching and tapping they expected. Canvases, on the other hand, gave them everything they needed. They were easy to project onto and were much less fragile, which was a nice reassurance given that they’d make their debut at a happy hour.

In order to turn the canvas into a sensor, they needed to put something conductive on the back of it and they opted for aluminium foil. Each canvas was divided into 4 separated sections and each section was connected to an electrode on a Touch Board. This allowed each section to serve as an individual sensor, giving them a total of 12 sensors.

At the heart of their project is the Touch Board, which can detect when a surface is touched, allowing you to create interactive pieces. On the visual side of things, they used the creative coding-oriented JavaScript library p5.js to create the colorful graphics that would be touch-triggered. In order to connect the touch data they got from the Arduino board to their p5.js sketch, they used p5.serialport and p5.serialserver. Finally, the p5.sound library was used for handling audio files and playback.

Their piece cycled between two different states: a grid-based state with shapes and a more free-form state featuring diagonal lines. Everything was randomly generated within those two systems, from color and shape selections to X and Y positions. To keep the visuals cohesive across their triptych of canvases, they chose to forgo using three different <canvas> elements, instead opting for just one large <canvas> element which they split into three sections using two black <div> columns on top.

They knew audio would play a big role in bringing this installation to life. They wanted to know what sound it would make when all these little shapes jump from canvas to canvas. What does this rainbow waterfall of color sound like? They went looking for an ambient sound that could worm around ones head when experiencing the space; a synthetic element that could actually convey the sounds that the animation might make.

This was not a task for the faint of heart, so they reached out to Eric Dietrick, a talented musician and tinkerer, to help them create some samples. They sent him recordings of the animations and within a few days they received a ton of amazing plug and play samples.

Working with a new medium with some pretty DIY variables, they naturally ran into a few obstacles. The biggest challenge for them was to make sure their sensors sent accurate data to the p5 program. There were times when a single touch would trigger multiple sensors at once because the sensitivity threshold was too low. Other times, the sensors weren’t sensitive enough so they weren’t receiving any signals; generally this was because the foil wasn’t glued properly to the canvas, or the conductive copper wire came loose from the Touch Board. Trial and error and using the Grapher tool helped a lot.

Through the course of this experiment, they learned a lot and had a ton of fun exploring the world of hardware and AV. It was fun to share it with their friends and the design community of SF.

This article first appeared on Medium.