The Water Footprint

What can we do about it?

Inspired by Charity Water (who's mission is to bring clean and safe drinking water to people in developing countries), we decided to spread awareness by creating an immersive experience through projection mapping.

We asked ourselves, "How do we create an immersive experience where users can physically interact with and learn about the implications of their water footprint?"

We decided to create an immersive experience using projection mapping.

No one in the group had experience with projection mapping, so we spent the initial phase doing research to prepare us for the technical portion. We compared several projectors and projection mapping software systems to find the best combination for our time restraint and feasibility.

Inspiration

Before diving into sketching we looked out into the vast creator communities who posted their projects on YouTube, Vimeo, and other niche forums to find some inspiration and see what was actually possible for a 3 month project.

A couple of our favorites were:

Firewall: Uses a membrane interface that is sensitive to depth. It integrated music as well into the experience.

Willow - Sweater: A music video that uses storytelling through projection mapping. The scene changes and the user interacts with the changes (yes he wears a cozy sweater).

Ideation

We found inspiration in projects that used a projector from behind spandex to create an interactive experience. We also played around with the idea of creating a performance where viewers could just watch. The idea of creating a game also came up into conversation. However, all of these ideas were either going to cost too much time or money.

We also considered the types of surfaces that we would either interact with or project onto. Rocks, sand, walls, boxes, sheets, tables, the floor, and even a pool were considered. With a short amount of time to complete the project, we experimented with more practical surfaces such as walls, floors, and tables.

Idea #1

Initially wanted to build an interactive experience where the user would press on a physical object that represented an item of food, the Kinect would recognize the item, and then project a fact about the item on a wall. We went back and forth on whether the item should be projected or physical for the user to interact with. However, we ran into a roadblock when attempting to program this idea. In order for this to have worked we would have had to use machine learning, which takes a lot of time and was outside of the scope of the project. At that point it became too technical and time consuming so we rethought our initial idea and created a new one that was purely projection mapping.

Idea #2

Our next idea was to create an environment where the Kinect would track a specific position of where a person was standing (which would represent an item on the floor). According to the specific position a series of filled boxes would project onto a wall with the number of gallons of water it took to produce the item. However, we then discovered that there would be a simpler idea.

In the end we chose an idea where the user would stand in one location and raise their arms up and down. The environment that was projected showed various food items at increasing heights correlating to the amount of water it took to produce the item. As the user would raise their arms, the item in the water would change as the sea level rose to signify how much more water it took to produce the item.

We chose this idea to implement.

Research

Moving ahead with idea #2, our next step was to investigate how much water it takes to produce different types of food. We decided to have three distinct categories so when the user is raising their arms they can clearly see the increase and decrease of water usage based on the image of the food.

Ultimately, we chose to showcase butter (46 gallons), a potato (66 gallons), pizza (317 gallons) and steak (1,232 gallons).

Hardware

We compared three different projectors: the Xbox Kinect, the BENQ TH6715T and the BENQ W10805T.

Based on accessibility, common integration to projection mapping software, and accuracy we decided to go with the Xbox Kinect.

Software

We compared three projection mapping software: Isadora, MadMapper and TouchDesigner to determine which would be the most effective and cost efficient for what we wanted to create. The developers had both Mac and PC laptops so it was important that the software be cross-compatible. Ultimately we chose Isadora because we already had full access to the software and it had fewer limitations in cost (because we already had access to 2 keys) and performance when compared to the others.

Building the experience

Kinect

Getting the Kinect and Isadora running and talking to each other was a challenge… that we accomplished!

Processing has different versions that will not run with certain hardware from the Kinect. For it to work, we had to figure out which libraries to download and which version of Processing to use. However, once we did, we had finished our first true technical hurdle!

Once they were able to communicate, we had control over specific points on the body but we needed to figure out how to recognize and analyze only hand movement.

Vital breakthrough

For our idea to work, we had to get the Kinect to track the participants hand as it goes up and down and to translate the location of the wave video within Isadora. The Kinect tracks the X, Y and Z coordinates of each key point on the body in the frame of view. To get the wave to move up or down according to how high or low the hands are, only the Y coordinate of the hand is needed. Initially, we simply connected the Y coordinate for the left hand to the vertical position of a square on the stage. After quickly discovering the discrepancy, we used the scaling feature that allows us to scale the Y coordinate of the Kinect to the vertical position on the stage in Isadora. Through trial and error we got the correct numbers for the scaling and accurate tracking!

Animation

Not having a graphic design background was definitely a challenge. My first attempt at animating waves was pretty rough. They weren’t the most aesthetically pleasing set of waves to look at and the motion didn’t achieve the feel that we wanted. So, I went back and looked up how to layer colors and watched countless videos of real and animated waves in motion. With this research I was able to transform our wave animation into a more effective and believable design.

Testing

Throughout the entire process, we were testing and iterating over and over again. Whether it was lines of code, assets, or animations. We asked questions like, “How often is there a user error?”, “How easy is it to pick up?”, “Does the core message get across or is the experience too difficult?” and “Is this enjoyable?”.

The day came for our presentation and this was (arguably) the best part — user testing! After months of hard work we had a presentation where members of the academic community could come in, watch a video about the water footprint (to get some context) and then dive right into the projection mapping experience.

What we learned

• Coffee is essential for all-nighters

• Creating something from scratch (with no prior background) is rewarding and quite challenging

• Users loved our message and really enjoyed the experience overall!