Cabinet of Wonder is a projection mapping project. Using Mad-mapper and physical computation creates a site-specific installation to construct new experiences. Users can enter the magical world brought by the projection by interacting with the cabinets that can be seen everywhere. This project was also shortlisted for the BAFTA student award immersive.

Relevant Tools
Madmapper, Arduino, After Effect, Python

My role
Programmer, Media Producer

Teammates
Zeshu Zhu, Lifei Wang

Ideation

For the drawer projection project, me and Lifei are thinking of creating a projection of a theater inside the drawer. We got this vintage-looking nightstand and planning to do projection from the outside to the inside of it. I imagine the project to be interactive as the audience pulls out the drawe

Inspired by the toy theater in Benjamin Pollock’s toy shop, we are planning to map the outside of the drawer into the outside of a theater, the first drawer to be the interior of a theater, and the second drawer to be the backstage of a theater.

Incorporate with a sensor and Arduino, ideally, we can match the opening process in a theater with the opening of the drawer.  

Considered that the drawers have a 90 degree angled wall on four sides which can make the projection become really difficult. We decide to build an additional layer inside the drawer that softens this angle for a better projection effect. Lifei modeled this out in rhino and laser cut out the frame. drawer that softens this angle for a better projection effect.

For the projection, we are thinking of using the pico projector. Lifei tested both the pico projector and the short throw projector, but she feels the pico projector is enough for projecting the drawer. The projector could be hang on the ceiling or put on a tripod.

Content

Next, we started working on the content in the cabinet, we made the themed theater, and through this cabinet, Lifei and I wanted to show users the different content generated by people and machines. So we decided that Lifei would create an animated short film for the upper cabinet, and I would use GANs to generate a completely computer-generated short film for the lower cabinet.

Animated short film by Lifei

Video I generated by GANs

I will explain here the part of GANs that I use. The model I used is pytti 5 beta, and the reason for using it is based on iterations of text to video and image to video GANs. It allows me to submit more samples than the previous two, thus generating an output that is more in line with my expectations.

I made it 60,100steps per scene and set its iterative rotation to (50+3*t)*sin(t/10*pi)**2 to give a less dizzying but deeper feel to provide it an inhaling effect. After 17 hours of computation, the GANs generated 1702 images, which I combined into 30 frames per second video.

Interaction

Since we are using a drawer, interaction will be necessary, and we are very excited about using Mad-Mapper’s new function and Arduino-linked features. So we decided to try to combine projection mapping with Arduino.

I installed four force sensors on the bottom of the drawer, two for each side, and use the reading of the sensors to change the screen inside Madmapper. Using the Firmata Arduino module to control the Madmapper projection. We also installed two tracks underneath the drawers to make sure that the drawer can be pulled out smoothly.

In mad mapper, I tried different values of pressure resistors and set them to trigger separate scenarios to pull out the drawer and close it. Arduino I used the Standard Firmata example, and the final result was as we expected.

Setup

Since we initially wanted to highlight the content, we used the InFocus IN2116 DLP Projector with very high brightness and resolution. After our initial setup, we felt that the quality of the projection was very high, but the disadvantage was also very obvious, that we sacrificed the experience and immersion of the piece for the high resolution. It’s hard for the user to feel like this is a cabinet that goes everywhere. Instead, the conspicuous projector and tripod make it more of an unfinished piece in production.

Since the cabinet and tripod cannot be fixed, the user must be very careful to open and close the cabinet; otherwise, the projection content will be very easily problematic. And when there is a problem, we will spend a very long time resetting and fixing it, so the result is not a mature work, so we decided to reset up completely.

Errors when setup

To start over completely, we decided to ditch the decision to have a practical tripod and instead use the MAGIC ARM to hang the projector from the ceiling. While this sacrifices brightness and resolution, we get an immersive experience for the user and can make our work more stable and less prone to errors.

As the projection area increases, we can add more projection content. For example, when the user is not interactive, there will be lights to guide the user where they can interact, and we added dust effects to make the overall sense of the atmosphere more intense.

And we reworked the content, the final result is very good. The sacrificed resolution did not become a disadvantage, but rather the brightness became less harsh.