Future Mapper
Week 1: Inspiration from Moma and Mood board
I went to MoMa this week and watched Liquid Reality by Shigeko Kubota. I chose it because it’s a physical device made of circuits and metal, but it’s combined with a liquid. I think this installation not only covers a combination of physical installations and virtual art, but it also has a lot of using of lighting. This is very consistent with the features of ITP and future mapper.
Shigeko developed a unique form of video sculpture that extended her otherworldly portraits and landscapes into three-dimensional forms made from plywood and sheet metal, often incorporating mirrors and flowing water. I think I could use some of the conceptions from Shigeko’s work. I can map virtual graphics or animations to real objects and combine the two to give the audience a more immersive experience also help the audience better understand what I want to express.
Shigeko uses visible displays and liquid sheds light on how these sculptures—which draw parallels between nature, technology, and time—continue to resonate in today’s digitally interconnected world. Viewers can observe the changes in light and media brought about by the liquid flow to present her video’s reality. Infinite variation becomes possible…freedom to dissolve, reconstruct, mutate all forms, shape, color, location, speed, scale…liquid reality.
Week 2: Start my mask projection mapping installation
Rather than a large installation, I hope to start with a mask projection mapping installation of small objects. I hope it will be some installation that combines natural objects and virtual artwork, and there is a strong connection between the two, such as projecting an ocean in a water glass would be a cool thing.
Things I want to do so far is to project some content that can change the shape of the materials. I’m spending lots of time thinking about why I should use the project mapping to the shape. Then I got the solution: I can use the projector to do something that won’t happen in real work, like changing the shape looks like magic.
And the way to achieve my dream project so far is to use 3 projectors or 1 to map three spheres of the cube. And if I have time, I can add some physical interaction like a force sensor or ultrasonic sensor to allow users to interact with the cube.
Week 3: Iterating on ideas for my dream project
After conception and going to different museums, Lifei and I decided to do a project that projects mapping onto physical objects. It will use Arduino as the essence interaction method. The specific projection mode is similar to the following video:
We planed to combine mapping and Arduino to create an interactive installation like the video above. But unlike the video, we won’t use unity or unreal as our core software, but mad mapper. And we intended to present our project similarly to Cabinet of Curiousness, as shown in the video above. We plan to map the project to the cabinet to show the cabinet with magical blushes. Various media will be presented to users as the user opens different drawers.
And this projection idea also comes from the work of our guest lecturer Motomichi. We got the idea of projecting projections onto objects and started thinking about why we wanted to project these onto a particular thing and whether it made more sense for both parties. With a professional perspective and experience, Motomichi made us think further about the following works and let us catch a lot of projects from him, which broadened our horizons.
Week 4-7: FInal Project: Cabinet of Wonder
Ideation
For the drawer projection project, me and Lifei are thinking of creating a projection of a theater inside the drawer. We got this vintage-looking nightstand and planning to do projection from the outside to the inside of it. I imagine the project to be interactive as the audience pulls out the drawe
Inspired by the toy theater in Benjamin Pollock’s toy shop, we are planning to map the outside of the drawer into the outside of a theater, the first drawer to be the interior of a theater, and the second drawer to be the backstage of a theater.
Incorporate with a sensor and Arduino, ideally, we can match the opening process in a theater with the opening of the drawer.
Considered that the drawers have a 90 degree angled wall on four sides which can make the projection become really difficult. We decide to build an additional layer inside the drawer that softens this angle for a better projection effect. Lifei modeled this out in rhino and laser cut out the frame. drawer that softens this angle for a better projection effect.
For the projection, we are thinking of using the pico projector. Lifei tested both the pico projector and the short throw projector, but she feels the pico projector is enough for projecting the drawer. The projector could be hang on the ceiling or put on a tripod.
Content
Next, we started working on the content in the cabinet, we made the themed theater, and through this cabinet, Lifei and I wanted to show users the different content generated by people and machines. So we decided that Lifei would create an animated short film for the upper cabinet, and I would use GANs to generate a completely computer-generated short film for the lower cabinet.
Animated short film by Lifei
Video I generated by GANs
I will explain here the part of GANs that I use. The model I used is pytti 5 beta, and the reason for using it is based on iterations of text to video and image to video GANs. It allows me to submit more samples than the previous two, thus generating an output that is more in line with my expectations.
I made it 60,100steps per scene and set its iterative rotation to (50+3*t)*sin(t/10*pi)**2 to give a less dizzying but deeper feel to provide it an inhaling effect. After 17 hours of computation, the GANs generated 1702 images, which I combined into 30 frames per second video.
Interaction
Since we are using a drawer, interaction will be necessary, and we are very excited about using Mad-Mapper’s new function and Arduino-linked features. So we decided to try to combine projection mapping with Arduino.
I installed four force sensors on the bottom of the drawer, two for each side, and use the reading of the sensors to change the screen inside Madmapper. Using the Firmata Arduino module to control the Madmapper projection. We also installed two tracks underneath the drawers to make sure that the drawer can be pulled out smoothly.
In mad mapper, I tried different values of pressure resistors and set them to trigger separate scenarios to pull out the drawer and close it. Arduino I used the Standard Firmata example, and the final result was as we expected.
Setup
Since we initially wanted to highlight the content, we used the InFocus IN2116 DLP Projector with very high brightness and resolution. After our initial setup, we felt that the quality of the projection was very high, but the disadvantage was also very obvious, that we sacrificed the experience and immersion of the piece for the high resolution. It’s hard for the user to feel like this is a cabinet that goes everywhere. Instead, the conspicuous projector and tripod make it more of an unfinished piece in production.
And because the cabinet and tripod cannot be fixed, the user must be very careful to open and close the cabinet; otherwise, the projection content will be very easily problematic. And when there is a problem, we will spend a very long time resetting and fixing it, so the result is not a mature work, so we decided to reset up completely.
Errors when setup
To start over completely, we decided to ditch the decision to have a practical tripod and instead use the MAGIC ARM to hang the projector from the ceiling. While this sacrifices brightness and resolution, we get an immersive experience for the user and can make our work more stable and less prone to errors.
As the projection area increases, we can add more projection content. For example, when the user is not interactive, there will be lights to guide the user where they can interact, and we added dust effects to make the overall sense of the atmosphere more intense.
And we reworked the content, the final result is very good. The sacrificed resolution did not become a disadvantage, but rather the brightness became less harsh.