Okaasan to Issho
Tech Stack

– Unity 3d
– Arduino
– Wamp Server

Deliverables

– Frontend Development
– Backend Development 
– Projection Mapping

Cosmic Lab

– Cosmic Lab
– Daiwa house

Year

2018

About

Okaasan to Issho (おかあさんといっしょ, With Mother) is a Japanese television series for children aired on NHK (the Japan Broadcasting Corporation). The show features seasonal songs, fun animated shorts, and the Galapico Poo puppets segment. Japanese building company Daiwa House partnered with NHK to create a fantastical event for kids to enjoy, all based on Okaasan to Issho.
My team was selected for the task of creating something interactive and memorable for kids. We decided to make an easy-to-use projection mapping activity that followed five main steps:
  1. The kids have their pictures taken by a tablet (we provide several booths for doing so).
  2. Each picture is sent to a server in the local network.
  3. The server arranges them in a first-in, first-out order.
  4. Our application assembles the pictures and projects them on a screen with dancing cartoon characters and other fun additions.
  5. Each child also receives a print of their photo

Challenges

For this project, we knew that since it was a live event surrounding a popular television series, there would be thousands of visitors coming through each day. In that sense, stress testing and performance testing were critical in avoiding possible crashes or system errors. 
As an experienced game developer, I was a little on edge during the event because I knew that if an error did occur, we’d need to address it at lightning speed so that people wouldn’t be left waiting. Thankfully, at the end of the event, our team held a sense of accomplishment for getting through without any unexpected errors.

Approach

To succeed in creating this project, we had two main tasks to achieve:
  1. Implementing UI and frontend using Unity.
  2. Communicating with external hardware (such as touch panels) using Arduino.
We expected a lot of participants at the event, so we knew that our approach had to be short and straightforward but still interesting enough to capture people’s attention. Since I had some expertise in projection mapping from my time working with Cosmic Lab, we decided to go in that direction using a giant screen and two wide-angle projectors that would be centered in the event space.
In an effort to keep the projection mapping fun and kid-friendly, we added cartoon characters around each child’s image and made them like they were dancing with the characters. We also left the projections open for interaction, adding areas that triggered unique character animations and sound effects when the kids touched them.
One of the main challenges was to get pictures of the kids and appear on the screen as they watched it. Our team of game developers used an Arduino microcontroller to quickly get the signals from the touch panel to the Unity 3D side. All of the frontend components, including relevant animations, were created in Unity. 
We also had to create a custom application for the tablets to capture the pictures of kids and send them to our local server. For the local server, we used the famous Wamp server. We also wanted to crop the images appropriately on the server-side, so we used the Image Magic framework for all of the image manipulation requirements.