Interactive Recipe in AR

Individual Senior Project

Project Overview

I did this project when I was an Integrated Design and Media senior at New York University.

This is an animated interactive recipe tutorial using augmented reality (AR). It is designed to entertain and facilitate people who live away from home, and would like to make their home-made food themselves. aimed to mitigate people's homesickness by providing interactive animated cooking tutorials using augmented reality.

Skills that I applied

  • 3D modeling, 3D animation, video editting.

  • Software I used were Autodesk Maya, Unity, Adobe CC.

  • AR wearable device- Microsoft HoloLens 2

Research

Backgroud Storytelling

The inspiration I've had for this project is the total failure I came across when I was trying to make dumplings on the lunar new year eve.

After reading from online people's posts, I found the common frustration and longingness that people away from home usually had when they missed their home food but were unable to replicate the dish themselves. Also, the tutorial online isn't always same as their family versions.

Design

A "family-taught" tutorial was helpful in all means. It helps users to recreate their home food, and with the interactivity, it gives an actual feeling that someone in the family is teaching them step by step. The first recipe in this series is the dumpling recipe from my mom.

Due to my passion about animated movies, I soon decided to use those techniques to build a useful, interactive and fun cooking tutorial series.

Implementation

After setting up the basic concept, I tested out the model and the animation constantly in AR environment in a software called Lens Studio.
When the animation file getting larger the limit of Lens Studio, I got introduced to the Microsoft HoloLens 2 by Professor Carla Gannis. That's where I self learnt the development of the gear with Unity game engine.

Professor Cannis trying on the glasses at the final showcase.

Presentation

The view users can see with HoloLens on, as well as the audio instruction of the recipe.

Real using scenario of the project. The laptop screen shows what the user is viewing synchronously.