PROGRAMMING
Secret Passageways with Avatar Faces
Term: Spring 2018, Rochester Institute of Technology
Project Description
Inspired by these secret corridors, this exhibit presents virtual passageways connecting various locations on campus and allows visitors to talk with others via specially created avatar faces. Using facial motion capture technology, facial expressions of a user are captured and are used to control these avatar faces, which are, in turn, viewed by others in the Golisano College of Computing and Information Sciences (GCCIS), the College of Imaging Arts and Sciences (CIAS), and the MAGIC center.
My Role: Technical Artist, Programmer and 3D Modeling Artist
My primary work involved reworking existing Facial models, originally developed by CIAS department, to suit the requirement of Faceware LIVE that included creating the required blend shapes in Maya and interfacing the 3D model with Faceware LIVE in Unity. All my work has been extensively detailed in the report above, available for download. Lastly, I was responsible for setting up the final project and exhibiting the same at ImagineRIT 2018.
Technical: Unity3D, Autodesk Maya, Faceware LIVE, Photon Voice Unity Asset.
Showcase
Exhibited at ImagineRIT 2018 and several Open Houses for incoming students at RIT in 2018
Virtual Karaoke
Term: Summer 2018, Rochester Institute of Technology
Project Description
Karaoke with a twist! You sing in the real world while a virtual puppet – which you control using facial motion capture – performs the song on a virtual stage. Fun for all budding pop stars not yet ready to show their real faces to the world.
My Role: Technical Artist and Programmer
An extension of Secret Passageways, my primary work was to develop the menu for the user and integrate it with Faceware LIVE. I have developed the User Interface in C# that allows the user to select the character and scene. Depending on the choices, the desired environment is provided to the user, by dynamically loading the respective prefabs and panel image. The Facial Expressions of the user are captured and mapped onto the character using Faceware LIVE thus providing real-time animation for live performance. The main learnings of this project involved C# scripting in Unity3D.
Technical: Unity3D, Faceware LIVE, Photon Voice Asset in Unity3D
Showcase
Exhibited at Rochester Fringe Festival 2018.
Unabridged Emotions
Term: Fall 2018, Rochester Institute of Technology
Project Description
Unabridged Emotions is a mixed reality dance performance that explores the expression of basic emotions through poetry, dance and facial expression. During the piece, facial avatars, controlled by users off-stage, are projected on stage while each dancer portrays a particular basic emotion. Users controlling the faces experience the performance from stage view via the use of networked cameras connected to Raspberry Pis. A 10-minute preview of the piece will be performed.
My Role: Technical Artist and 3D Modeling Artist
My primary work as a Research Assistant familiar with working with 3D Face models was to ensure that the 3D Face models of the dancers, developed by CIAS department fulfill the requirements of Faceware LIVE client in Unity and function accordingly when interfaced in Unity.
Technical: Autodesk Maya, Unity3D and Faceware LIVE.
Showcase
Exhibited at 3rd Annual Frameless Symposium at RIT.
Still Life with GL
Term: Spring 2017, Rochester Institute of Technology
Project Description
Imitated a still life scene using OpenGL in C++. This required implementing the 2D graphics pipeline that includes Shading, Lighting and Texture Mapping and reads the object file formats of the 3D models. Modeled the objects for this scene in Blender.
Language: C++ and OpenGL
Euclidean Shortest Path Problem
Term: Fall 2017, Rochester Institute of Technology
Project Description
Implemented a solution in Java to solve the Euclidean Shortest Path Problem. The problem space contained a set of polyhedral obstacles in euclidean space and a starting point and an ending point. JAVA Swing was used to developing the User Interface that aids the user to draw his obstacles, plot the starting point and destination point and shows the visualization of the traced shortest path.
My Role
I was responsible for creating the User Interface using Java Swing that takes in the obstacles and starting and destination point and gives the collection of points to the next steps. After deriving the shortest path points, I was responsible for retrieving the same to visualize the shortest path as the end result.
Please see the detailed report at
For code samples please see
1. PointO. java
2. Vertex.java
3. Obstacle.java
4. learnSwing.java
5. Graph.java
6. Dijkstras.java
Language: Java
Rendering Lights and Shadows in Augmented Reality
Term: Spring 2018, Rochester Institute of Technology
Project Description
Experiment with Lighting and Shadows in Augmented Reality world using Unity to achieve realistic lighting that provides a vivid visual. Since my focus is on light and shadows, I incorporated a Disco light concept.
My Role: Programmer and 3D Modeling Artist
I used Unity and Vuforia Augmented Reality SDK which is an Augmented Reality Software Development Kit for mobile devices that enables the creation of Augmented Reality Applications. I used a shader that was a Vertex Fragment Shader that receives the shadows and renders them in real time. I created the models in Maya and imported them into Unity. I have also added flickering lights functionality for every 0.2 seconds after setting up the lights in the scene accordingly. The object is placed on the Image Target. There is a transparent Mesh underneath the models that is mapped with the shader and we made the mesh transparent so that it looks like the shadows are falling on the real world and not on any mesh. I added a sound clip to the AR Camera so that it plays when the scene is played.
Technical: Autodesk Maya, Unity3D, Vuforia
