Fengyuan Zhu's Research Space

📧 fyzhu@dgp.toronto.edu

Hello! I’m Fengyuan Zhu, a PhD candidate in Computer Science at the University of Toronto, focusing on spatial computing. I’m working with Prof. Tovi Grossman at the Dynamic Graphics Project Lab.

My doctoral research is motivated by a vision to transcend traditional 2D screen interactions and delve into the realm of three-dimensional space. This journey is twofold: firstly, I am exploring the integration of smart devices with VR and AR to enrich digital experiences. Secondly, I am focused on enhancing input precision in VR and AR environments to facilitate more natural and intuitive interactions.

The emergence of Large Models has recently shifted part of my research, and I am excited about the potential of combining sensory perception and content generation in three-dimensional spaces with LMs. Some of my works in this exciting new field will soon be published on arXiv and shared with you, so keep an eye out!

Before this, at NYU’s Interactive Telecommunications Program (ITP), I led the Holokit open-source project, achieving significant impact.

I hold a Bachelor’s in Physics from Peking University with a minor in Art, a background that drives my approach to human-computer interaction and creative projects.

selected publications

BISHARE: Exploring Bidirectional Interactions Between Smartphones and Head-Mounted Augmented Reality

Fengyuan Zhu, and Tovi Grossman

In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems , Honolulu, HI, USA, 2020

Abs PDF Video

In pursuit of a future where HMD devices can be used in tandem with smartphones and other smart devices, we present BISHARE, a design space of cross-device interactions between smartphones and ARHMDs. Our design space is unique in that it is bidirectional in nature, as it examines how both the HMD can be used to enhance smartphone tasks, and how the smartphone can be used to enhance HMD tasks. We then present an interactive prototype that enables cross-device interactions across the proposed design space. A 12-participant user study demonstrates the promise of the design space and provides insights, observations, and guidance for the future.
PhoneInVR: An Evaluation of Spatial Anchoring and Interaction Techniques for Smartphone Usage in Virtual Reality

Fengyuan Zhu, Mauricio Sousa , Ludwig Sidenmark , and Tovi Grossman

In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems , Honolulu, HI, USA, 2024

Abs PDF Video

When users wear a virtual reality (VR) headset, they lose access to their smartphone and accompanying apps. Past work has proposed smartphones as enhanced VR controllers, but little work has explored using existing smartphone apps and performing traditional smartphone interactions while in VR. In this paper, we consider three potential spatial anchorings for rendering smartphones in VR: On top of a tracked physical smartphone which the user holds (Phone-locked), on top of the users empty hand, as if holding a virtual smartphone (Hand-locked), or in a static position in front of the user (World-locked). We conducted a comparative study of target acquisition, swiping, and scrolling tasks across these anchorings using direct Touch or above-the-surface Pinch. Our findings indicate that physically holding a smartphone with Touch improves accuracy and speed for all tasks, and Pinch performed better with virtual smartphones. These findings provide a valuable foundation to enable smartphones in VR.
PinchLens: Applying Spatial Magnification and Adaptive Control-Display Gain for Precise Selection in Virtual Reality

Fengyuan Zhu, Ludwig Sidenmark , Mauricio Sousa , and Tovi Grossman

In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) , Sydney, New South Wales, Australia, 2023

Abs PDF Video

We present PinchLens, a new free-hand target selection technique for acquiring small and dense targets in Virtual Reality. Traditional pinch-based selection does not allow people to precisely manipulate small and dense objects effectively due to tracking and perceptual inaccuracies. Our approach combines spatial magnification, an adaptive control-display gain, and visual feedback to improve selection accuracy. When a user starts the pinching selection process, a magnifying bubble expands the scale of nearby targets, an adaptive control-to-display ratio is applied to the user‘s hand for precision, and a cursor is displayed at the estimated pinch point for enhanced visual feedback. We performed a user study to compare our technique to traditional pinch selection and several variations to isolate the impact of each of the technique’s features. The results showed that PinchLens significantly outperformed traditional pinch selection, reducing error rates from 18.9% to 1.9%. Furthermore, we found that magnification was the dominant feature to produce this improvement, while the adaptive control-display gain and visual cursor of pinch were also helpful in several conditions.