Publications
As a Research Scientist at Google, I devote to inventing future interactive technologies in 3D graphics, fusing the information from the physical and virtual worlds, and making it interactive, accessible, and useful in VR, AR, and MR. I have published over 25 peer-reviewed publications in top venues of HCI, Computer Graphics, and Computer Vision, including CHI, SIGGRAPH Asia, UIST, TVCG, CVPR, ICCV, ECCV, ISMAR, VR, I3D, Web3D, etc. Please feel free to search keywords / authors / journal / conference below or visit my Google Scholar for more details.
Peer-reviewed Publications [bibTeX]
DepthLab: Real-Time 3D Interaction With Depth Maps for Mobile Augmented Reality
Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (UIST), 2020.
Keywords: depth map; interactive 3D graphics; real time; interaction; augmented reality; mobile AR; rendering; GPU; ARCore
Geollery: A Mixed Reality Social Media Platform
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI), 2019.
Keywords: metaverse, virtual reality, augmented reality, social media, GIS, street view, visualization, 3D user interface, 3D reconstruction, digital twins, mirrored world
Montage4D: Real-Time Seamless Fusion and Stylization of Multiview Video Textures
Journal of Computer Graphics Techniques (JCGT), 2019.
Keywords: texture montage, 3d reconstruction, texture stitching, view-dependent rendering, discrete geodesics, projective texture mapping, differential geometry, temporal texture fields
Social Street View: Blending Immersive Street Views With Geo-Tagged Social Media
Proceedings of the 21st International Conference on Web3D Technology (Web3D), 2016.
Keywords: metaverse, spatial-temporal virtual reality; social media; street view; geographical information systems; mixed reality; WebGL; digital twins
Fusing Multimedia Data Into Dynamic Virtual Environments
Ph.D. Dissertation, Computer Science Department., University of Maryland, College Park., 2018.
Keywords: social street view, geollery, spherical harmonics, 360 video, multiview video, montage4d, haptics, cryptography, metaverse, mirrored world
RetroSphere: Self-Contained Passive 3D Controller Tracking for Augmented Reality
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT), 2022.
Keywords: Retroreflectors, Augmented reality, Virtual reality, Infrared marker tracking, Augmented reality glasses
“Slurp” Revisited: Using Software Reconstruction to Reflect on Spatial Interactivity and Locative Media
Proceedings of the Designing Interactive Systems Conference (DIS), 2022.
Keywords: system re-presencing, affordances, metaphor, software reconstruction, historical precedents, gestural interface, augmented reality, spatial interaction
ProtoSound: A Personalized and Scalable Sound Recognition System for Deaf and Hard of Hearing Users
Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems (CHI), 2022.
Keywords: accessibility, deaf, Deaf, hard of hearing, sound awareness
Opportunistic Interfaces for Augmented Reality: Transforming Everyday Objects Into Tangible 6DoF Interfaces Using Ad Hoc UI
Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI), 2022.
Keywords: augmented reality, everyday objects, tangible user interface, 3D user interface, 6 DoF, spatial interaction, markerless tracking, tangible interaction, hand gestures
OmniSyn: Intermediate View Synthesis Between Wide-Baseline Panoramas
2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2022.
Keywords: 360 image, virtual reality, view synthesis, panorama, neural rendering, depth map, mesh rendering, inpainting
GazeChat: Enhancing Virtual Conferences With Gaze-Aware 3D Photos
Proceedings of the 34th Annual ACM Symposium on User Interface Software and Technology (UIST), 2021.
Keywords: eye contact, gaze awareness, video conferencing, video-mediated communication, gaze interaction, augmented communication, augmented conversation
HumanGPS: Geodesic PreServing Feature for Dense Human Correspondence
2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021.
Keywords: correspondences, geodesic distance, embeddings, neural networks
A Log-Rectilinear Transformation for Foveated 360-Degree Video Streaming
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2021.
Keywords: 360° video, foveation, virtual reality, live video stream-ing, log-rectilinear, summed-area table
Saliency Computation for Virtual Cinematography in 360° Videos
IEEE Computer Graphics and Applications (CGA), 2021.
Keywords: spherical harmonics, virtual reality, visual saliency, 360°videos, omnidirectional videos, perception, Itti model, spectralresidual, GPGPU, CUDA
CollaboVR: A Reconfigurable Framework for Multi-User to Communicate in Virtual Reality
2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2020.
Keywords: chalktalk, virtual reality, collaborative work, layout, telepresence, communication
Experiencing Real-Time 3D Interaction With Depth Maps for Mobile Augmented Reality in DepthLab
Adjunct Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (UIST), 2020.
Keywords: depth map; interactive 3D graphics; real time; interaction; augmented reality; mobile AR; rendering; GPU; ARCore
MeteoVis: Visualizing Meteorological Events in Virtual Reality
Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems (CHI EA), 2020.
Keywords: scientific visualization, virtual reality, meteorological data, immersion, interactive visualization, vector field
Eye-Dominance-Guided Foveated Rendering
IEEE Transactions on Visualization and Computer Graphics (TVCG, Special Issue of IEEE Conference on Virtual Reality and 3D User Interfaces), 2020.
Keywords: virtual reality, foveated rendering, perception, gaze-contingent rendering, ocular dominance, eye tracking
Language-Based Colorization of Scene Sketches
ACM Transactions on Graphics (SIGGRAPH Asia), 2019.
Keywords: deep neural networks; image segmentation; language-based editing; scene sketch; sketch colorization
ORC Layout: Adaptive GUI Layout With OR-Constraints
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI), 2019.
Keywords: GUI builder, layout manager, constraint-based layout, visual interface design, visual programming
Kernel Foveated Rendering
Proceedings of the ACM on Computer Graphics and Interactive Techniques (I3D), 2018.
Keywords: foveated rendering, perception, log-polar mapping, eye-tracking, virtual reality, head-mounted displays
Project Geollery.com: Reconstructing a Live Mirrored World With Geotagged Social Media
Proceedings of the 24th International Conference on Web3D Technology (Web3D), 2019.
Keywords: virtual reality, mixed reality, 360° image, GIS, 3D reconstruction, projection mapping, mirrored world, social media, WebGL, metaverse, mirrored world
Montage4D: Interactive Seamless Fusion of Multiview Video Textures
Proceedings of ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D), 2018.
Keywords: texture montage, 3d reconstruction, texture stitching, view-dependent rendering, discrete geodesics, projective texture mapping, differential geometry, temporal texture fields
Evaluating Haptic and Auditory Directional Guidance to Assist Blind People in Reading Printed Text Using Finger-Mounted Cameras
ACM Transactions on Accessible Computing (TACCESS), 2016.
Keywords: accessibility, real-time OCR, visual impairments, wearables
Video Fields: Fusing Multiple Surveillance Videos Into a Dynamic Virtual Environment
Proceedings of the 21st International Conference on Web3D Technology (Web3D), 2016.
Keywords: virtual reality; mixed-reality; video-based rendering; projection mapping; surveillance video; WebGL; WebVR
Experiencing a Mirrored World With Geotagged Social Media in Geollery
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA), 2019.
Keywords: virtual reality, augmented reality, social media, GIS, street view, visualization, 3D user interface, 3D reconstruction, metaverse, mirrored world
Interactive Fusion of 360° Images for a Mirrored World
2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2019.
Keywords: virtual reality, 360° image, 3D reconstruction, mixed reality, projection mapping, mirrored world, metaverse, mirrored world
VRSurus: Enhancing Interactivity and Tangibility of Puppets in Virtual Reality
Proceedings of the 34th Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA), 2016.
Keywords: Virtual Reality; Tangible User Interface; Haptics; Gesture Recognition; Head-Mounted Display
AtmoSPHERE: Representing Space and Movement Using Sand Traces in an Interactive Zen Garden
Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA), 2015.
Keywords: Visualization; Tangible Interactive Art; Machine Aesthetics; Calm Technology; XY Servo Table; Kinect
The Design and Preliminary Evaluation of a Finger-Mounted Camera and Feedback System to Enable Reading of Printed Text for the Blind
Computer Vision - ECCV 2014 Workshops (ECCVW), 2014.
Keywords: Accessibility, Wearables, Real-time OCR, Text Reading for Blind
Supporting Everyday Activities for Persons With Visual Impairments Through Computer Vision
Proceedings of the 17th International ACM SIGACCESS Conference on Computers Accessibility (ASSETS), 2015.
Keywords: Blind; visually impaired; wearable computing; computer vision; vision-augmented touch
Online Vigilance Analysis Combining Video and Electrooculography Features
Neural Information Processing - 19th International Conference (ICONIP), 2012.
Keywords: Vigilance Analysis Fatigue Detection Active Shape Model Electrooculography Support Vector Machine
Technical Reports
Rapsai: Accelerating Machine Learning Prototyping of Multimedia Applications Through Visual Programming
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI), 2023.
Keywords: visual programming, node-graph editor, deep neural networks, data augmentation, deep learning, model comparison, visual analytics
Visual Captions: Augmenting Verbal Communication With On-the-Fly Visuals
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI), 2023.
Keywords: augmented communication, large language models, video-mediated communication, online meeting, collaborative work, augmented reality
ThingShare: Ad-Hoc Digital Copies of Physical Objects for Sharing Things in Video Meetings
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI), 2023.
Keywords: video-mediated communication, object-centered meetings, online meeting, collaborative work, augmented communication
C-Flow: Visualizing Foot Traffic and Profit Data to Make Informative Decisions
University of Maryland, College Park. Department of Computer Science, 2012.
Keywords: Information Visualization; Data Mapping; Indoor Visualization; Business; Usability Testing
