We present Geollery, an interactive mixed reality social media platform for creating, sharing, and exploring geotagged information. Geollery introduces a real-time pipeline to progressively render an interactive mirrored world with three-dimensional (3D) buildings, internal user-generated content, and external geotagged social media. This mirrored world allows users to see, chat, and collaborate with remote participants with the same spatial context in an immersive virtual environment. We describe the system architecture of Geollery, its key interactive capabilities, and our design decisions. Finally, we conduct a user study with 20 participants to qualitatively compare Geollery with another social media system, Social Street View. Based on the participants' responses, we discuss the benefits and drawbacks of each system and derive key insights for designing an interactive mirrored world with geotagged social media. User feedback from our study reveals several use cases for Geollery including travel planning, virtual meetings, and family gathering.
VIDEO
Publications Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI), 2019.
Keywords: metaverse, virtual reality, augmented reality, social media, GIS, street view, visualization, 3D user interface, 3D reconstruction, digital twins, mirrored world; digital world; digital world; augmented communication
Proceedings of the 21st International Conference on Web3D Technology (Web3D), 2016.
Keywords: metaverse, spatial-temporal virtual reality; social media; street view; geographical information systems; mixed reality; WebGL; digital twins; digital world
Ruofei Du
Ph.D. Dissertation, Computer Science Department., University of Maryland, College Park., 2018.
Keywords: social street view, geollery, spherical harmonics, 360 video, multiview video, montage4d, haptics, cryptography, metaverse, mirrored world
Proceedings of the 24th International Conference on Web3D Technology (Web3D), 2019.
Keywords: virtual reality, mixed reality, 360° image, GIS, 3D reconstruction, projection mapping, mirrored world, social media, WebGL, metaverse, mirrored world, interactive graphics, digital world
Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA), 2019.
Keywords: virtual reality, augmented reality, social media, GIS, street view, visualization, 3D user interface, 3D reconstruction, metaverse, mirrored world
2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2019.
Keywords: virtual reality, 360° image, 3D reconstruction, mixed reality, projection mapping, mirrored world, metaverse, mirrored world
Web3D 2019, 2018.
Keywords:
US Patent 11,087,553, 2019.
Keywords:
US Patent 10,380,726, 2016.
Keywords: virtual reality; social media; street view; geographical information systems; mixed reality; WebGL
Cited By A Systematic Literature Review of Mixed Reality Environments in K-12 Education Education and Information Technologies. Nikolaos Pellas , Ioannis Kazanidis , and George Palaigeorgiou . source | cite | search Digital Twin As a Mixed Reality Platform for Art Exhibition Curation 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). Inhwa Yeom and Woontack Woo . source | cite | search GAVIN: A New Platform for Enriching 3D Virtual Indoor Navigation With Social-based Geotags 26th IEEE Symposium on Computers and Communications (ISCC 2021). Christian Sicari , Valeria Lukaj , Antonio Celesti , Maria Fazio , and Massimo Villari . source | cite | search What Was Hybrid? a Systematic Review of Hybrid Collaboration and Meetings Research https://arxiv.org/pdf/2111.06172.pdf. Thomas Neumayr , Banu Saatci , Sean Rintel , Clemens Klokmose , and Mirjam Augstein . source | cite | search Mapping HCI Research Methods for Studying Social Media Interaction: A Systematic Literature Review Computers in Human Behavior. Yuya Shibuya , Andrea Hamm , and Teresa Cerratto Pargman . source | cite | search Digitizing Intangible Cultural Heritage in China: A Pedagogical Model for Innovation and Entrepreneurship Among New Media Art Students Lecture Notes in Computer Science. YunQiao Su , Sze Seau Jill Lee , BeiLe Su , and LiLi Zhang . source | cite | search Improving Information Acquisition in City Tours Via Simplified Virtual Scenes With Location-Based POIs Lecture Notes in Computer Science. Yujia Cao , Yiyi Zhang , and Tatsuo Nakajima . source | cite | search A Virtual Reality Whiteboard System for Remote Collaboration Using Natural Handwriting Electronics. Jiangkun Wang and Lei Jing . source | cite | search Working Together Apart Through Embodiment Proceedings of the ACM on Human-Computer Interaction. Guo Freeman , Dane Acena , Nathan J. McNeese , and Kelsea Schulenberg . source | cite | search Visualization/AR/VR/MR Systems Active Lighting and Its Application for Computer Vision. Katsushi Ikeuchi , Yasuyuki Matsushita , Ryusuke Sagawa , Hiroshi Kawasaki , Yasuhiro Mukaigawa , Ryo Furukawa , and Daisuke Miyazaki . source | cite | search Geolocated Visual Summarization of Social Media Data 2020 28th Signal Processing and Communications Applications Conference (SIU). Elif Sanlialp and M. Abdullah Bulbul . source | cite | search Automatic Discovery and Geotagging of Objects From Street View Imagery Remote Sensing. Vladimir Krylov , Eamonn Kenny , and Rozenn Dahyot . source | cite | search Deep Depth Estimation on 360\textdegree Images With a Double Quaternion Loss 2020 International Conference on 3D Vision (3DV). Brandon Feng , Wangjue Yao , Zheyuan Liu , and Amitabh Varshney . source | cite | search Foveated Rendering Technique in Modern Graphics University of Maryland, College Park. Xiaoxu Meng . source | cite | search Apparatus, Method, and System for Presentation of Multimedia Content Including Augmented Reality Content U.S. Patent 10,984,600. Leon Chen , Brian Keffer , and Jocelyn Seever . source | cite | search An Antiquity to the Contemporary of Secret Sharing Scheme Journal of Innovative Image Processing. K. Chitra and V. Prasanna Venkatesan . source | cite | search Impact of Key Shape and Dimension on Text Entry in Virtual Reality Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems. Tafadzwa Joseph Dube and Ahmed Arif . source | cite | search Virtual and Augmented Reality Applications in Science and Engineering Computing in Science & Engineering. Nargess Memarsadeghi and Amitabh Varshney . source | cite | search Leveraging the Physical Properties of Real Objects to Manage Digital Photography in Augmented Reality Journal of KIISE. H. Chae , Youli Chang , Minji Kim , Gwanmo Park , and Jinwook Seo . source | cite | search Students' Expectations and Social Media Sharing in Adopting Augmented Reality The International Journal of Information and Learning Technology. Ari Alamaki , Amir Dirin , and Jyrki Suomala . source | cite | search Surveying Consumer Understanding \& Sentiment of VR Proceedings of the International Workshop on Immersive Mixed and Virtual Environment Systems (MMVE '21). Joseph O'Hagan , Mohamed Khamis , and Julie Williamson . source | cite | search Culture Meets Immersive Environments: A New Media Landscape Across Europe Universitat Autònoma de Barcelona. Marta Brescia-Zapata . source | cite | search Exploring the Effectiveness of Face-to-face Mixed Reality for Teaching With Chalktalk https://arxiv.org/abs/1912.03863. Zhenyi He and Ken Perlin . source | cite | search Video Content Representation to Support the Hyper-reality Experience in Virtual Reality 2021 IEEE Virtual Reality and 3D User Interfaces (VR). Hyerim Park and Woontack Woo . source | cite | search A Survey on Synchronous Augmented, Virtual and Mixed Reality Remote Collaboration Systems ACM Computing Surveys. Alexander Schäfer , Gerd Reis , and Didier Stricker . source | cite | search Interactive Social Media Vsualization Ankara Yildirim Beyazit University. Elif Sanlialp . source | cite | search Teleportation Links for Mixed Reality Environments U.S. Patent 10,831,334. Paul Simon Martin . source | cite | search Telelocation: Location Sharing for Users in Augmented and Virtual Reality Environments U.S. Patent 10,403,044. Paul Simon Martin . source | cite | search Fine-grain Placement and Viewing of Virtual Objects in Wide-area Augmented Reality Environments U.S. Patent Application 15/600,449. Paul Simon Martin . source | cite | search Media Tags Location-anchored Digital Media for Augmented Reality and Virtual Reality Environments U.S. Patent Application 15/600,446. Paul Simon Martin . source | cite | search Methods, Systems, and Media for Enhancing Two-dimensional Video Content Items With Spherical Video Content U.S. Patent Application 15/366,432. Leon Bayliss and Richard Hale . source | cite | search Six-degree of Freedom Video Playback of a Single Monoscopic 360-degree Video U.S. Patent 10,368,047. Che Chen , Zhil Zhili , Duygu Ceylan Aksit , Jingwei Huang , and Hailin Jin . source | cite | search Impression Tagging System for Locations U.S. Patent 10,545,996. David Grochocki and Silvana P. Moncayo Torres . source | cite | search Augmented Reality Messaging Syste U.K. Patent GB2588838A. Wang Yue . source | cite | search Information Processing Apparatus, Object Recognition Apparatus, Device Control System, Movable Body, Image Processing Method, and Computer-readable Recording Medium U.S. Patent 10,937,181. Aman Amano , Seiy Seiya , Hiroyoshi Sekiguchi , and Soichiro Yokota . source | cite | search Computer Vision System That Provides Information Relative to Animal Wellness and Habitat/intervention Design U.S. Patent 10,943,102. Mark Raymond Miller , Archana Ramachandran , Christopher C. Anderson , and Aaron Spafford . source | cite | search Generating Three-dimensional Content From Two-dimensional Images U.S. Patent 11,113,887. Daniel Kopeinigg , Arthur Van Hoff , Philip Lee , Solmaz Hajmohammadi , Sourabh Khire , Simion Venshtain , and Andrew Walkingshaw . source | cite | search Generating Immersive Trip Photograph Visualizations U.S. Patent 11,113,882. Michal Luka , Zhili Chen , Jan Brejcha , and Martin Cadik . source | cite | search Immersive Street-level Social Media in the 3D Virtual City Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing. Hannu Kukka , Minna Pakanen , Mahmoud Badri , and Timo Ojala . source | cite | search Instant Panoramic Texture Mapping With Semantic Object Matching for Large-Scale Urban Scene Reproduction IEEE Transactions on Visualization and Computer Graphics. Jinwoo Park , Ik-Beom Jeon , Sung-Eui Yoon , and Woontack Woo . source | cite | search Immersive Trip Reports Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology. Jan Brejcha , Michal Lukác , Zhili Chen , Stephen DiVerdi , and Martin Cad{\'{\i}}k . source | cite | search Social Media Based 3D Visual Popularity Computers & Graphics. Abdullah Bulbul and Rozenn Dahyot . source | cite | search A Review on Mixed Reality: Current Trends, Challenges and Prospects Applied Sciences. Somaiieh Rokhsaritalemi , Abolghasem Sadeghi-Niaraki , and Soo-Mi Choi . source | cite | search A Review of Mixed Reality Telepresence IOP Conference Series: Materials Science and Engineering. Fazliaty Edora Fadzli , Ajune Wanis Ismail , Mohamad Yahya Fekri Aladin , and Nur Zuraifah Syazrah Othman . source | cite | search Worldsheet: Wrapping the World in a 3D Sheet for View Synthesis From a Single Image 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Ronghang Hu , Nikhila Ravi , Alexander C. Berg , and Deepak Pathak . source | cite | search Deep Learning-based Semantic Segmentation of Urban-scale 3D Meshes in Remote Sensing: A Survey International Journal of Applied Earth Observation and Geoinformation. Jibril Muhammad Adam , Weiquan Liu , Yu Zang , Muhammad Kamran Afzal , Saifullahi Aminu Bello , Abdullahi Uwaisu Muhammad , Cheng Wang , and Jonathan Li . source | cite | search Co-Speculating on Dark Scenarios and Unintended Consequences of a Ubiquitous(ly) Augmented Reality Proceedings of the 2023 ACM Designing Interactive Systems Conference. Chloe Eghtebas , Gudrun Klinker , Susanne Boll , and Marion Koelle . source | cite | search Mixed Reality: The Interface of the Future Technical Report. Dipesh Gyawali . source | cite | search Augmented Reality Content Generator for Suggesting Activities at a Destination Geolocation US Patent App. 17/146,973. Kaveh Anvaripour , Ilteris Kaan Canberk , Virginia Drummond , Jean Luo , Alek Matthiessen , and Celia Nicole Mourkogiannis . source | cite | search Mixed Reality in Education: A Bibliometric Analysis of Ten Years of Research Proceedings of the 4th International Conference on Virtual Reality. Das Runi Mani and J. V Madhusudan . source | cite | search 360RVW: Fusing Real 360\textdegree Videos and Interactive Virtual Worlds Proceedings of the 31st ACM International Conference on Multimedia. Mizuki Takenawa , Naoki Sugimoto , Leslie Wöhler , Satoshi Ikehata , and Kiyoharu Aizawa . source | cite | search Building Social Presence for Transnational Families Through Mixed Reality Shared Experiences 7. Shuting Zhou , Alexis Morris , and Sara Diamond . source | cite | search Using Google My Maps to Collect Vernacular Toponyms Onomastics in Interaction With Other Branches of Science Volume 1 Keynote Lectures Toponomastics. Sara Racca . source | cite | search 
