SKTel/Google: Changdeok ARirang

Just before COVID hit the world, Nexus Studios asked me to help out with a 5G-powered, AR-enhanced experience for Changdeok Palace, a UNESCO World Heritage site in South Korea.

The project took advantage of the latest Google AR technologies (ARCore and Cloud Anchors) to enable a rich mobile experience for both physical and virtual visitors to the site, bringing the location's history to life by conjuring AR versions of traditional Korean folklore characters.

Visitors to the site contributed their own personalised content in the form of virtual kites, saved into the scene using Cloud Anchors and stored in a remote backend API.

As this was such a rich experience it required a large amount of volumetric 3D data to be transmitted to the users' phones, but WiFI and phone availability at the site are limited. SKTel, the project's Korean telco co-sponsor, stepped in to provide 5G access, helping users get much faster downloads and hence much snappier response times in the application.

I worked with Nexus's 3D team to design and build the cloud API on Google Cloud Platform infrastructure, and co-ordinated with SKTel's network teams, implementing a Mobile Edge Cache system providing a site-local, 5G-optimised data store, with assets smart-mirrored to edge servers, keeping them close to the end-user devices. This way SKTel 5G users would have the most streamlined experience, with data rates much faster than most WiFi.

I wrote the backend API in node.js from scratch in fairly short order, running on Google Cloud Run and using the Firestore Native Database, enabling realtime data updates during complex 3D development phases, and ensuring that the 247,000 virtual users who downloaded the app could enjoy the site experience remotely with similar performance to the 5G users at the real-world location.

I set up CI/CD using CloudBuild in multiple test and production environments so that the 3D team could continue to tweak and iterate as necessary. COVID-19 happened at exactly the right time to mess up the installation plans, but I kept in touch with Nexus as they worked tirelessly over that time to ensure a smooth delivery later in the year when physical access was possible again.