Igor Clark

MWC 2019: synchronized dioramas

Igor Clark -

Google's NYC-based Hardware Marketing team came up with a really nice concept for a series of automated dynamic tableaux, showcasing various features of their product lineup at MWC 2019 in Barcelona.

The displays included products ranging from Pixel phones and Slate tablets to Nest home automation lines, each of which should display custom content on-device to highlight various product features out of the normal working context - in a physical installation with constraints making it impossible to use the normal software.

Synchronising multiple streams of activity, on multiple OSes and different products with physical display mechanisms and lighting systems, was proving challenging. So the team asked me to help co-ordinate the various systems and come up with solutions for reliably getting the right content to the right displays, on the right devices, at the right times.

Consulting with the creative, product and engineering teams to work out exactly what we could and couldn't do with the various devices, I quickly established that there was no way of getting custom OS builds for the Home Hub devices in the time available.

So I settled instead on a web-based display system using Google Cast to send content to the Home Hubs, and an event-based messaging system using open Internet protocols to communicate with the products and physical display mechanisms.


Included in the installation were:

  • 5 Pixel phones running a complete web-based mock-up of the Lens UI, mounted on a rotating carousel synced with the control server via UDP, "recognising" a pre-planned sequence of commercial products
  • A custom USB control mechanism to handle automated state changes on the Nest thermostat (with an SSH fallback in case the physical constraints made wiring impossible), synchronising lighting in the diorama by sending control messages to a Pharos lighting system
  • More UDP messaging with a revolving segment of a front-door controlling what to display via custom live streaming video from a display-mounted camera, Cast-ed to HTML content on a Home Hub video screen
  • Another Pixel mounted on a moving platform, alternately connecting and disconnecting to a wireless charger, sync'ed via the control server to know when to display appropriate content depending on whether it was "charging" or not
  • A Home Hub displaying animated segments of the Home Hub UI as though in stop-motion animation Cast'ed to the device and then sequenced via Websocket in time with the Pharos, to give the impression of a calendar day's events taking place
  • Marvel Universe content streaming to a Pixel Slate in an animatronic miniature movie theater

I took responsibility for the end-to-end design, build and delivery of all the software across the installation. I sourced and organised resource for front-end UI programming support, collaborated with the the hardware and lighting production companies, designed an overall system with a central control server synchronising everything, and got cracking.

Time was short so I wrote the control server in Erlang/OTP, which made it a lot quicker to get to a system that received and passed messages to and from the devices using the necessary mixture of UDP streams, Websocket and USB connections, and Web content.

I spec'd and sourced redundant hardware servers to run and serve the control software using Docker, built a test rig of the system at the production company's workshop in London, flew to Barcelona with the Google and production company teams, and installed and set up all the devices and software in situ at MWC.

Finally, I stayed on-site and kept an eye on all the systems through til the end of the event. Fortunately things went well enough that the last few days were pretty relaxed. Boring, even.