In this work, we tackle the problem of real-time virtual reality streaming in low-bandwidth network conditions. In low-bandwidth settings, VR streaming faces an inherent trade-off between responsiveness and image resolution, both of which are crucial to a high quality user experience. To overcome this trade-off, we propose to send low-resolution frames over the network and then apply neural super-resolution on the client to generate high-resolution frames. We implement a client/server setup for a 3D virtual environment as a proof of concept, and demonstrate that even under low-bandwidth (12~Mbps) network conditions, our system is still able to deliver a fast, responsive, and high-resolution VR experience to the end user.
Authors
Xingyuan Sun, Jimmy Wu, Runzhe Yang