A WebTransport-based System for Real-time Game Streaming
ACM Mile-High Video
Feb 18 – 20, Denver, USA
[PDF]
Authors: Minh Nguyen, Dominik Busse, Stefan Pham, Stephan Steglich, Stefan Arbanowski
Abstract: In recent years, the demand for remote rendering and cloud gaming solutions has grown significantly, pushing the boundaries of traditional game development and distribution methods. This work aims to explore and implement a novel approach to streaming rendered frames from the Unity game engine to web clients using the WebTransport protocol. Unity, one of the world’s leading game development platforms, provides a powerful environment for creating interactive 3D and 2D content. Its versatility and extensive feature set make it an ideal candidate for exploring advanced rendering and streaming techniques. While Unity already offers solutions to enable the direct streaming of rendered content, we aim to incorporate the new web standard “WebTransport” and test the
feasibility of using this new method of data transfer.
Built on top of HTTP/3, WebTransport provides low-latency, bidirectional, and multiplexed data transfer between clients and servers. Its ability to handle both reliable and unreliable data streams makes it particularly suitable for real-time applications like game streaming. We utilized a multi-process architecture to achieve the goal of streaming rendered frames from Unity to a web client via Web-
Transport.
The system comprised the following key components:
(1) Unity Game Engine: This serves as the primary environment for rendering frames and managing the overall process flow.
(2) WebTransport Server: We implemented a WebTransport server using Go, based on the webtransport-go-example. This server facilitated the transmission of encoded video frames to the client via a unidirectional stream as well as the sending and receiving of simple messages via datagrams.
(3) FFmpeg: This widespread video encoding tool was employed to encode the raw video frames generated by Unity into a fragmented MP4 format suitable for streaming.
(4) Web Client: We developed a simple client application using HTML and JavaScript to display the streamed video. This client was also based on the aforementioned webtransport-go-example.
