Q2VKPT

Q2VKPT is the first playable game that is entirely raytraced and efficiently simulates fully dynamic lighting in real-time, with the same modern techniques as used in the movie industry (see Disney's practical guide to path tracing). The recent release of GPUs with raytracing capabilities has opened up entirely new possibilities for the future of game graphics, yet making good use of raytracing is non-trivial. While some games have started to explore improvements in shadow and reflection rendering, Q2VKPT is the first project to implement an efficient unified solution for all types of light transport: direct, scattered, and reflected light (see media). This kind of unification has led to a dramatic increase in both flexibility and productivity in the movie industry. The chance to have the same development in games promises a similar increase in visual fidelity and realism for game graphics in the coming years.

This project is meant to serve as a proof-of-concept for computer graphics research and the game industry alike, and to give enthusiasts a glimpse into the potential future of game graphics. Besides the use of hardware-accelerated raytracing, Q2VKPT mainly gains its efficiency from an adaptive image filtering technique that intelligently tracks changes in the scene illumination to re-use as much information as possible from previous computations.

About

The project is released as open source on GitHub, integrating our Vulkan path tracer into the Q2PRO client. The project was spawned by the need for fast-paced test content in computer graphics research, and was motivated by the first intriguing results of an experimental path tracing renderer in 2016.

History

VKPT and Q2VKPT were created by Christoph Schied as a spare-time project to validate the results of computer graphics research in an actual game. The project currently encompasses 12K lines of code and completely replaces the original Quake II graphics code. Initially, it was prototyped in OpenGL with contributions by Johannes Hanika (Experimental Ray Tracing, Shaders, GL/Vulkan Fixes), Addis Dittebrandt (Light Hierarchy, Debug Visualisation), Tobias Zirr (Light Sampling, Hack Supervision and Instigation, Website, Info Texts), and Florian Reibold (Initial Light Hierarchy). Additional help was provided by Stephan Bergmann, Emanuel Schrade, Alisa Jung, and Christoph Peters (made some noise).

Download

Videos

Development outtakes: glitch video 1, glitch video 2

Screenshots

We advise you to watch the video or play the game yourself, since the experience with the fully dynamic lighting is much better in motion. :-)
Loading
Fully dynamic global illumination using path tracing, with raytraced shadows, glossy reflections and one bounce of indirect lighting.

Screenshots: [1] [2] [3] [4] [5] [6] [7] [8] [9]

Output of the path tracer (left) and the denoising filter applied (right).

Slides

Q2VKPT was presented at GTC and GDC 2019. The slides contain plenty of animations, therefore we recommend you take a look at the pptx version. pptx, pdf

Tech

Q2VKPT is implemented in the Vulkan API to be able to use the new hardware-accelerated raytracing features that were made available earlier this year. Thanks to these, the game can actually come close to 60 FPS (2560x1440, RTX2080Ti), while being fully raytraced and dynamically shaded with realistic global lighting models in real-time. Using path tracing for fully dynamic lighting allows for a lot more detail in the shading of game scenes, naturally producing complex interplay of hard and soft shadows, glossy material appearances and perspectively correct reflections everywhere. Moreover, light can naturally flow anywhere, tying the scene together in the ways we would expect from the real world. Traditional approaches like precomputed lighting or coarse real-time raster approximations could never interactively reach this detail at a comparable resolution, since full storage of this lighting information would exceed any memory bounds.

Technical Details

Q2VKPT makes use of a variety of techniques to adapt computationally expensive methods previously only used in the movie industry to games. At the core is an adaptive temporal filter that intelligently re-uses the results of previous frames (a former research project of Christoph's). On top of the nowadays well-established temporal anti-aliasing, this filter extends the tracking of changes over time from mere image space to the high-dimensional space of light transport paths. Path change tracking is necessary because path tracing is a randomized algorithm, which is both its strength and its weakness: While it can generically handle all kinds of light transport, results may need a long time to become reliable, too long for one frame in a real-time game. A mere image-space approach like temporal anti-aliasing cannot deal with this amount of uncertainty.

On top of temporal filtering, we have explored several techniques of finding the lights that influence each surface in the game. Picking the right lights is crucial for image quality, since picking the wrong lights results in highly unreliable pathtracer output, which then forces the temporal filter to remove all the beautiful details we wanted the pathtracer to produce in the first place. An initial prototype used a full light hierarchy as is also common in movie production: By clustering lights hierarchically, the influence of many lights can be estimated at once, allowing for quick exclusion of far-away, dim lights and of lights pointing the wrong way. However, such estimates turned out hard to get precise, and the computational cost of traversing the hierarchy hard to control. Since Quake II originally shipped with a Potentially-Visible-Set for culling occluded parts of the scene, we instead resorted to extracting lists of potentially visible lights from these sets for every part of the scene. In the current version, we implemented a semi-accurate estimation of the influence of every light in the list, by randomly picking a representative subset of these lists in every frame. Thus, the renderer quickly hits all light sources over time, and we can also do all influence estimation in a controlled, constant time budget.

FAQ

Why does the game crash for me?

We are very sorry, since this is a spare-time project, we do not have the capacity for thorough quality assurance testing. You are welcome to file a bug report into the issue tracker of our GitHub repository!

Why are there no particles? I want my flies and rail gun sparks back!

Unfortunately, we ran out of time to add lights for the rail gun. As to other particles, the decision to fully raytrace everything in the game made rendering particles a bit less trivial. If we find the time, we might fix it in the future!

Games already look real today! Why would they use path tracing?

Today's games have taken the capabilities of traditional rasterization-based graphics very far. However, these advancements come with a price: The rendering engines of modern games have become extremely complex heaps of special-purpose techniques. Lighting, shadows and reflections each have to be computed at many different resolutions and scales, in order to ensure an acceptably artifact-free visual experience. Path tracing and other Monte Carlo Rendering techniques could open up a way out of this ever-growing complexity. In fact, it has already achieved that for the movie industry. This prototype is a first step towards answering a few of the open questions on how to accomplish the same in the games industry.

Quake II is ancient! If these techniques have any future, it should run at 6000 FPS by now!

While it is true that Quake II is a relatively old game with rather low geometric complexity, the limiting factor of path tracing is not primarily raytracing or geometric complexity. In fact, the current prototype could trace many more rays without a notable change in frame rate. The computational cost of the techniques used in the Q2VKPT prototype mainly depend on the number of (indirect) light scattering computations and the number of light sources. Quake II was already designed with many light sources when it was first released, in that sense it is still quite a modern game. Also, the number of light scattering events does not depend on scene complexity. It is therefore thinkable that the techniques we use could well scale up to more recent games.

Why Quake II?

Since Quake II is open source and has a long-standing modding tradition, it is a great sandbox for putting academic research to the test in the real world. Particularly, the game has fast-paced action and is played competitively, setting high standards for the performance and robustness of any implemented rendering techniques. Finally, in some sense Quake II is to this day quite a modern game, since it already shipped with complex and artistic light design back when it was first released.

How can I use Q2VKPT as a benchmarking tool?

Open the console using the "~" key. Depending whether you are using the demo or the full version of Quake 2 you can use "timedemo 1; demo q2demo1" or "timedemo 1; demo demo1" respectively. You can enable the inbuilt profiler using "vkpt_profiler 1" to get details about the GPU timings for the individual rendering steps.

How is path tracing different from raytracing?

Path tracing is an elegant algorithm that can simulate many of the complex ways that light travels and scatters in virtual scenes. Its physically-based simulation of light allows highly realistic rendering. Path tracing uses Raytracing in order to determine the visibility in-between scattering events. However, Raytracing is merely a primitive operation that can be used for many things. Therefore, Raytracing alone does not automatically produce realistic images. Light transport algorithms like Path tracing can be used for that. However, while elegant and very powerful, naive path tracing is very costly and takes a long time to produce stable images. This project uses a smart adaptive filter that re-uses as much information as possible across many frames and pixels in order to produce robust and stable images.

Are path tracing and raytracing the future of game graphics?

The recent release of GPUs with raytracing capabilities has opened up entirely new possibilities for the future of game graphics, yet making good use of raytracing is non-trivial. The purpose of this project is to find out exactly what's still missing for a clearer pathway into a raytraced future of game graphics. While some problems have already been addressed by academic research, many open real-world problems slip unnoticed until one actually tries to implement a full game renderer. We plan to look into some of these issues, like better light sampling, better filtering, and more consistent renderer software architecture in the future. Good solutions for the new, different issues of raytracing- and path tracing-based renderers will be necessary for this change to happen on a broad scale.

Will path tracing finally solve all graphics problems?

Path tracing and similar methods elegantly solve many hard problems of game graphics. However, their randomized nature also introduces a whole new set of challenges in order to be able to produce clean (noise-free) high-quality images. The filtering method in the Q2VKPT prototype is a first step towards robust unified methods that allow a fully unified, brute-force path tracing approach to rendering. We expect and hope to see more research into that direction in the next months and years!

Does Q2VKPT use the original light maps?

The original Quake 2 engine uses precomputed light maps that contain soft shadows and diffuse indirect illumination. In contrast, Q2VKPT entirely replaces the static illumination using a fully dynamic simulation that unifies both the static and dynamic light sources.

How many rays does Q2VKPT cast per pixel?

The number of rays that are cast are dependent on the first visible surface. For opaque surfaces Q2VKPT uses one ray each to find the direct and indirectly visible surface. Additionally, for both surfaces Q2VKPT casts one ray each towards randomly chosen light sources. Therefore Q2VKPT will cast at least 4 rays for each pixel.

Does Q2VKPT rely on rasterization?

Traditionally, games use the rasterizer to find the surfaces visible from the camera. Q2VKPT is fully raytraced and does not rely on the rasterizer (except for the 2D user interface elements).

Contact

This project was initiated by Christoph Schied. Inquiries about the project via mail can be sent to q2vkpt.