Lecture 007

Ambient Occlusion (AO)

To pre-compute AO, we bake it into texture (just like normal map)

Screen Space Ambient Occlusion (SSAO)

The key is to use depth map and do a sampling around point $p$ to calculate the occlusion of point $p$.

But the above equation is wrong because it doesn't incorporate normal, we fix it in another paper.

But the above equation will produce AO for occlusion that is very far from point $p$.

This method can also be used in volumetric settings where each sampling will become a density.

Now comes a big improvement: GTAO

It first fixes above to method with a $\cos \theta$ to incorporate direct illumination versus side illumination.

It also adds multi-scattering by fitting a polynomial between single-scattering and multi-scattering.

GPU-Driven AO

With GPU Power, we can shoot couple rays to determin AO in one frame and accumulate them.

Fog

Old Fog

Height Fog

In height fog, we model the fog as having uniform density near the bottom and decreasing density at very high sea-level. When calculating opacity, we have analytical solution to integration of fog density.

Volumetric Fog

Volumetric fog act much like volumetric cloud as it use ray marching to calculate opacity. Note that the grid follows the camera as we see larger (denser) particles near the front. The grid dimension therefore closely align to screen resolution for smoother sampling.

Anti-Aliasing

Super Sample AA

Super-sample AA (SSAA): brute force render an image two or four times the resolution.

Multi Sample AA

Multi-Sample AA (MSAA): we first do 4 samples to know which triangle the pixel is looking at. If all the samples are landing into the same triangle, then we only shade once for color, otherwise we shade 4 times for color.

A problem for modern game is that we usually have more triangles in a screen than the number of pixels.

Fast Approximate Anti-Aliasing (FXAA)

We use the current unprocessed frame to predict where to do more sampling near the edge of a shape.

There are 4 steps:

Temporal Anti-Aliasing (TAA)

Temporal Anti-Aliasing (TAA): we blend with last frame. Note that we should put more weight on the current frame for fast-moving objects than low-moving objects.

There might be a small offset artifact for this method, causing ghost shadow:

One fix is to add a magic number to prevent TAA when colors are too different.

Also TAA should process before post-processing to prevent blinking bloom.

Post-Processing

Post Processing: for 2 purpose, correctness and style

• bloom: can be explained with imperfect focus or diffusion through participating media

• tone mapping

• color grading

Bloom

But above method requires big kernel and it is inefficient. Therefore we use a pyramid method to do gaussian blur.

Tone Mapping

A common issue, especially in a in-door environment, camera will produce images that is either very black inside or very bright outside. This is because, for HDR rendering, the brightness from outside environment is large and for indoor is small. Tone mapping is a easy fix for this.

Unlike Naughty Dog, ACS concentrates on film production. It provides a better curve for visual consistency between HDR and SDR screen.

Color Grading

Color grading is a color-to-color mapping in post processing often implemented as a lookup table.

The Rendering Pipeline

We have learned about rendering:

• the rendering equation

• mesh, texture, shader, culling, transformation

• lighting, shadow, global illumination

• PBR material

• terrain, sky, cloud, fog

• ambient occlusion, antialiasing

• bloom, tone mapping, color grading

Forward Rendering Pipeline

In forward rendering, we calculate every pixel by considering the mesh, texture, and lights.

We need to render transparent objects that is far away from us in order.

However, as more lights are in the game, the forward rendering pipeline is heavy.

Deferred Rendering Pipeline

We first render images of Albedo, Specular, Normals, and Depth first to G-buffer and then only use those images to compute final shading.

Pros:

• lighting only computed for visible fragments

• data from G-buffer can be used for post-processing

Cons:

• high memory and bandwidth cost

• no supporting transparent object

• not friendly to MSAA

Tile-based Rendering

Tile-based rendering is inspired by mobile games:

• heat is generated mostly in RAM

• SRAM is fast (and better cooling), DRAM is slow (same idea as tiling in matrix multiplication)

• Therefore it is good if we split the screen into grid-like tile and only use SRAM to store frame buffer

With tile-based method, if we use point light, for example, we can know for sure whether any pixel in the current tile is lit by a point light source. This saves us from having many unrelated lights.

Cluster-based Rendering

Instead of 2D tile, we can also split the rendering task to 3D clusters

Unity Engine: Opaque objects will most likely be shaded using the tile system, whereas transparent ones will rely on the cluster system. (https://computergraphics.stackexchange.com/questions/8510/benefits-of-clustered-shading-vs-tiled-shading)

Forward+ Rendering (Forward Rendering but with Tiles)

This is achieved with:

1. Depth pre-pass (prevent overdraw and provide tile depth bounds)
2. Tile light culling (generate a list of lights per tile)
3. Shading per object (iterate over light list)

Visibility Buffer

We can also use visibility buffer: storing depth, primitiveID, barycentric for each pixel. Therefore, to render an object, we simply look up information such as normal.

Standard G-Buffer used in deferred rendering assume rendering texture is more complex than geometry while V-BUffer assumes that rendering geometry is more difficult as AAA games are having more triangles than pixels.

Sometimes GPU kernel rasterization is faster than hardware rasterization. (mentioned by Xi Wang in GAME 104)

The lecture doesn't cover in-depth about visibility buffer, I highly recommend you to read this article

Frame Graph

Rendering pipeline is a "complex" system, each trick add $n^2$ complexity to code. To make a good pipeline is to invent a tool for decoupling for better collaboration. URP and HDRP are developed using graphs to ensure good state management.

Frame graph is a relatively low level management system for better program architecture. It is still in exploration. It could be viewed as a low level rendering programming language.

Monitor Display

Tearing Problem: when the frame rate of the monitor is higher than variable FPS, some pixel are in the next frame while others are still in the previous frame.

Another method is to use variable refresh rate: force monitor to refresh right after a frame is finished.