To pre-compute AO, we bake it into texture (just like normal map)
Screen Space Ambient Occlusion (SSAO)
Screen Space Ambient Occlusion
The key is to use depth map and do a sampling around point p to calculate the occlusion of point p.
But the above equation is wrong because it doesn't incorporate normal, we fix it in another paper.
SSAO+
But the above equation will produce AO for occlusion that is very far from point p.
Or You Can Calculate Angle for Nearby Pixels Instead of Sampling
Angle Based Method using Ray Marching
This method can also be used in volumetric settings where each sampling will become a density.
Now comes a big improvement: GTAO
Ground Truth-based Ambient Occlusion
It first fixes above to method with a \cos \theta to incorporate direct illumination versus side illumination.
It also adds multi-scattering by fitting a polynomial between single-scattering and multi-scattering.
Multi-scattering by fitting polynomial
GPU-Driven AO
With GPU Power, we can shoot couple rays to determin AO in one frame and accumulate them.
Fog
Old Fog
Old Fog
Height Fog
Height Fog
In height fog, we model the fog as having uniform density near the bottom and decreasing density at very high sea-level. When calculating opacity, we have analytical solution to integration of fog density.
Volumetric Fog
Volumetric Fog
Volumetric fog act much like volumetric cloud as it use ray marching to calculate opacity. Note that the grid follows the camera as we see larger (denser) particles near the front. The grid dimension therefore closely align to screen resolution for smoother sampling.
Anti-Aliasing
Anti-aliasing problem
Super Sample AA
Super-sample AA (SSAA): brute force render an image two or four times the resolution.
Multi Sample AA
MSAA
Multi-Sample AA (MSAA): we first do 4 samples to know which triangle the pixel is looking at. If all the samples are landing into the same triangle, then we only shade once for color, otherwise we shade 4 times for color.
A problem for modern game is that we usually have more triangles in a screen than the number of pixels.
Fast Approximate Anti-Aliasing (FXAA)
Fast Approximate Anti-Aliasing (FXAA)
We use the current unprocessed frame to predict where to do more sampling near the edge of a shape.
There are 4 steps:
1. First we determine whether a pixel is classified as "horizontal" or "vertical" aliasing
2. Then we search along the edge to determine the slope of the line we are blending accross
3. We calculate the blending coefficient according to the slope
4. Blend it
FXAA Result
Temporal Anti-Aliasing (TAA)
Temporal Anti-Aliasing (TAA)
Temporal Anti-Aliasing (TAA): we blend with last frame. Note that we should put more weight on the current frame for fast-moving objects than low-moving objects.
There might be a small offset artifact for this method, causing ghost shadow:
Unpleasant Artifect for TAA
One fix is to add a magic number to prevent TAA when colors are too different.
Also TAA should process before post-processing to prevent blinking bloom.
Post-Processing
Post Processing: for 2 purpose, correctness and style
bloom: can be explained with imperfect focus or diffusion through participating media
tone mapping
color grading
Bloom
Gaussian Kernel for Blooming
But above method requires big kernel and it is inefficient. Therefore we use a pyramid method to do gaussian blur.
Pyramid Gaussian Blur
Tone Mapping
A common issue, especially in a in-door environment, camera will produce images that is either very black inside or very bright outside. This is because, for HDR rendering, the brightness from outside environment is large and for indoor is small. Tone mapping is a easy fix for this.
Naughty Dog's Filmic S-Curve Based On Polynomial Regression
ACS's Exposure Curve
Unlike Naughty Dog, ACS concentrates on film production. It provides a better curve for visual consistency between HDR and SDR screen.
Tone Mapping Curve Comparison
Color Grading
Color grading is a color-to-color mapping in post processing often implemented as a lookup table.
32x32x32look up table with linear interpolation
The Rendering Pipeline
We have learned about rendering:
the rendering equation
mesh, texture, shader, culling, transformation
lighting, shadow, global illumination
PBR material
terrain, sky, cloud, fog
ambient occlusion, antialiasing
bloom, tone mapping, color grading
Forward Rendering Pipeline
In forward rendering, we calculate every pixel by considering the mesh, texture, and lights.
Transparent Sorting
We need to render transparent objects that is far away from us in order.
However, as more lights are in the game, the forward rendering pipeline is heavy.
Deferred Rendering Pipeline
We first render images of Albedo, Specular, Normals, and Depth first to G-buffer and then only use those images to compute final shading.
Pros:
lighting only computed for visible fragments
data from G-buffer can be used for post-processing
Cons:
high memory and bandwidth cost
no supporting transparent object
not friendly to MSAA
Tile-based Rendering
Tile-based rendering is inspired by mobile games:
heat is generated mostly in RAM
SRAM is fast (and better cooling), DRAM is slow (same idea as tiling in matrix multiplication)
Therefore it is good if we split the screen into grid-like tile and only use SRAM to store frame buffer
Light Culling by Tiles in Deferred Rendering
With tile-based method, if we use point light, for example, we can know for sure whether any pixel in the current tile is lit by a point light source. This saves us from having many unrelated lights.
Tile-based Light Rendering
Cluster-based Rendering
Instead of 2D tile, we can also split the rendering task to 3D clusters
Cluster-based Rendering
Unity Engine: Opaque objects will most likely be shaded using the tile system, whereas transparent ones will rely on the cluster system. (https://computergraphics.stackexchange.com/questions/8510/benefits-of-clustered-shading-vs-tiled-shading)
Forward+ Rendering (Forward Rendering but with Tiles)
This is achieved with:
Depth pre-pass (prevent overdraw and provide tile depth bounds)
Tile light culling (generate a list of lights per tile)
Shading per object (iterate over light list)
Visibility Buffer
Visibility Buffer
We can also use visibility buffer: storing depth, primitiveID, barycentric for each pixel. Therefore, to render an object, we simply look up information such as normal.
Standard G-Buffer used in deferred rendering assume rendering texture is more complex than geometry while V-BUffer assumes that rendering geometry is more difficult as AAA games are having more triangles than pixels.
Sometimes GPU kernel rasterization is faster than hardware rasterization. (mentioned by Xi Wang in GAME 104)
The lecture doesn't cover in-depth about visibility buffer, I highly recommend you to read this article
Frame Graph
Unreal Rendering Pipeline from https://unrealcommunity.wiki/unreal-schematics-2d6859
Rendering pipeline is a "complex" system, each trick add n^2 complexity to code. To make a good pipeline is to invent a tool for decoupling for better collaboration. URP and HDRP are developed using graphs to ensure good state management.
Frame Graph
Frame graph is a relatively low level management system for better program architecture. It is still in exploration. It could be viewed as a low level rendering programming language.
Monitor Display
Tearing Problem: when the frame rate of the monitor is higher than variable FPS, some pixel are in the next frame while others are still in the previous frame.
BSync: force monitor to not refresh when a frame is not finished.
Another method is to use variable refresh rate: force monitor to refresh right after a frame is finished.