To pre-compute AO, we bake it into texture (just like normal map)
The key is to use depth map and do a sampling around point p to calculate the occlusion of point p.
But the above equation is wrong because it doesn't incorporate normal, we fix it in another paper.
But the above equation will produce AO for occlusion that is very far from point p.
This method can also be used in volumetric settings where each sampling will become a density.
Now comes a big improvement: GTAO
It first fixes above to method with a \cos \theta to incorporate direct illumination versus side illumination.
It also adds multi-scattering by fitting a polynomial between single-scattering and multi-scattering.
With GPU Power, we can shoot couple rays to determin AO in one frame and accumulate them.
In height fog, we model the fog as having uniform density near the bottom and decreasing density at very high sea-level. When calculating opacity, we have analytical solution to integration of fog density.
Volumetric fog act much like volumetric cloud as it use ray marching to calculate opacity. Note that the grid follows the camera as we see larger (denser) particles near the front. The grid dimension therefore closely align to screen resolution for smoother sampling.
Super-sample AA (SSAA): brute force render an image two or four times the resolution.
Multi-Sample AA (MSAA): we first do 4 samples to know which triangle the pixel is looking at. If all the samples are landing into the same triangle, then we only shade once for color, otherwise we shade 4 times for color.
A problem for modern game is that we usually have more triangles in a screen than the number of pixels.
We use the current unprocessed frame to predict where to do more sampling near the edge of a shape.
There are 4 steps:
Temporal Anti-Aliasing (TAA): we blend with last frame. Note that we should put more weight on the current frame for fast-moving objects than low-moving objects.
There might be a small offset artifact for this method, causing ghost shadow:
One fix is to add a magic number to prevent TAA when colors are too different.
Also TAA should process before post-processing to prevent blinking bloom.
Post Processing: for 2 purpose, correctness and style
bloom: can be explained with imperfect focus or diffusion through participating media
But above method requires big kernel and it is inefficient. Therefore we use a pyramid method to do gaussian blur.
A common issue, especially in a in-door environment, camera will produce images that is either very black inside or very bright outside. This is because, for HDR rendering, the brightness from outside environment is large and for indoor is small. Tone mapping is a easy fix for this.
Unlike Naughty Dog, ACS concentrates on film production. It provides a better curve for visual consistency between HDR and SDR screen.
Color grading is a color-to-color mapping in post processing often implemented as a lookup table.
We have learned about rendering:
the rendering equation
mesh, texture, shader, culling, transformation
lighting, shadow, global illumination
terrain, sky, cloud, fog
ambient occlusion, antialiasing
bloom, tone mapping, color grading
In forward rendering, we calculate every pixel by considering the mesh, texture, and lights.
We need to render transparent objects that is far away from us in order.
However, as more lights are in the game, the forward rendering pipeline is heavy.
We first render images of Albedo, Specular, Normals, and Depth first to G-buffer and then only use those images to compute final shading.
lighting only computed for visible fragments
data from G-buffer can be used for post-processing
high memory and bandwidth cost
no supporting transparent object
not friendly to MSAA
Tile-based rendering is inspired by mobile games:
heat is generated mostly in RAM
SRAM is fast (and better cooling), DRAM is slow (same idea as tiling in matrix multiplication)
Therefore it is good if we split the screen into grid-like tile and only use SRAM to store frame buffer
With tile-based method, if we use point light, for example, we can know for sure whether any pixel in the current tile is lit by a point light source. This saves us from having many unrelated lights.
Instead of 2D tile, we can also split the rendering task to 3D clusters
Unity Engine: Opaque objects will most likely be shaded using the tile system, whereas transparent ones will rely on the cluster system. (https://computergraphics.stackexchange.com/questions/8510/benefits-of-clustered-shading-vs-tiled-shading)
This is achieved with:
We can also use visibility buffer: storing depth, primitiveID, barycentric for each pixel. Therefore, to render an object, we simply look up information such as normal.
Standard G-Buffer used in deferred rendering assume rendering texture is more complex than geometry while V-BUffer assumes that rendering geometry is more difficult as AAA games are having more triangles than pixels.
Sometimes GPU kernel rasterization is faster than hardware rasterization. (mentioned by Xi Wang in GAME 104)
The lecture doesn't cover in-depth about visibility buffer, I highly recommend you to read this article
Rendering pipeline is a "complex" system, each trick add n^2 complexity to code. To make a good pipeline is to invent a tool for decoupling for better collaboration. URP and HDRP are developed using graphs to ensure good state management.
Frame graph is a relatively low level management system for better program architecture. It is still in exploration. It could be viewed as a low level rendering programming language.
Tearing Problem: when the frame rate of the monitor is higher than variable FPS, some pixel are in the next frame while others are still in the previous frame.
Another method is to use variable refresh rate: force monitor to refresh right after a frame is finished.
Table of Content