Shadow Covenant

Well, after three days of work, I finally have a working lightmap renderer… I’m not very happy with the results, to be honest, but there’s still some places for improvement… to go further, I need a real example to work with, with some moving characters to see how enabling/disabling lights will work and the effect of having a priority queue for the light updates (so I can have stuff that only updates once every four or five frames)… more on this below…

Anyway, as I said in my previous post, I’m using the UV-unwraper that comes with D3DX, since this is just for a preprocess tool… the results are quite nice, but I had to tweak it a bit…

This was my first test scene, renderer without shadowmaps in my normal deferred renderer… There’s only diffuse color maps in the objects.

This is the first iteration of the lightmap (rendered in 6 seconds in debug mode, a single 256×256 map).

A closer look shows some edge artifacts… This got solved by simply doing a "bleed" effect on the generated lightmap.

The black spots in the bottom of the pyramid are due to sampling artifacts (since it’s at the same height as the plane, so it intersects). This can be solved through multisampling (on my wish list).

Here I had a big error (didn’t take a screenshot), where the orange and the yellow would fight, since that edge was shared in the lightmap. Fixing this was just a matter of tweaking the UV generation so that that edge wouldn’t be shared; so I added code that took in consideration that triangles whose normals differed more than 30 degrees wouldn’t be considered adjacent, and the results were good.

Mixing static and dynamic lights worked well. The red shadow cast by the red block is the result of a light circling the scene. Note that the edges are much more defined.

Adding the stained-window effect was pretty easy with the current system, and I love it, to be honest… 🙂

Adding normal mapping was also pretty easy, but problems become evident when we zoom in:

The lack of resolution of the lightmap shows pretty obviously… I think this can be solved through multisampling, or just doing a better sampling of the normal map to consider the resolution… But it’s pretty obvious that normal maps won’t work well in lightmaps, since lightmaps are good for low-frequency lighting, and normal maps add high-frequency data to it… There’s other solutions for this, like what Valve did in the Source engine: http://www2.ati.com/developer/gdc/D3DTutorial10_Half-Life2_Shading.pdf

This considers the direction in which light is coming and applies normal mapping in runtime on top of the lightmapping, which improves quite considerably the results, besides enabling us to use specular highlights aswell. Still need to think if this is worth implementing, since I’m not sure about lightmaps anyway… Since I have my own renderer, I can change it suit me…

The worst problems came with the dynamic objects… Casting shadows on dynamic objects is simple enough, just use the standard shadowmap technique… Problem is that dynamic objects cast shadows on static (lightmapped) objects, and this is where things become complicated…

The way I solved it was by rendering the static lights three times… First time is the generic lightmap pass, in which all static objects are rendered, and a stencil mask is created. 0 on the stencil buffer means that pixel comes from a dynamic object, 1 means its from a static object.

Second time it’s for the dynamic objects (stencil buffer=0), and the results are as expected (both the static objects and the dynamic objects cast shadow on the dynamic objects).

Third time it’s for the static objects (stencil buffer=1), and it requires a subtractive pass… Basically, for each pixel of static objects, I compute the light that would reach that point, check if the point is being shadowed and subtract that light from what’s in the render buffer. Problem with this approach is that the lightmapped shadows will be twice as dark, since they’re already present in the shadowmap and the value will be subtracted.

To fix this, I had to find a trick, which almost does it… Basically, when rendering the shadowmap, I render the pixels in the [0..1] range if the object is static, and in the [1..2] range if the object is dynamic. Then, when I’m computing the subtraction, I only account for the dynamic objects (since the shadows cast by static objects are already accounted for in the lightmap). As I said, this almost works, with two caveats:

In this screenshot, you can see the "ragged edge" between the static-cast shadow of the block, and the dynamic-cast shadow of the skinned cylinder… This is due to the resolution of the shadow map, and it’s extremely hard to get rid of, without raising the shadowmap’s resolution to proibitive values… I think we might be able to solve this by multi-sampling the shadowmap, but that would required more cycles, and for the game we’re working on, I don’t think these small mistakes will matter…

A slightly worse problem:

The shadow from the cylinder in this case subtracts light to the block-cast shadow, which shouldn’t happen… This is due to the fact that the computed shadowmap only "sees" the cylinder, and the block behind, so the system doesn’t know that the block is also casting a shadow at that pixel… I can think of a couple of ways to solve this (two different shadowmaps for static and dynamic objects, for example), but most of them just consume more GPU/memory/etc, and for a small gain…

Finally, the biggest problem: due to the limited precision of the render buffer (8-bit per channel), if we have multiple lights illuminating a place and casting a shadow on that place, the shadows will become darker than they should… Reason for this is the following:

Imagine 2 lights, illuminating a certain pixel… Light A casts 80% light, light B casts 60% light… Summed together, this is 140% light, which gets clamped on the buffer to 100%. Then, we want a dynamic object casting a shadow on that spot (because it blocks light B). Because of the current algorithm, light B is subtracted to the frame buffer, which makes the intensity there go to 40%, instead of the 80% it should…

This could be fixed with HDR buffers (which wouldn’t be so hard to add), but by this time I’m under the feeling that I’m just fixing one problem after another, without having a complete lighting solution, or worst, having a complete lighting solution that requires too much tweaking…

So now I’m thinking that I may ditch all of this and go for a full dynamic lighted system. Of course, I can’t have 100 lights casting shadows, etc, but I think I can reduce this substantially by making lights that aren’t near the player use less resolution and updating less frequently, and go around interpolating the lights to get the correct results… Now I just need a working scenario so I can see the impact of this, with some characters moving around and some real geometry to check perfomance…

Anyway, this isn’t a complete lost… worst case scenario, I’ll be able to use this system to create ambient occlusion maps to get a bit more realistic look without having to resort to SSAO (which I only liked in Crysis)…

Unfortunately, for the next foreseeable days, I’ll be busy with hard work stuff, which means no energy in my free time to work this…

Stay tuned for more adventures with lighting pipelines! 🙂

Finally some of the work insanity has died down, and as so I can go back to do some cool stuff on my spare time again…

After the basis of the renderer for our next project is done, I still feel there’s loads of stuff to be done in the environment lighting… since it is unfeasible to have shadowmaps in all visible lights (still have to determine a good algorithm to select lights to be shadowmapped in real time),I’ve decided to add lightmaps to the environment, and in the process (since it’s easy to add), also add pre-rendered ambient occlusion maps.

I’ve worked on lightmap generators in the past.

First of my experiments were with direct lighting lightmaps, which only account for the light that shines directly in objects. I built my own UV-map generator, which didn’t work that badly, but it was slow and some of the results were terrible… The advantage of that renderer was that I could add stained-window effects (through alpha maps) and other nifty effects to the resulting lightmap…

Some pictures of the obtained results (over 3 years ago):

Two sources of light, one red and one white, no textures.

Same as above, but textured, which makes everything look infinitely better, since it disguises the imperfections of the limited resolution of the shadowmap.

An alpha object that blocks part of the light and tints it.

Again an alpha object, but with a textured alpha-map… The effect is quite nice.

Although the results I obtained with this were nice enough, I was never too happy with the UV-generation and the lack of soft-shadows, so a couple of years ago, I did some experiments with hardware-accelerated radiosity.

I replaced my UV-generator with the one that DirectX 3DX library has, and used the normal HW-accelerated radiosity algorithm (render scene from the point of view of the patch, average results, rinse and repeat).

While the results weren’t terrible, it demanded too much tweaking to get good results:

The wall is an emissive light source.

After 6 iterations, the shadows become more “real” and you can see some color blending near the green “barrels”, which is pretty neat. The window is a glowing polygon, to simulate intense outside light.

Using a light that’s not white, and a spherical light source (with volume), that’s not visible from the whole room. After 6 iterations, the scene looks warmer, but it has too many artifacts on the floor (sampling artifacts), and in the near wall (because the wall was too close to the patch, it wouldn’t get drawn in the buffer, due to the near plane. This would lead to that silly illumination).

Using a colored floor (at a much lower resolution that the previous tests), the scene again becomes warmer and indirectly lit.

I supported normal maps in the rendering, and the result is quite nice, disguising some artifacts behind the more complex lighting.

I even did an application that would enable me to change the reflectivity parameters:

But it was too much work getting this to work properly, too many small tweaks and too much time to actually see results… This is when I abandoned this project and moved on to other stuff (I know, I lack focus sometimes).

Anyway, back to the present, I decided to pick this up again… For the current game, I think radiosity is overkill, but I intend to divide the generation of lightmaps in two parts, so I can replace the direct lighting for radiosity later:

• Preprocessor
• Lighting engine

Basically, the preprocessor will take care of all the tasks that are common to any lightmap processor:

1. See what meshes are present in the scene
2. Create UV-map per mesh (using as a metric for resolution the surface area of the mesh and an adjustable parameter)
3. Create instances of objects that share the same mesh (effectively copying the mesh with the new UVs)
4. Create an UV-atlas that aggregates UV-maps
5. Load into RAM the textures used (diffuse map, alpha map, normal map, emissive, etc). This data needs to be in accessible memory since we’ll need to sample it
6. Initialize the lighting engine (give it all the meshes, etc, so he can build acceleration structures for raycasts, etc)
7. For all lumels in all the lightmaps, call the light evaluation function on the lighting engine. This will probably be called multiple times for each lumel, taking into account the possible area of the lumel, so that we get anti-aliasing and reduce the artifacts caused by edge conditions. These samples will be averaged (either by standard average or some kind of distribution).
8. This last step will be repeated a certain ammount of times, depending on the instructions of the lighting engine.
9. If requested, build ambient occlusion map, by requesting the lighting engine an ambient occlusion factor. This will also take in consideration the area of the lumel, like in step 7.
10. Save lightmaps generated

The lighting engine for now will be a simple Direct Light system. I want to take in consideration the following things:

1. Take into account diffuse, normal and emissive maps
2. An octree will be generated with all the world geometry for faster raycasting
3. Raycasting will take in account the possible intersection of alpha/tinted objects and do the correct coloring to account for that
4. Ambient occlusion will be computed by raycasting in an hemisphere around the lumel being computed, and considering what amount of rays gets intersected.

This system should also consider multi-processor usage (for direct light at least, since the HW-accelerated radiosity can’t take advantage of multiple processors, since there’s just one GPU).

Most of this stuff I already know how to do, since I have in done in previous projects… the exception is the octree generation (I usually use loose octrees, so there’s loads of code for splitting triangles, etc, that has to be done), and the part where I consider the area of the lumel for smoothing the lightmap.

Rendering the lightmaps in the deferred rendering pipeline will require an additional passage, since I don’t have any space left on the G-Buffer for three components (RGB, since I want colored lightmaps), but that’s ok, since it will enable me to do fill up the stencil buffer to avoid drawing the static lights onto the static geometry again, while I still have the possibility of using the static lights on the dynamic geometry. This will also enable me to light the environment with dynamic lights (in static and dynamic geometry) without wasting too much performance. Adding the concept of light volumes to both types of lights will also be easier this way, since I can filter out what parts of the scene shouldn’t be affected. The ambient occlusion term can be stored with the ambient component already in the G-Buffer, so that shouldn’t have a big performance hit.

All the lights and geometry that are present in the input scene will be considered static… Bad part of this is that opening doors won’t have the dramatic/cool effect I would like, unless I use dynamic lights and shadowmaps at the correct circumstance.

Anyway, I’ll hopefully have something working by next week (hopefully in time for the Screenshot Saturday thingie which I enjoy seeing every week).

Ah, just recalled… why do my own lightmap generator, instead of using an existing (free) one? Well, I never found one that produced nice results while making the trouble of writing an importer/exporter wasn’t a nightmare, to be honest… I feel that this is the kind of thing that’s kind of linked with your pipeline and your own way of doing stuff… besides, this is fun code! 🙂

So, wish me luck!