Diffuse light/shadow

Question

I just implemented a light system in my engine. In the following screenshot you can see a light (the yellow square) in action:

Take into account that on top of the light illuminating the scenario, its also implemented a FOV which will occlude anything outside your field of view. Thats why the left part of the shadow seems so off.

As you can see the light's shadows are pretty "hard", as they won't even illuminate one bit of the area outside its direct reach. In order to make the lights look better, I applied a filter to them, which pretty much limits the range to be illuminated, and also iluminate the area slightly within this limit:

In the big yellow circle you can see how the area is illuminated even if no direct light reaches it.

This solution however comes with some undesirable side effects. As you can see in the following screenshot, even if no light at all reaches an area, it will be illuminated if its too close to the light source:

I was wondering if there is any way to achieve what im trying to do by using shaders properly. The main problem that I encounter comes from how I draw these shadows.

1) First I take the structures within the light's range.

At this point, I'm working with vertex, as they define the area of the shadow casting items:

2) Then, for each of these objects, I calculate the shadow they cast individually:

The shadow they cast is done by the CPU, by calpulating projections for each vertex of the body.

3) Then the GPU draws these shapes into a texture to compose the final shadow:

The problem I find is that making this difuse shadow effect, I need the final shadow. If I were to calculate the diffused shadows in step 2), a gap of light would appear between solids B and C.

But if I difuse the shadows in step 3, I no longer have the vertex information, as all the info I have are the 3 local textures added up together in one final texture.

So, is there anyway to achieve this? My first idea would be to pass a varying to the fragmentshader to calculate how much light comes into the dark area, but since I'll be processing this info on the final shadow, which has no vertex information, I'm completely lost about what approach I should use to make this.

I might be completely wrong on this approach, since I have very very limited experience with shaders.

Here is an example of what I have right now, and what I desire: What I have: Plain illumination withing the light radius (which causes the light to clip through walls.

What I want: Shadow is more intense the farther away it is from where the light ends.

Answer 1

I think that no matter what you do, you're going to have to calculate the light falloff differently for the areas of the scene that don't have direct visibility to the original light source. I'm not sure how you're actually applying the shadow volume to the scene, but you might be able to do something like classify the edges as you're generating the shadow volume (i.e. did they come from a wall or from the line from the light to a corner?), and treat them accordingly.

In short I don't think there's any clever shader-specific trick you can pull off to fix this problem. It's a limitation of the algorithm you're using, so you need to improve the algorithm to take into account the different nature of the edges in your shadow volume.

--- Edit ---

Ok I think your best bet is going to be to create two shadow volumes (or rather shadow areas since we're working in 2D). One will be exactly as you have now, the other will be smaller -- you'll want to exclude the areas that are in "soft" shadow. Then you'll have three categories in your fragment shader: total shadow, soft shadow, and unshadowed.

To create the second shadow map, I think you'll want to do something like add a fixed angle to the edges that are created by the light shining around a corner.

--- Edit #2 ---

I think you can solve the problem of the gap between objects B and C by taking the silhouette of the shadow areas and the outer box of the scene. So for each of your shadow areas, you'd find the two outermost points that intersect the outer box, then throw out all the line segments in between and replace them with that portion of the box. I feel like I should be able to name that algorithm but it escapes me at the moment...

Original Scene:

Individual "hard" shadows:

Now union the shadows together:

Finally, trace around the shadows, keeping to the edges of the box. The corners you want to identify are the ones in yellow. As you are tracing around the perimeter of the shadows, you want to cut across the edge of the box until you reach the opposite corner. Not the easiest thing to code but if you've gotten this far I think you can figure it out :)

Alternatively, it might be easier to simply consider the lit areas. For example, you could do something like take the difference of the scene box and the unioned shadowed area This will usually give you multiple polygons; discard everything except the one that contains the light because the rest are the false gaps.