How do I manipulate shadows in Three.js without editing the underlying mesh?

Question

I'm working on an app that should allow users to manipulate 3D objects in the scene and observe how their changes affect the ground shadow:

In this scene, the yellow cylinder casts a shadow on a white plane with the middle of the cylinder contained in the green cube. What I would like to happen is for the cube to remove the middle of the shadow, like so:

Obviosly, my first thought was to subtract the green cube volume from the yellow cylinder volume and after a bit of googling I found CSG.js. Unfortunately, CSG.js is too slow for the actual model that I'm going to use, which will going to have at least 15k vertices.

I started digging into the Three.js source and reading about shadow maps to understand how shadows are produced, but my shader-fu is not strong enough yet to fully grasp how I can tweak shadow rendering.

How can I achieve this "shadow subtraction" effect?

var camera, scene, renderer;

init();
animate();

function init() {
  scene = new THREE.Scene();

  camera = new THREE.PerspectiveCamera( 70, window.innerWidth / window.innerHeight, 1, 1000 );
  camera.position.z = 500;
  camera.position.y = 100;
  camera.lookAt(scene.position);

  var ambient = new THREE.AmbientLight(0x909090);
  scene.add(ambient);

  var directionalLight = new THREE.DirectionalLight( 0xffffff, 1.0 );
  directionalLight.position.set( -300, 300, 0 );
  directionalLight.castShadow = true;
  directionalLight.shadow.camera.near    =   10;
  directionalLight.shadow.camera.far     =   2000;
  directionalLight.shadow.camera.right   =   350;
  directionalLight.shadow.camera.left    =  -350;
  directionalLight.shadow.camera.top     =   350;
  directionalLight.shadow.camera.bottom  =  -350;
  directionalLight.shadow.mapSize.width  = 1024;
  directionalLight.shadow.mapSize.height = 1024;
  scene.add( directionalLight );

  //var lightHelper = new THREE.CameraHelper(directionalLight.shadow.camera);
  //scene.add(lightHelper);

  var geometry = new THREE.CylinderGeometry( 50, 50, 400, 32 );
  var material = new THREE.MeshPhongMaterial( {color: 0xffff00} );
  var cylinder = new THREE.Mesh( geometry, material );
  cylinder.castShadow = true;
  scene.add( cylinder );

  var geometry = new THREE.BoxGeometry( 110, 110, 110 );
  var material = new THREE.MeshPhongMaterial( {color: 0x00ff00} );
  var cube = new THREE.Mesh( geometry, material );
  cube.castShadow = true;
  scene.add( cube );

  var geometry = new THREE.PlaneGeometry( 3000, 3000, 32 );
  var material = new THREE.MeshPhongMaterial( {color: 0xffffff, side: THREE.DoubleSide} );
  var plane = new THREE.Mesh( geometry, material );
  plane.lookAt(new THREE.Vector3(0, 1, 0));
  plane.position.y = -200;
  plane.receiveShadow = true;
  scene.add( plane );

  renderer = new THREE.WebGLRenderer();
  renderer.setPixelRatio( window.devicePixelRatio );
  renderer.setSize( window.innerWidth, window.innerHeight );
  renderer.shadowMap.enabled = true;
  renderer.shadowMap.type = THREE.BasicShadowMap;
  document.body.appendChild( renderer.domElement );

  window.addEventListener( 'resize', onWindowResize, false );
}

function onWindowResize() {
  camera.aspect = window.innerWidth / window.innerHeight;
  camera.updateProjectionMatrix();

  renderer.setSize( window.innerWidth, window.innerHeight );
}

function animate() {
  requestAnimationFrame( animate );

  renderer.render( scene, camera );
}

jsFiddle

Update:

What about a more complicated scene? Is it possible for the shadow from the red cylinder to be unaffected (you can see it being cut in half with cube.customDepthMaterial = new THREE.MeshBasicMaterial({ depthTest: false}))?

Updated jsFiddle

Answer 1

You can subtract an object's shadow from the rest of scene by setting the object's .customDepthMaterial property like so:

var cube = new THREE.Mesh( geometry, material );
cube.castShadow = true;
cube.receiveShadow = false;

// The secret sauce
cube.customDepthMaterial =
  new THREE.MeshBasicMaterial({ depthTest: false});

scene.add( cube );

jsFiddle

No shader-fu required.

Why This Works
When the shadow map is rendered, each object's depth material ( .customDepthMaterial or the default ) is used to render the scene from the light's perspective. The depth material's resulting render represents the object's depth from the camera packed as RGBA. Since THREE.MeshBasicMaterial defaults to { color: 0xffffff, opacity: 1 }, it will return the maximum depth which makes the object further than the shadow camera's far.

I disabled depthTest because in your desired result screenshot you clipped the area where the cube's given the cylinder wasn't there. Disabling depthTest means that parts of the cube which are blocked by the cylinder will still cut out the shadow, giving you your desired result.

Documentation
There unfortunately is no documentation on .customDepthMaterial yet but I did find an official example where it is used.

Updated Answer:
To allow an object's shadow to always show:

1. You can use the same trick as above just setting the material's color and opacity to 0
2. Make sure it's added to the scene after the 'subtractive shadow' object. This way the additive shadow will win out even though they both have depthTest disabled.

updated jsFiddle

If you have anything more complicated, it will be up to you to figure out a way to manage the order of the shadow rendering.

Tested in r77