add mouseevents to webgl objects

Question

im using xtk to visualize medical data in a webgl canvas. currently im playing around with this lesson:

lesson 10

this library is pretty good but not very well documented. i want to get rid of that gui and add some mouseevents. if i load the mesh from the gui how can i add a mouse event to the mesh? i actually don't know where to start. it's a little bit confusing to get started with this library....

i tried

mesh.click(function(){
    alert("yes");
  })

or

mesh.mousedown(function(){
    alert("yes");
}

Answer 1

Objects rendered in WebGL are not part of the DOM, and as such don't generate events like DOM elements do. This means that for events like these you have to implement the mouse interaction code yourself.

Traditionally in WebGL/OpenGL this process is known as "Picking", and there's several decent resources for it online. (For example: http://webgldemos.thoughtsincomputation.com/engine_tests/picking) The core process is something like this, though:

• For each pickable object in your scene, assign it a color. Put this in a lookup table somewhere
• Re-render the entire scene to a texture, rendering each pickable object with it's assigned color
• Once the scene is rendered, determine your mouse coordinates and read back the color of the texture at that X/Y.
• Fetch the object associated with that color from your lookup table. This is the object your mouse cursor is pointing at!

As you can see, while not a difficult method conceptually this also involves several mid-level WebGL topics, such as rendering to a texture, and as such is not usually recommended for beginners. I'm not sure if there are any features in xtk to assist with this (honestly I had never heard of the library before your post), but I would guess that this is something that you'll have to implement on your own.