WebGL/VR on Worker thread

WebGL on main thread

As before. Developing a WebGL application, the only approach is put all the stuff at the main thread. But it would definitely bring some limitations for the performance. As the above picture shows, in a 3D game, it might need to do lots of stuff in a update frame. For example, updating the transformation of 3D objects, visibility culling, AI, network, and physics, etc. Then, we finally can hand over it to the render process for executing WebGL functions.

If we expect it could done all things in the V-Sync time (16 ms), it would be a challenge for developers. Therefore, people look forward if there is another way to share the performance bottleneck to other threads. WebGL on worker thread is happened under this situation. But, please don't consider anything put into WebWorker would resolve your problems totally. It will bring us some new challenge as well. Following, I would like to tell you how to use WebGL on worker to increase the performance and give you a physics WebGL demo that is based on three.js and cannon.js, even proving that I can integrate it with WebVR API as well.

WebWorker

First of all, I would like to introduce how to use WebWorker. WebWorker can help you execute your script at another thread to avoid pauses from the JavaScript Virtual Machine’s Garbage Collector. Therefore, this is a good idea for developers to use WebWorker to solve the performance bottleneck issue. The sample code is like below:

worker = new Worker("js/worker.js"); // load worker script
worker.onmessage = function( evt ) { // The receiver of worker's message
    //console.log('Message received from worker ' + evt.data );
};

worker.postMessage( { test: 'webgl_offscreen'); // Send message to worker

In worker.js

onmessage = function(evt) {
    //console.log( 'Message received from main script' );
    postMessage( 'Send script to the main script.' ); // Post message back to the main thread.
}

These script looks quite simple and we can start to put some computation at onmessage function in worker.js to relief the work of the main thread. However, we have to know WebWorker brings some constraints for us as well, it would make us feel inconvenient compare to the general JavaScript usage at the main thread. The limitation of WebWorker are: 

 - Can't read/write DOM

 - Can't access global variable / function

 - Can't use file system (file://) to access local files

 - No requestAnimationFrame

WebGL on worker

After understanding how to use WebWorker and its constraints. Let's start to make our first WebGL on worker application.

The benefit of worker is we can put a part of tiny computation functions into another thread. In case of WebGL worker, we can put WebGL function calls into the Worker thread. So in the example of the above picture, I put my render part to the WebGL Worker.  Firefox Nightly has landed offscreencanvas feature for supporting WebGL on worker thread. In order to utilize this feature, we need to do some setup:

• Download Firefox Nightly
• Enter about:config, make gfx.offscreencanvas.enabled;true
• Now, it works in Chrome instead of Firefox

Then, we have activated WebGL Worker. Go to finish it! The sample code is like below.

var canvas = document.getElementById('c');
canvas.width = window.innerWidth;
canvas.height = window.innerHeight;

var proxy = canvas.transferControlToOffscreen();   // new interface added by offscreencanvas for getting offscreen canvas
var worker = new Worker("js/gl_worker.js");
var positions = new Float32Array(num*3);           // Transferable object of web worker. Transformation info
var quaternions = new Float32Array(num*4);         // For the update/render functions to update their variable.
                                                   // in the main/worker threads.
var cameraState = new Float32Array(7);             // Camera state for the update/render functions

worker.onmessage = function( evt ) {               // worker message receiving function
    if ( evt.data.positions && evt.data.quaternions
    && evt.data.cameraState ) {
    
      positions = evt.data.positions;
      quaternions = evt.data.quaternions;
      cameraState = evt.data.cameraState;
      updateWorker();
    }
}

worker.postMessage( { canvas: proxy }, [proxy]);    // Send offscreenCanvas to worker

function updateWorker() {
    // Update camera state

    // Update position, quaternion

    // Send these buffer back the worker
    worker.postMessage( { cameraState: cameraState, positions: positions, quaternions: quaternions }, 
    [cameraState.buffer, positions.buffer, quaternions.buffer]);
}

In worker.js

var renderer;
var canvas;
var scene = null;
var camera = null;

onmessage = function( evt ) {      // Receiving messages from the main thread
  var window = self;
  
  if ( typeof evt.data.canvas !== 'undefined') {
    console.log( 'import script... ' );
    importScripts('../lib/three.js');             // load script at worker
    importScripts('../js/threejs/VREffect.js');
    importScripts('../js/threejs/TGALoader.js');

    canvas = evt.data.canvas;
    renderer = new THREE.WebGLRenderer( { canvas: canvas } ); // Initialize THREE.js WebGLRenderer
    scene = new THREE.Scene();
    camera = new THREE.PerspectiveCamera( 30, canvas.width / canvas.height, 0.5, 10000 );

    window.addEventListener( 'resize', onWindowResize, false ); // Register 'resize' event

    // Get bufffers that are sent from main thread.
    var cameraState = evt.data.cameraState;
    var positions = evt.data.positions;
    var quaternions = evt.data.quaternions;
    camera.position.set( cameraState[0], cameraState[1], cameraState[2] );
    camera.quaternion.set( cameraState[3], cameraState[4], cameraState[5], cameraState[6] );

    for ( var i = 0; i < visuals.length; i++ ) {    // Setup transformation info for visual objects in scene
      visuals[i].position.set(
        positions[3 * i + 0],
        positions[3 * i + 1],
        positions[3 * i + 2] );

      visuals[i].quaternion.set(
        quaternions[4 * i + 0],
        quaternions[4 * i + 1],
        quaternions[4 * i + 2],
        quaternions[4 * i + 3] );
    }

    render();        // Call render via the main thread requestAnimationTime

    postMessage({ cameraState: cameraState, positions:positions, quaternions:quaternions}
  , [ cameraState.buffer, positions.buffer, quaternions.buffer ]);  // Send back transferable 
                                                                                  // object to the main thread
  }
}

function render() {
    renderer.render( scene, camera );
    renderer.context.commit();       // New for webgl worker to end this frame (only support in Firefox. Running in Chrome, we need mark this line.)
}

function onWindowResize( width, height ) {  // Resize window listener
  canvas.width = width;
  canvas.height = height;
  camera.aspect = canvas.width / canvas.height;
  camera.updateProjectionMatrix();
  renderer.setSize( canvas.width, canvas.height, false );
}

WebVR on Worker

Although most parameters of WebVR exist at dom API, the worker thread can't get them directly. But it is not a big deal, we can get them at the main thread and pass them to the worker.

In the main thread

var vrHMD;
function gotVRDevices( devices ) {
vrHMD = devices[ 0 ];
worker.postMessage( {        // Pass them to the worker
    eyeTranslationL: eyeTranslationL.x, 
    eyeTranslationR: eyeTranslationR.x, 
    eyeFOVLUp: eyeFOVL.upDegrees, eyeFOVLDown: eyeFOVL.downDegrees, 
    eyeFOVLLeft: eyeFOVL.leftDegrees, eyeFOVLRight: eyeFOVL.rightDegrees, 
    eyeFOVRUp: eyeFOVR.upDegrees, eyeFOVRDown: eyeFOVR.downDegrees, 
    eyeFOVRLeft: eyeFOVR.leftDegrees, eyeFOVRRight: eyeFOVR.rightDegrees });
}

function updateVR() {       // Update camera orientation via VR state
  var state = vrPosSensor.getState();

  if ( state.hasOrientation ) {
    camera.quaternion.set(
      state.orientation.x, 
      state.orientation.y, 
      state.orientation.z, 
      state.orientation.w);
}

function triggerFullscreen() {
    canvas.mozRequestFullScreen( { vrDisplay: vrHMD } );  // Fullscreen must be requested at the main thread.
}                                                         // Thankfully, it works for WebGL on worker.

In worker.js

var vrDeviceEffect = new THREE.VREffect(renderer);

onmessage = function(evt) {                // Send VRDevice to work for stereo render.
    vrDeviceEffect.eyeTranslationL.x = evt.data.eyeTranslationL;
    vrDeviceEffect.eyeTranslationR.x = evt.data.eyeTranslationR;
    vrDeviceEffect.eyeFOVL.upDegrees = evt.data.eyeFOVLUp;
    vrDeviceEffect.eyeFOVL.downDegrees = evt.data.eyeFOVLDown;
    vrDeviceEffect.eyeFOVL.leftDegrees = evt.data.eyeFOVLLeft;
    vrDeviceEffect.eyeFOVL.rightDegrees = evt.data.eyeFOVLRight;
    vrDeviceEffect.eyeFOVR.upDegrees = evt.data.eyeFOVRUp;
    vrDeviceEffect.eyeFOVR.downDegrees = evt.data.eyeFOVRDown;
    vrDeviceEffect.eyeFOVR.leftDegrees = evt.data.eyeFOVRLeft;
    vrDeviceEffect.eyeFOVR.rightDegrees = evt.data.eyeFOVRRight;
}

Others

Besides WebGL and WebVR, some problems that are solved when I made this demo. I list them and discuss how I solve them:
  - Can’t access DOM (read/modify)

    var workerCanvas = canvas.transferControlToOffscreen();
    worker.postMessage( {canvas: workerCanvas}, [workerCanvas] );

  - Can’t use filesystem (file://) to access local files
    Use XMLHttpRequest. Taking load texture as an example, in three.js, we need to use

var loader = new THREE.TGALoader();
var texture = loader.load( 'images/brick_bump.tga' );
var solidMaterial = new THREE.MeshLambertMaterial( { map: texture } );

  - No requestAnimationFrame
    Updating transferable objects and render need to via worker.onmessage(), we have to execute the worker update at the main reqestAnimationFrame. This limitation would bring the chance of blocking by the main thread requestAnimationFrame because it possibly would happen GC pauses in the main thread and block the worker thread. The best solution is by looking forward the implementation of requestAnimationFrame for Worker.

Demo

Physics/WebGL on the main thread
Physics on the main thread, WebGL on worker
Source code