Skinned mesh animation with collada

Demo: engine.xogames.org/demo_skinning/

During development of my just-for-fun game engine I needed to add support for skinned mesh animations.
I decided to take collada as intermediate format and make a script that converts collada into a custom binary format.

It was really a lot of work (mostly with collada parsing and exporting).

My plan was to make debug draw for the CPU skinning and then transfer it to the GPU. And it finally worked.

The must-read docs for collada format processing are spec and reference card.
There I found a nice formula that explains vertices calculation:

I premultiply vertices with bind-shape matrices during export.
The joints become a hierarchy of GameObjects within the engine. So objects can be easily attached to the joints just by adding them as a child.

I support only collada files with baked transforms into the matrices. Matrix then decomposed into translation, rotation quaternion and scale (TRS) and in this form is written into the file.
But here is the tricky part.

Here is the approach I use for getting rotation quaternion from the matrix.
It turned out that it works only for orthogonal matrices (actually it was specified in the doc but who reads it?) but joint matrices in collada export may be non-orthogonal (e.g. contain scale). I spent a lot of time trying to find out why I was getting a wrong matrix after conversion matrix -> TRS -> matrix.
Finally I was advised to normalize matrix basis before getting rotation quaternion. It solved the problem.


var pos = math.getMat4Translation(matrix);
var s = math.getMat4Scaling(matrix);
normalizeAxes(matrix);
var rot = math.getMat4Rotation(matrix);

// ........
normalizeAxes = function (m) {
    helperVec = vec3(m[0], m[1], m[2]);
    helperVec.normalize();
    m[0] = helperVec.x;
    m[1] = helperVec.y;
    m[2] = helperVec.z;

    helperVec = vec3(m[4], m[5], m[6]);
    helperVec.normalize();
    m[4] = helperVec.x;
    m[5] = helperVec.y;
    m[6] = helperVec.z;

    helperVec = vec3(m[8], m[9], m[10]);
    helperVec.normalize();
    m[8] = helperVec.x;
    m[9] = helperVec.y;
    m[10] = helperVec.z;
}

var pos = math.getMat4Translation(matrix);

var s = math.getMat4Scaling(matrix);

normalizeAxes(matrix);

var rot = math.getMat4Rotation(matrix);

// ........

normalizeAxes = function (m) {

helperVec = vec3(m[0], m[1], m[2]);

helperVec.normalize();

m[0] = helperVec.x;

m[1] = helperVec.y;

m[2] = helperVec.z;

helperVec = vec3(m[4], m[5], m[6]);

helperVec.normalize();

m[4] = helperVec.x;

m[5] = helperVec.y;

m[6] = helperVec.z;

helperVec = vec3(m[8], m[9], m[10]);

helperVec.normalize();

m[8] = helperVec.x;

m[9] = helperVec.y;

m[10] = helperVec.z;

}

Note that obtaining scale should happen before matrix normalization.

Moving vertex calculation to the GPU was pretty straightforward.


// Generic data

attribute vec2 aTexCoord0;
uniform vec3 uLightDir;
uniform mat4 uViewMatrix;
uniform mat4 uPMatrix;
varying vec3 vNormal_cameraspace;
varying vec3 vEyeDirection_cameraspace;
varying vec3 vLightDir_cameraspace;
varying vec2 vTexCoord0;

// Data related to skinning

const int MAX_JOINTS = 60;
const int JOINTS_PER_VERTEX = 3;

attribute vec3 aNormal;
attribute vec3 aPosition;
attribute vec3 aJointWeights;
attribute vec3 aJointIndexes;
uniform mat4 uJointTransforms[MAX_JOINTS];

void main(void) {
  vec4 position = vec4(0.0, 0.0, 0.0, 1.0);
  vec4 normal = vec4(0.0, 0.0, 0.0, 0.0);

  // Loop through all joins that affect current vertex
  for (int i = 0; i < JOINTS_PER_VERTEX; i++) {
    float jointWeight = aJointWeights[i];
    int jointIndex = int(aJointIndexes[i]);
    mat4 jointMatrix = uJointTransforms[jointIndex];
    position += jointMatrix * vec4(aPosition, 1.0) * jointWeight;
    normal += jointMatrix * vec4(aNormal, 0.0) * jointWeight; // w component is zero so translation will be skipped
  }

  normal = normalize(normal);
  vec4 position_cameraspace = uViewMatrix * vec4(position.xyz, 1.0);

  vNormal_cameraspace = (uViewMatrix * normal).xyz;
  vLightDir_cameraspace = normalize(vec3(uViewMatrix * vec4(uLightDir, 0.0)));
  vEyeDirection_cameraspace = vec3(0, 0, 0) - position_cameraspace.xyz;

  vTexCoord0 = aTexCoord0;
  gl_Position = uPMatrix * position_cameraspace;
}

// Generic data

attribute vec2 aTexCoord0;

uniform vec3 uLightDir;

uniform mat4 uViewMatrix;

uniform mat4 uPMatrix;

varying vec3 vNormal_cameraspace;

varying vec3 vEyeDirection_cameraspace;

varying vec3 vLightDir_cameraspace;

varying vec2 vTexCoord0;

// Data related to skinning

const int MAX_JOINTS = 60;

const int JOINTS_PER_VERTEX = 3;

attribute vec3 aNormal;

attribute vec3 aPosition;

attribute vec3 aJointWeights;

attribute vec3 aJointIndexes;

uniform mat4 uJointTransforms[MAX_JOINTS];

void main(void) {

vec4 position = vec4(0.0, 0.0, 0.0, 1.0);

vec4 normal = vec4(0.0, 0.0, 0.0, 0.0);

// Loop through all joins that affect current vertex

for (int i = 0; i < JOINTS_PER_VERTEX; i++) {

float jointWeight = aJointWeights[i];

int jointIndex = int(aJointIndexes[i]);

mat4 jointMatrix = uJointTransforms[jointIndex];

position += jointMatrix * vec4(aPosition, 1.0) * jointWeight;

normal += jointMatrix * vec4(aNormal, 0.0) * jointWeight; // w component is zero so translation will be skipped

}

normal = normalize(normal);

vec4 position_cameraspace = uViewMatrix * vec4(position.xyz, 1.0);

vNormal_cameraspace = (uViewMatrix * normal).xyz;

vLightDir_cameraspace = normalize(vec3(uViewMatrix * vec4(uLightDir, 0.0)));

vEyeDirection_cameraspace = vec3(0, 0, 0) - position_cameraspace.xyz;

vTexCoord0 = aTexCoord0;

gl_Position = uPMatrix * position_cameraspace;

}

Finally, I would say that I’m comfortable with collada but using SDKs such as FBX SDK may make things easier. Also native Maya collada export doesn’t support custom properties so OpenCollada plugin may be needed to make export from the 3D modelling tool.

Export script:
running example:
> node scripts/collada-convert.js resources/models/girl.dae
Script sources: https://github.com/Division/engine/blob/demo-skinning/scripts/collada-convert.js

Demo:
Play: engine.xogames.org/demo_skinning/
Demo sources: https://github.com/Division/engine/tree/demo-skinning

Running demo:
> npm install > npm run serve
References:

Share this: