WebGL 3D Engine from Scratch Part 10: Specular Lighting

So far we've covered diffuse lighting at poly, vertex and pixel levels. This gives a general lighting that estimates the way light scatters from a surface in all directions. To make other materials, particularly shiny materials we need to have a concept of light reflection.

For example here's an example of a gold Christmas ornament:

Notice that unlike our existing spheres it doesn't just have a gradient color but also bright spots where the light from the scene are reflected back into the camera. This is what specular lighting is trying to emulate.

Phong

The most classical way to do this is with Phong lighting. It's not too different from what we were doing with diffuse lighting but it does require an extra component, the camera location. So we'll set this up with our uniforms in getGlobalUniforms:

const cameraPosition = this.cameras.default.getPosition();
const cameraLocation = this.context.getUniformLocation(program, "uCamera");
this.context.uniform3fv(cameraLocation, Float32Array.from(cameraPosition));

And in the fragment shader we can access it:

uniform mediump vec3 uCamera;

And then test it by outputting the position as a color. If the camera is at 2, 0, 0 then the object should be red.

Here's what we do:

1) Get the direction (normalized) vector to the camera
2) Get the direction to the light source
3) Get the reversed direction to the light source (from light) and reflect it against the surface normal.
4) Dot the direction to the camera with the reflected vector to get the magnitude of light going towards the camera.
5) Multiply by color

Steps 1 and 2 are fairly trivial, you just do some subtraction.

Reflection involves a bit of math. GLSL is kind enough to give us this out-of-the-box reflect(vector, normal) but if you need to calculate it yourself it's:

export function reflect(vec, normal){
    return [
        vec[0] - 2 * dotVector(vec, normal) * normal[0],
        vec[1] - 2 * dotVector(vec, normal) * normal[1],
        vec[2] - 2 * dotVector(vec, normal) * normal[2],
    ];
}

And the rest is straightforward.

Using the existing per-pixel lighting shader we can modify it to add specular. In the end we get the following shader (we're only showing the specular part of light for now, the diffuse is commented out):

precision mediump float;

varying vec4 vColor;
varying vec2 vUV;
varying vec3 vNormal;
varying vec3 vPosition;

uniform mat4 uLight1;
uniform sampler2D uSampler;
uniform vec3 uCamera;

void main() {
    bool isPoint = uLight1[3][3] == 1.0;
    vec3 normal = normalize(vNormal);

    if(isPoint) {
        //point light + color
        vec3 toLight = normalize(uLight1[0].xyz - vPosition);
        float light = dot(normal, toLight);

        vec3 toCameraDir = normalize(uCamera - vPosition);
        vec3 reflectedLightDir = reflect(-toLight, normal);
        float specularLight = dot(reflectedLightDir, toCameraDir);

        gl_FragColor = vec4(specularLight, specularLight, specularLight, 1.0);
        //gl_FragColor = vColor * uLight1[2] * vec4(light, light, light, 1);
    } else {
        //directional light + color
        float light = dot(normal, uLight1[1].xyz);
        gl_FragColor = vColor * uLight1[2] * vec4(light, light, light, 1);
    }
}

Note that I also added a global precision and pre-normalized vNormal to make things simpler.

If we take a look it seems to produce a light spot that varies based on how we look at it.

Glossiness

While we have specular lighting now, we'd like to be able to fine-tune it a bit. That is, we want a parameter that will change exactly how shiny or matte it looks. A typical way to do this is by introducing gloss value and then taking the magnitude of the specular light to that power. I guess this is because the scale is more exponential.

In order to pipe in the gloss parameter, we'll make some changes to material.js in order to take in those extra parameters.

//material.js
export class Material {
    #program;
    #textures;
    #uniforms;

    constructor(material){
        this.#program = material.program;
        this.#textures = material.textures ?? [];
        this.#uniforms = material.uniforms;
    }

    get program(){
        return this.#program;
    }

    get textures(){
        return this.#textures;
    }

    get uniforms(){
        return this.#unforms;
    }
}

It's really just an interface. uniforms will just be an object with key equal to the name and value equal to the value of the uniform. We'll also set it up so that we don't need to worry about the size of the uniforms. For this I have a small function:

export function autoBindUniform(context, program, uniformName, value){
    const location = context.getUniformLocation(program, uniformName);
    if (!location) return;
    if (Array.isArray(value)) {
        switch (value.length) {
            case 1: {
                this.context.uniform1fv(location, value);
            }
            case 2: {
                this.context.uniform2fv(location, value);
            }
            case 3: {
                this.context.uniform3fv(location, value);
            }
            case 4: {
                this.context.uniform4fv(location, value);
            }
            default: {
                console.error(`Invalid dimension for binding uniforms. ${uniformName} with value of length ${value.length}`);
            }
        }
    } else {
        context.uniform1f(location, value);
    }
}

This could be expanded to matrices as well. Now in bindMaterial we just add a new line to bind all the associated uniform parameters:

if(material.uniforms){
  Object.entries(material.uniforms).forEach(([uniformName, uniformValue]) => {
    autoBindUniform(this.context, material.program,uniformName, uniformValue);
  });
}

And we can update the material like so:

pixelShadedSpecular: new Material({
    program: await loadProgram(this.context, "shaders/pixel-shaded-specular"),
    uniforms: {
        gloss: 4.0
    }
}),

And this will let us easily change the value. And finally the shader:

precision mediump float;

varying vec4 vColor;
varying vec2 vUV;
varying vec3 vNormal;
varying vec3 vPosition;

uniform mat4 uLight1;
uniform sampler2D uSampler;
uniform vec3 uCamera;
uniform float gloss;

void main() {
    bool isPoint = uLight1[3][3] == 1.0;
    vec3 normal = normalize(vNormal);

    if(isPoint) {
        //point light + color
        vec3 toLight = normalize(uLight1[0].xyz - vPosition);
        float light = dot(normal, toLight);

        vec3 toCameraDir = normalize(uCamera - vPosition);
        vec3 reflectedLightDir = reflect(-toLight, normal);
        float baseSpecular = clamp(dot(reflectedLightDir, toCameraDir), 0.0, 1.0);
        float specularLight = pow(baseSpecular, gloss);

        gl_FragColor = (uLight1[2] * vec4(specularLight, specularLight, specularLight, 1.0)) + (vColor * uLight1[2] * vec4(light, light, light, 1));
    } else {
        //directional light + color
        float light = dot(normal, uLight1[1].xyz);

        vec3 toCameraDir = normalize(uCamera - vPosition);
        vec3 reflectedLightDir = reflect(-uLight1[1].xyz, normal);  
        float baseSpecular = clamp(dot(reflectedLightDir, toCameraDir), 0.0, 1.0);
        float specularLight = pow(baseSpecular, gloss);

        gl_FragColor = (uLight1[2] * vec4(specularLight, specularLight, specularLight, 1.0)) + (vColor * uLight1[2] * vec4(light, light, light, 1));
    }
}

Note that clamp was added to baseSpecular without this you'll get weird behavior where light is negative. To deal with the final color output we'll just add the two lights together. For directional lights we just use the direction.

You can see the light get very blown out. This works well enough for spheres and such but less well for flat surfaces.

In this case the highlight piece is still circular.

Blinn-Phong

A variation of Phong that helps fix the flat surface issues is called Blinn-Phong. It's calculated somewhat differently and a bit simpler.

1) Get the direction (normalized) vector to the camera
2) Get the half vector, which is toLight + toCamera (normalized)
3) Dot the half vector with the normal to get the magnitude of light going towards the camera.
4) Multiply by color

It's not especially clear to me why this works but it does. Here's the code:

precision mediump float;

varying vec4 vColor;
varying vec2 vUV;
varying vec3 vNormal;
varying vec3 vPosition;

uniform mat4 uLight1;
uniform sampler2D uSampler;
uniform vec3 uCamera;
uniform float gloss;

void main() {
    bool isPoint = uLight1[3][3] == 1.0;
    vec3 normal = normalize(vNormal);

    if(isPoint) {
        //point light + color
        vec3 toLightDir = normalize(uLight1[0].xyz - vPosition);
        float light = dot(normal, toLightDir);

        vec3 toCameraDir = normalize(uCamera - vPosition);
        vec3 halfVector = normalize(toLightDir + toCameraDir);
        float baseSpecular = clamp(dot(halfVector, normal), 0.0, 1.0);
        float specularLight = pow(baseSpecular, gloss);

        gl_FragColor = (uLight1[2] * vec4(specularLight, specularLight, specularLight, 1.0)) + (vColor * uLight1[2] * vec4(light, light, light, 1));
    } else {
        //directional light + color
        float light = dot(normal, uLight1[1].xyz);

        vec3 toCameraDir = normalize(uCamera - vPosition);
        vec3 halfVector = normalize(uLight1[1].xyz + toCameraDir);
        float baseSpecular = clamp(dot(halfVector, normal), 0.0, 1.0);
        float specularLight = pow(baseSpecular, gloss);

        gl_FragColor = (uLight1[2] * vec4(specularLight, specularLight, specularLight, 1.0)) + (vColor * uLight1[2] * vec4(light, light, light, 1));
    }
}

At glancing angles the shape of the highlight is a little more stretched which is what we want.

The sphere looks identical.

Defects

Blinn-Phong has a small defect. If we rotate the sphere around enough we see the the specular highlight can actually clip through to the backside:

We can fix this by making sure that we do not calculate a specular light value if the light itself is facing away from the surface:

float baseSpecular = clamp(dot(halfVector, normal), 0.0, 1.0) * float(light > 0.0);

That is if the dot product between the light and the surface is negative the whole thing becomes zero.

It's hard to tell what this is, but it's at the same angle as the last one but without the weird defect.

Specular Mapping

One other thing we can is instead of making the entire material one value of specular, we can do it per-texel. To start, I created a cube using a copy of the textured fragment shader:

The Texture itself I exported from Wolfenstien 3D (if you want to know a little more about how I did this you can read my blog about building a Rise of the Triad source port: https://github.com/ndesmic/webrott/blob/master/part9/walls3.md).

This is because I wanted something small and easy to edit by hand without complex tools. What we'll try to do is make the slime look shiny and wet and the stones will have a slight sheen. This is not an exact science at all given that, as good pixel art, the use of color blending makes it hard to tell exactly what thing is what. After some manual editing with MS paint:

A greyscale png slimewall.specmap.png. First let's add diffuse lighting to the texture:

precision mediump float;

varying vec4 vColor;
varying vec2 vUV;
varying vec3 vNormal;
varying vec3 vPosition;
uniform mat4 uLight1;
uniform sampler2D uSampler;

void main() {
    bool isPoint = uLight1[3][3] == 1.0;
    if(isPoint) {
        //point light + color
        vec4 color = texture2D(uSampler, vUV);
        mediump vec3 toLight = normalize(uLight1[0].xyz - vPosition);
        mediump float light = dot(normalize(vNormal), toLight);
        gl_FragColor = color * uLight1[2] * vec4(light, light, light, 1);
    } else {
        //directional light + color
        vec4 color = texture2D(uSampler, vUV);
        mediump float light = dot(normalize(vNormal), uLight1[1].xyz);
        gl_FragColor = color * uLight1[2] * vec4(light, light, light, 1);
    }
}

It's just swapping the color for the sampled texel color.

Using multiple textures

Some small changes are necessary to use multiple textures. First, we need to actually load them:

specMapped: new Material({
    program: await loadProgram(this.context, "shaders/spec-mapped"),
    textures: [
        await loadTexture(this.context, "./img/slimewall.png"),
        await loadTexture(this.context, "./img/slimewall.specmap.png")
    ]
})

Note that this could be parallelized but I was kinda lazy. Next we'll modify the texture binding in bindMaterial:

bindMaterial(material){
    this.context.useProgram(material.program);
    material.textures.forEach((tex, index) => {
        const location = this.context.getUniformLocation(material.program, `uSampler${index}`);
        if(!location) return;
        this.context.uniform1i(location, index);
        this.context.activeTexture(this.context[`TEXTURE${index}`]);
        this.context.bindTexture(this.context.TEXTURE_2D, tex);
    });
    if(material.uniforms){
        Object.entries(material.uniforms).forEach(([uniformName, uniformValue]) => {
            autoBindUniform(this.context, material.program, uniformName, uniformValue);
        });
    }
}

We've been using defaults for the sampler up until now. This time we'll actually bind the uniform. It's like any other uniform bind. It's a single integer representing the texture unit and if you remember from the chapter on textures there can be many, up-to 32. So all we're doing is just saying which texture slot index the sampler should sample from (default was 0). There's also a shortcut in case the shader doesn't actually use it (location == null). Next we set the active texture, that is, the slot we want to bind to. The have enum names like TEXTURE0 and TEXTURE1 so we do some string concatenation to line them up. Then we bind, which is saying "associate this texture with the current slot." Once all that is done, using uSamplerX where X is a number between 0 and 32 will use the texture in that slot and the textures are bound in the order they are specified on the material. Not the most semantic way to do it, but it works.

---

Back in shader-land, we can use two samplers:

uniform sampler2D uSampler0;
uniform sampler2D uSampler1;

A good way to test it's working is to flip back and forth between them. uSampler0 in this case is the slimewall texture and uSampler1 is the specular map.

Now we can do all sorts of cool things as we can add new information per-texel. In this case the specular map contains the specularity value of that texel (that is, how much specular applies). Keep in mind the specular map is a greyscale png, which means we still have 4 color channels but they are all duplicated. Not very efficient, but it'll do for now. We can just take the red component for simplicity.

precision mediump float;

varying vec4 vColor;
varying vec2 vUV;
varying vec3 vNormal;
varying vec3 vPosition;
uniform mat4 uLight1;
uniform vec3 uCamera;
uniform float gloss;
uniform sampler2D uSampler0;
uniform sampler2D uSampler1;

void main() {
    bool isPoint = uLight1[3][3] == 1.0;
    vec3 normal = normalize(vNormal);

    if(isPoint) {
        //point light + color
        vec4 color = texture2D(uSampler0, vUV);
        float specularity = texture2D(uSampler1, vUV).r;
        vec3 toLightDir = normalize(uLight1[0].xyz - vPosition);
        float diffuseMagnitude = dot(normal, toLightDir);
        vec4 diffuseLight = color * uLight1[2] * vec4(diffuseMagnitude, diffuseMagnitude, diffuseMagnitude, 1);

        vec3 toCameraDir = normalize(uCamera - vPosition);
        vec3 halfVector = normalize(toLightDir + toCameraDir);
        float baseSpecular = clamp(dot(halfVector, normal), 0.0, 1.0) * float(clamp(diffuseMagnitude, 0.0, 1.0) > 0.0);
        float specularMagnitude = pow(baseSpecular, gloss);
        vec4 specularLight = uLight1[2] * specularity * vec4(specularMagnitude, specularMagnitude, specularMagnitude, 1.0);

        gl_FragColor = diffuseLight + specularLight;
    } else {
        //directional light + color
        vec4 color = texture2D(uSampler0, vUV);
        float specularity = texture2D(uSampler1, vUV).r;
        float diffuseMagnitude = dot(normal, uLight1[1].xyz);
        vec4 diffuseLight = color * uLight1[2] * vec4(diffuseMagnitude, diffuseMagnitude, diffuseMagnitude, 1);

        vec3 toCameraDir = normalize(uCamera - vPosition);
        vec3 halfVector = normalize(uLight1[1].xyz + toCameraDir);
        float baseSpecular = clamp(dot(halfVector, normal), 0.0, 1.0);
        float specularMagnitude = pow(baseSpecular, gloss);
        vec4 specularLight = uLight1[2] * specularity * vec4(specularMagnitude, specularMagnitude, specularMagnitude, 1.0);

        gl_FragColor = specularLight + diffuseLight;
    }
}

What you'll notice is that I added an extra term when deciding the specularLight, the color is multiplied by this amount and it comes from the specular map. This mean dark areas (0) will not apply specular and bright areas will.

Well, that's not exactly what I was expecting but it is sort of cool. Instead of wet slime its a bit more like kintsugi.

Fixing Texture orientation

One other thing to note is that the texture is up-side-down. This is a disagreement between web and OpenGL on if images are bottom up or top-down. Since the normal web paradigm is top-down we can add the following to bootGpu:

this.context.pixelStorei(this.context.UNPACK_FLIP_Y_WEBGL, true);

To orient the textures the other direction.

Glossmaps

Glossmaps are the same thing as specular maps, they just contain the specular exponent data (what we've been calling gloss) instead. These can also be called roughness maps with roughness being the opposite of gloss (the specular highlight is more spread out).

For this I'll just try a colored quad with a glossmap that's a linear gradient from 0.0 to 1.0.

The shader:

precision mediump float;

varying vec4 vColor;
varying vec2 vUV;
varying vec3 vNormal;
varying vec3 vPosition;
uniform mat4 uLight1;
uniform vec3 uCamera;
uniform float specularity;
uniform sampler2D uSampler0;

void main() {
    bool isPoint = uLight1[3][3] == 1.0;
    vec3 normal = normalize(vNormal);

    if(isPoint) {
        //point light + color
        float gloss = exp2(texture2D(uSampler0, vUV).r * 6.0) + 2.0;
        vec3 toLightDir = normalize(uLight1[0].xyz - vPosition);
        float diffuseMagnitude = dot(normal, toLightDir);
        vec4 diffuseLight = vColor * uLight1[2] * vec4(diffuseMagnitude, diffuseMagnitude, diffuseMagnitude, 1);

        vec3 toCameraDir = normalize(uCamera - vPosition);
        vec3 halfVector = normalize(toLightDir + toCameraDir);
        float baseSpecular = clamp(dot(halfVector, normal), 0.0, 1.0) * float(clamp(diffuseMagnitude, 0.0, 1.0) > 0.0);
        float specularMagnitude = pow(baseSpecular, gloss);
        vec4 specularLight = uLight1[2] * specularity * vec4(specularMagnitude, specularMagnitude, specularMagnitude, 1.0);

        gl_FragColor = diffuseLight + specularLight;
    } else {
        //directional light + color
        float gloss = exp2(texture2D(uSampler0, vUV).r * 6.0) + 2.0;
        float diffuseMagnitude = dot(normal, uLight1[1].xyz);
        vec4 diffuseLight = vColor * uLight1[2] * vec4(diffuseMagnitude, diffuseMagnitude, diffuseMagnitude, 1);

        vec3 toCameraDir = normalize(uCamera - vPosition);
        vec3 halfVector = normalize(uLight1[1].xyz + toCameraDir);
        float baseSpecular = clamp(dot(halfVector, normal), 0.0, 1.0);
        float specularMagnitude = pow(baseSpecular, gloss);
        vec4 specularLight = uLight1[2] * specularity * vec4(specularMagnitude, specularMagnitude, specularMagnitude, 1.0);

        gl_FragColor = specularLight + diffuseLight;    
    }
}

This is almost the same as the specular shader but gloss has been swapped with specularity (specularity is now a uniform). One specific thing I did was remap the gloss value. Since gloss is an exponent but we want things in a general 0 - 1 range it's been mapped exponentially. The exact formula I used comes from Freya Holmer as a recommendation for how to do it: https://www.youtube.com/watch?v=mL8U8tIiRRg&t=11831s.

The exact effect isn't too easy to intuit in such a simple test but it works. You'll notice the right side is more polished.

You can of course combine specular maps and glossmaps in the same shader. You might also see normal textures called "diffuse maps" because they pretty directly determine color for diffuse lighting.

That's quite a lot more than I expected. Hope it helps and see you next time! You can find the code here: https://github.com/ndesmic/geogl/tree/v7