I remember back in the PS2, Gamecube, Xbox days gaming enthusiasts were crazy about bump-mapping and normal mapping. Specifically, the term "bump-mapping" jumps out as a popular buzz-word and while I had an intuition for what it was, I finally took a look at how they actually worked. These are special textures that change the way light hits a surface to approximate detailed geometry. The closer and wider the angle you look at it, the more off it'll be because it's still a flat texture but it's still a cheap way to make things look a lot more detailed without adding more geometry.
Bump-maps specifically refer to a single-color texture that contains height information. These can also be called "height-maps." The closely related "normal-maps" are 3 color textures that tell which way a surface is pointing. These are more commonly used the graphics today as they essentially pre-compute the information you actually want from the bump-map, the normals. Overall bump-mapping requires more calculation in the pixel shader and is a little less precise as the trade-off for taking up 1/3 less texture memory but they are more-or-less different representations of the same thing.
Creating a bump map
Normally this would be done in some sort of program that could convert geometry into a height map (basically taking a camera looking at the model from the top and recording the heights). However since I'm mostly using my own simple tools I'm just going to make one from a pattern instead as an example. Using the radial gradient code from a previous project (https://dev.to/ndesmic/circular-gradients-from-scratch-4hgg) we can generate a simple bump-map by making a radial gradient with black white stops:
We also should define what the values mean. 0 (black) is a low spot and 1 (white) is a high spot. Exactly how high at the max can be determined by a parameter.
Converting to GLSL 300
Previously we've been using GLSL 100 which is the default. I'd like to start using version 300. To do this we need to add #version 300 es to the very first line of both the vertex and fragment shader. This will also change the syntax so if we wanted to convert existing shaders we need to make some changes to them.
You can find a list of changes here: https://webgl2fundamentals.org/webgl/lessons/webgl1-to-webgl2.html#switch-to-glsl-300-es
Tangent Space
In order to use the bump-map we need to know how it's orientation relates to world-space. That is, as we move across the texture along say the U-axis, which direction is that exactly? We know that elevation points the normal direction (outward) but we need to pick 2 basis vectors for the other 2 dimensions. These must lay in a plane that is orthogonal to the normal but we have infinitely many choices of which direction they are oriented.
What we want is for the bump-map to be oriented the same way as the texture so they line up, therefore we need to align the other two vectors so that they match up with the U and V directions but in world-space. Specifically, tangent will line up with U and bitangent with V. To find the tangent for some triangle we can use the change in texture-space position (UV) in relation to world-space position (XYZ) to find the basis vectors.
We can think of moving along an edge of the triangle, we'll move some amount of U and V. We have 2 edges P0 -> P1 and P0 -> P2 that describe the triangle and we'll need both (I believe the 2 chosen edges aren't important as long as they are different but this is the convention I found). The change in UV between vertices will be the same as moving from the first vertex to the second vertex in world space. That is (here's comes the maths...):
variables:
T = tangent vector (3d)
B = bitangent vector (3d)
E1 = edge 1 vector, p0 -> p1 (3d)
E2 = edge 2 vector, p0 -> p2 (3d)
u0 = u-coordinate of vertex 0 in a triangle
v0 = v-coordinate of vertex 0 in a triangle
u1 = u-coordinate of vertex 1 in a triangle
v1 = v-coordinate of vertex 1 in a triangle
remember uppercase are vectors, lowercase are scalars!
equations:
E1 = (u1 - u0)T + (v1-v0)B
E2 = (u2 - u0)T + (v2-v0)B
E1 = P1 - P0
E2 = P2 - P0
The left-hand side are the 2 edges, that is E1 = P1 - P0 and E2 = P2 - P0. The right-hand side says a change in U in tangent direction plus a change in V in bitangent direction will be the same as moving across that edge. This is because as long as B and T are orthogonal, any point in the plane can be described as a linear combination of the two vectors. We can rewrite this as a matrix equation:
[E1x, E1y, E1z] [ delta_u1, delta_v1 ] [Tx, Ty, Tz]
[E2x, E2y, E2z] = [ delta_u2, delta_v2 ] * [Bx, By, Bz]
Here E1, E2, T, and B have been decomposed into their XYZ elements. What we want to find are T and B or Tx, Ty, Tz and Bx, By, Bz. To solve this equation we take the middle matrix with the UV deltas, invert it and multiply to both sides which will remove it from the right side:
inverse([ delta_u1, delta_v1 ] [E1x, E1y, E1z] [Tx, Ty, Tz]
[ delta_u2, delta_v2 ]) * [E2x, E2y, E2z] = [Bx, By, Bz]
Or more compactly you can think of it like this:
ΔP = position delta matrix
ΔUV = UV delta matrix
ΔUV⁻¹ = inverse UV delta matrix
B = T and B basis matrix
ΔP = ΔUV * B
multiply by inverse:
ΔUV⁻¹ * ΔP = B
To solve this we need the inverse, which we previously made a function for.
export function getTangentVectors(points, UVs){
const deltaUV = [
subtractVector(UVs[1], UVs[0]),
subtractVector(UVs[2], UVs[0])
];
const deltaPositions = [
subtractVector(points[1], points[0]),
subtractVector(points[2], points[0])
];
const inverseDeltaUV = getInverse(deltaUV);
return multiplyMatrix(inverseDeltaUV, deltaPositions);
}
There's probably a more generic way to find the basis vectors between any sets of points but we're not worrying about it today.
I actually found a bug in getInverse while doing this. It seems that I forgot the case in getDeterminant when the matrix is 1x1. This is just the value itself but it was forgotten. So watch out for that if you've been following along (I've updated the previous post).
The return value here is a 2x2 matrix, the top row is the tangent basis vector, the bottom row is the bitangent basis vector. Combined with the normal (Z) vector we have all 3 basis vectors for tangent-space.
Note that we can throw away the bitangent if we want as it's trivially computed from the cross product of the tangent and normal. So we'll actually only pass in tangents and then compute the bitangent once per vertex in the vertex shader.
We can update the mesh class with the additional tangent property which works the same a normals. We also need to pass them in as an attribute the vertex shader. I called it aVertexTangent (the Hungarian notion is looking a bit weird with the new shader syntax...).
Using a bump map
Bump maps aren't straight-forward and require a little bit of calculation. Once we have the tangent vectors we can pass in the bump map into the shader. To start I used a test texture.
This allows us to more easily test the algorithm (I had originally used an alternating 1px pattern but this was a very bad idea because where it sampled caused it to be blind to height changes). The idea here is that we need the slope or change in height at the current pixel. To do this, we need to sample some height values around the current pixel to determine the change. First, we need to know how big a pixel is in terms of UV coordinates. To get this we need to know how big the texture is. This is why we upgraded the shader version, newer version have access to a handy function that will get this without us having to pass in another uniform.
//fragment.glsl
ivec2 size = textureSize(uSampler0, 0);
float uSize = 1.0 / float(size[0]);
float vSize = 1.0 / float(size[1]);
textureSize is a version 300+ function that gets the texture dimensions. We divide 1.0 by this to translate a pixel into UV coordinates. Then we can sample the pixel immediately before and immediately after (both X and Y) of the current one.
//fragment.glsl
float positiveU = texture(uSampler0, uv + vec2(uSize, 0))[0];
float positiveV = texture(uSampler0, uv + vec2(0, vSize))[0];
float negativeU = texture(uSampler0, uv + vec2(-uSize, 0))[0];
float negativeV = texture(uSampler0, uv + vec2(0, -vSize))[0];
Note that because we're using a black/white png this is a complete waste because we only need to sample one channel. To fix this, we could either make it just red or pack it in a more compact format but I left it black because it illustrates better.
From this we can construct the tangent-space normal:
//fragment.glsl
float changeU = negativeU - positiveU;
float changeV = negativeV - positiveV;
vec3 tangentSpaceNormal = normalize(vec3(changeU, changeV, 1.0/scale));
It's the change in each direction. The final term is how high it goes in the normal direction which is where we can scale it. Normalizing produces a unit vector which makes the ratios correct.
Next we can convert the tangent space to world space using a matrix:
//fragment.glsl
mat3x3 tangentToWorld = mat3x3(
tangent.x, bitangent.x, normal.x,
tangent.y, bitangent.y, normal.y,
tangent.z, bitangent.z, normal.z
);
vec3 worldSpaceNormal = tangentToWorld * tangentSpaceNormal;
Nothing new here, it's just the tangent space matrix we calculated before using the tangent space basis vectors.
Finally we can add lighting since without it this technique is useless:
//fragment.glsl
bool isPoint = uLight1[3][3] == 1.0;
if(isPoint) {
//point light + color
vec3 toLight = normalize(uLight1[0].xyz - position);
float light = dot(worldSpaceNormal, toLight);
fragColor = color * uLight1[2] * vec4(light, light, light, 1);
} else {
//directional light + color
float light = dot(worldSpaceNormal, uLight1[1].xyz);
fragColor = color * uLight1[2] * vec4(light, light, light, 1);
}
Here's the full thing:
//shader.frag.glsl
#version 300 es
precision mediump float;
in vec2 uv;
in vec3 position;
in vec3 normal;
in vec3 tangent;
in vec3 bitangent;
in vec4 color;
out vec4 fragColor;
uniform sampler2D uSampler0;
uniform float scale;
uniform mat4x4 uLight1;
void main() {
//for normal maps
//blue normal (UP)
//green V (Y)
//red U (x)
ivec2 size = textureSize(uSampler0, 0);
float uSize = 1.0 / float(size[0]);
float vSize = 1.0 / float(size[1]);
//sample a pixel to either side
float positiveU = texture(uSampler0, uv + vec2(uSize, 0))[0];
float positiveV = texture(uSampler0, uv + vec2(0, vSize))[0];
float negativeU = texture(uSampler0, uv + vec2(-uSize, 0))[0];
float negativeV = texture(uSampler0, uv + vec2(0, -vSize))[0];
mat3x3 tangentToWorld = mat3x3(
tangent.x, bitangent.x, normal.x,
tangent.y, bitangent.y, normal.y,
tangent.z, bitangent.z, normal.z
);
float changeU = negativeU - positiveU;
float changeV = negativeV - positiveV;
vec3 tangentSpaceNormal = normalize(vec3(changeU, changeV, 1.0/scale));
vec3 worldSpaceNormal = tangentToWorld * tangentSpaceNormal;
bool isPoint = uLight1[3][3] == 1.0;
if(isPoint) {
//point light + color
vec3 toLight = normalize(uLight1[0].xyz - position);
float light = dot(worldSpaceNormal, toLight);
fragColor = color * uLight1[2] * vec4(light, light, light, 1);
} else {
//directional light + color
float light = dot(worldSpaceNormal, uLight1[1].xyz);
fragColor = color * uLight1[2] * vec4(light, light, light, 1);
}
}
Animation
It's actually really hard to see if this works with the lights and object being static (we move the camera, but the direction light hits the object never changes). In order to fix this, I implemented a rudimentary animation system. Here's what the class looks like:
//animation.js
export class Animation {
#target;
#property;
#from;
#to;
#duration;
#firstTimestamp;
#repeat;
constructor(animation){
this.#target = animation.target;
this.#property = animation.property;
this.#from = animation.from;
this.#to = animation.to;
this.#duration = animation.duration;
this.#repeat = animation.repeat;
}
run(timestamp){
if(!this.#firstTimestamp){
this.#firstTimestamp = timestamp;
}
const elapsedTime = timestamp - this.#firstTimestamp;
const animationRatio = this.#repeat
? (elapsedTime % this.#duration) / this.#duration
: Math.min(elapsedTime, this.#duration) / this.#duration
const length = this.#to - this.#from;
this.#target.resetTransforms();
switch(this.#property){
case "rotate-y": {
this.#target.rotate({ y: this.#from + (animationRatio * length) })
break;
}
}
}
}
You give it some parameters of starting value, ending value, property, whether it repeats, how long it takes and a reference to the object it operates on. That last design decision I'm not entire sure about. Should objects have animations? Should animation have objects? Should they weakly reference each other and have the engine apply animation? I'm not experienced enough to know but this made the most sense at the time. I only implemented one property but it's pretty easy to see how to add others. The first time run is called it sets up #firstTimeStamp. This tells us when the animation started. Using the elapsed time and the #duration we can calculate how far along in the animation we are. We then apply this to the "length" or distance between the first value and second value. We can then set the value to the length * animationRatio but we also need to add the initial value incase we didn't start at 0.
It's not sophisticated, only operates on one object and only very simple single properties but this works enough to see what happens when the light direction changes.
Before that, I should explain the change made to mesh.js. Before, I had it setup such that changes to rotation, scale, translation etc were applied directly to the model matrix. This isn't great because we can't just set a rotation, which is how the animation code works, we'd need to know how much to rotate to get from the current position to the next position which is complicated. So instead I made it so that rotations, scales, translates etc. instead just add a new matrix to a list of transforms (I also made the names a little better). Then the model matrix is generated on demand. We can reset all transforms by clearing that array.
//mesh.js
export class Mesh {
#transforms = [];
//...
set tangents(val){
this.#tangents = val;
}
get tangents(){
return this.#tangents;
}
translate({ x = 0, y = 0, z = 0 }) {
this.#transforms.push(getTranslationMatrix(x, y ,z));
return this;
}
scale({ x = 1, y = 1, z = 1 }) {
this.#transforms.push(getScaleMatrix(x, y, z));
return this;
}
rotate({ x, y, z }) {
//there's an order dependency here... something something quaterions...
if (x) {
this.#transforms.push(getRotationXMatrix(x));
}
if (y) {
this.#transforms.push(getRotationYMatrix(y));
}
if (z) {
this.#transforms.push(getRotationZMatrix(z));
}
return this;
}
resetTransforms() {
this.#transforms = [];
}
getModelMatrix() {
const modelMatrix = this.#transforms.reduce((mm, tm) => multiplyMatrix(tm, mm), getIdentityMatrix());
return transpose(modelMatrix).flat();
}
//....
}
This is still not the most ideal form, we might not want to just remove all transforms because two animations operating on different properties will clobber each other but it's enough for now.
Next, we can take the animation and add a new function where we set all of these up:
wc-geo-gl.js
//class WcGeoGl { ...
createAnimations(){
this.animations = {
rotateQuad: new Animation({
from: 0,
to: TWO_PI,
property: "rotate-y",
duration: 5000,
target: this.meshes.quad, //whatever the mesh is
repeat: true
})
}
}
Then in the render function, right before we start rendering objects we create another loop that applies the animations:
for (const animation of Object.values(this.animations)){
animation.run(timestamp)
}
Here's what the final quad looks like when rotated (it cuts off at the back due to the normals facing the wrong direction).
It's not a great example but you can visibly see the "protruding detail". We can apply to a cube to see it a little better (using hard-coded tangents):
Using the vertical striped test texture we can see if the orientation works with our process to get the tangents from the UVs:
These seem to line up so I'm guessing it works okay but it's hard without better artistic test cases.
You can find the code for this at: https://github.com/ndesmic/geogl/tree/v10
Appendix
Here's a chart that helps link the various representation of spaces:
| TBN-space | XYZ-space | Color-space | UV-space |
|---|---|---|---|
| Tangent | X | Red | U |
| Bitangent | Y | Green | V |
| Normal | Z | Blue | n/a |
Oldest comments (0)