Textures

You may need to read the articles about uniforms and varyings before this one.

A texture is an array of data that can be randomly accessed. Textures are mostly used to store image data.

Texture space is a coordinate system that represents a texture in the range $[0, 1]$ from left to right, bottom to top. Sampling is retrieving the data from a texture at the given texture coordinates. A sampler is a uniform that is used to sample a texture (via the built-in texture function). A discrete point of data in a texture is called a texel.

#version 300 es

precision mediump float;

in vec2 v_texcoord;

uniform sampler2D u_texture;

out vec4 outColor;

void main() {
	outColor = texture(u_texture, v_texcoord);
}

A WebGLTexture can be created with createTexture. Like buffers, textures need to be bound to a binding point with bindTexture in order to be manipulated. The TEXTURE_2D binding point is used for two-dimensional textures, which can be supplied data with texImage2D. The following example creates a one-by-one magenta placeholder texture, then loads an image into it.

const texture = gl.createTexture();

gl.bindTexture(gl.TEXTURE_2D, texture);

gl.texImage2D(
	gl.TEXTURE_2D,
	0,
	gl.RGBA,
	1,
	1,
	0,
	gl.RGBA,
	gl.UNSIGNED_BYTE,
	new Uint8Array([0xff, 0, 0xff, 0xff])
);

const image = new Image();
image.addEventListener("load", () => {
	gl.bindTexture(gl.TEXTURE_2D, texture);
	gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, image);
	gl.generateMipmap(gl.TEXTURE_2D); // See below.
});
image.crossOrigin = "";
image.src = "https://www.lakuna.pw/images/webgl-example-texture.png";

To pass a texture to a sampler uniform, it must be bound to a texture unit that represents it. The active texture unit can be specified with activeTexture, after which point binding a texture to a binding point assigns it to that texture unit. The texture unit is passed to the shader program instead of the texture.

gl.activeTexture(gl.TEXTURE0);
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.uniform1i(imageUniformLocation, 0);

Texture Parameters

Textures have various parameters that can be modified with texParameter[fi].

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);

The value that is returned when sampling a value outside of texture space is controlled by the TEXTURE_WRAP_S and TEXTURE_WRAP_T parameters for the $s$ and $t$ axes, respectively.

• REPEAT makes the texture repeat. This is the default functionality.
• CLAMP_TO_EDGE repeats the last texel.
• MIRRORED_REPEAT repeats the texture, but mirrors it for each repetition.

A mipmap is a collection of mips, which are smaller versions of a texture that are used to sample that texture at different resolutions. The way that a mipmap is generated can be specified with the minification and magnification filter parameters (TEXTURE_MIN_FILTER and TEXTURE_MAG_FILTER, respectively). A minification filter is used when rendering anything smaller than the largest mip, and a magnification filter is used when rendering anything larger than the largest mip.

• NEAREST chooses one pixel from the largest mip.
• LINEAR chooses four pixels from the largest mip and blends them.
• NEAREST_MIPMAP_NEAREST chooses the best mip, then picks one pixel from that mip.
• LINEAR_MIPMAP_NEAREST chooses the best mip, then blends four pixels from that mip.
• NEAREST_MIPMAP_LINEAR chooses the best two mips, then chooses one pixel from each and blends them.
• LINEAR_MIPMAP_LINEAR chooses the best two mips, then chooses four pixels from each and blends them.

Since the magnification filter is only used when rendering larger than the largest mip, it can only be NEAREST or LINEAR.

In order for a texture to be sampled, it must be texture complete, meaning that only the largest mip is sampled from (the minification filter is either NEAREST or LINEAR) or the texture has a complete mipmap. The easiest way to generate a complete mipmap is with generateMipmap.

gl.generateMipmap(gl.TEXTURE_2D);

Texture Atlases

A texture atlas is a texture that contains multiple images. Using a texture atlas can simplify a shader and reduce the number of calls to rendering functions. To use a texture atlas, use texture coordinates that outline only the portion of the texture atlas that contains the desired image.

Data Textures

Rather than loading texture data from an image, texture data can also be supplied from an array or buffer. The way that the data is interpreted depends on the internal format of the texture. For example, setting the internal format to R8 causes the texture to expect one unsigned byte per texel, which it normalizes to color space. Each internal format has a corresponding format and data type that must also be specified. For example, with an internal format of R8, the format must be set to RED and the data type must be set to UNSIGNED_BYTE.

gl.texImage2D(
	gl.TEXTURE_2D,
	0,
	gl.R8,
	3,
	2,
	0,
	gl.RED,
	gl.UNSIGNED_BYTE,
	new Uint8Array([0x80, 0x40, 0x80, 0x00, 0xc0, 0x00])
);

Textures will be malformed if their row alignment is not a multiple of the unpack alignment, which specifies the expected alignment of the rows in the supplied texture data. The unpack alignment defaults to four in order to maintain backwards compatibility, but it can be set to one, two, four, or eight.

For example, the texture above has a width of three pixels per row and uses one byte per pixel, meaning that each row is three bytes wide. Since three is not a multiple of two, four, or eight, the unpack alignment must be set to one with pixelStorei.

gl.pixelStorei(gl.UNPACK_ALIGNMENT, 1);

Projection Mapping

Projection mapping is the process of projecting an image onto a surface. It can be accomplished by making another camera to act as a projector, then passing the projector's view projection matrix to the shader program so that the surface's position can be determined from the point of view of both the viewer and the projector.

#version 300 es

in vec4 a_position;
in vec2 a_texcoord;

uniform mat4 u_world;
uniform mat4 u_viewerViewProjection;
uniform mat4 u_textureMatrix;

out vec2 v_texcoord;
out vec4 v_projectedTexcoord;

void main() {
	gl_Position = u_viewerViewProjection * u_world * a_position;
	v_texcoord = a_texcoord;
	v_projectedTexcoord = u_textureMatrix * u_world * a_position;
}

The texture matrix is a slightly modified version of the projector's view projection matrix that also converts from clip space to texture space. It can be constructed by multiplying a matrix that translates and then scales by $\frac{1}{2}$ on each axis by the projector's view projection matrix.

const half = Vector3.fromValues(0.5, 0.5, 0.5);
const texMat = new Matrix4().translate(half).scale(half).multiply(viewProj);

See my math library, μMath, for documentation of the example above.

The fragment shader needs to determine which texture to use at each position based on whether the position falls within the frustum of the projector or not.

#version 300 es

precision mediump float;

in vec2 v_texcoord;
in vec4 v_projectedTexcoord;

uniform sampler2D u_texture;
uniform sampler2D u_projectedTexture;

out vec4 outColor;

void main() {
	vec2 projectedTexcoord =
		(v_projectedTexcoord.xyz / v_projectedTexcoord.w).xy;

	bool inRange = projectedTexcoord.x >= 0.0
		&& projectedTexcoord.x <= 1.0
		&& projectedTexcoord.y >= 0.0
		&& projectedTexcoord.y <= 1.0;

	vec4 projectedTextureColor =
		texture(u_projectedTexture, projectedTexcoord);

	vec4 textureColor = texture(u_texture, v_texcoord);

	outColor = inRange ? projectedTextureColor : textureColor;
}

Notice that v_projectorTexcoord.xyz is manually divided by v_projectorTexcoord.w to determine the projected texture coordinates because perspective is only applied automatically to gl_Position.

Also notice that rather than drawing the texture on top of existing fragments, texture projection works by checking if a fragment is within the projector's frustum and using the correct texel based on that. Regardless of which texel is used, both textures must be sampled. This is because, according to section 8.8 of the GLSL ES 3.00 specification:

Some texture functions (non-“Lod” and non-“Grad” versions) may require implicit derivatives. Implicit derivatives are undefined within non-uniform control flow and for vertex texture fetches.

In other words, although we might not always use the sampled texel, we always have to sample each texture.

The frustum of the projector is shown in the example above by using the inverse of the projector's view projection matrix as the world matrix of a wireframe cube that spans the entirety of clip space.

The next article is about framebuffers.