Dreamstate Coding: OpenGL 4 with OpenTK in C# Part 11: Mipmap

Mipmap what?

Mipmapping is an optimization tool that lets OpenGL pick a texture that is closer to the needed size depending on the size of the geometry being rendered. This lets OpenGL sample from an image that has the data needed closer together, i.e. the stride between texels is not very far. Also, this helps preventing scintillation or grid illusions when the object is far away. All this to a small penalty of memory overhead.
An example of a manually created mipmap texture below:

Wood texture with mipmap levels included

Manual mipmap loading

So, lets see how to load a manually created mipmap texture with OpenTK.

Lets first create a new class, basically a copy from the existing TexturedRenderObject, called MipmapManualRenderObject.

    public class MipMapManualRenderObject : ARenderable
    {
        private int _minMipmapLevel = 0;
        private int _maxMipmapLevel;
        private int _texture;
        public MipMapManualRenderObject(TexturedVertex[] vertices, int program, string filename, int maxMipmapLevel)
            : base(program, vertices.Length)

Note the addition of the maxMipmapLevel parameter in the constructor. This will help us load the texture.
Now lets modify the LoadTexture method a bit to allow for it to load all the levels and return them to the InitializeTextures method

private List<MipLevel> LoadTexture(string filename)
{
    var mipmapLevels = new List<MipLevel>();
    using (var bmp = (Bitmap)Image.FromFile(filename))
    {
        int xOffset = 0;
        int width = bmp.Width;
        int height = (bmp.Height/3)*2;
        var originalHeight = height;
        for (int m = 0; m < _maxMipmapLevel; m++)
        {
            xOffset += m == 0 || m == 1 ? 0 : width*2;
            var yOffset = m == 0 ? 0 : originalHeight;

            MipLevel mipLevel;
            mipLevel.Level = m;
            mipLevel.Width = width;
            mipLevel.Height = height;
            mipLevel.Data = new float[mipLevel.Width * mipLevel.Height * 4];
            int index = 0;
            ExtractMipmapLevel(yOffset, mipLevel, xOffset, bmp, index);
            mipmapLevels.Add(mipLevel);

            if (width == 1 || height == 1)
            {
                _maxMipmapLevel = m;
                break;
            }

            width /= 2;
            height /= 2;
        }
    }
    return mipmapLevels;
}

Basically what we do here is to load the image from disk, iterate over it foreach mipmap level and create a MipLevel object that holds the data for that level.
For every iteration, the width and height varibles are halved, for example: 256, 128, 64, 32, 16, 8, 4, 2, 1 would yield in 0 as the original texture followed by 8 mipmaps.
If the width or height goes to 1 before the other, we will continue until both reach 1. If they reach 1 before we reach the max levels provided as input, we just reset the variable to whatever level we are on at the moment. The MipLevel struct looks like following.

public struct MipLevel
{
    public int Level;
    public int Width;
    public int Height;
    public float[] Data;
}

For each level the following method will extract the data from the bitmap. The assumed structure is as the example images in this post.

private static void ExtractMipmapLevel(int yOffset, MipLevel mipLevel, int xOffset, Bitmap bmp, int index)
{
    var width = xOffset + mipLevel.Width;
    var height = yOffset + mipLevel.Height;
    for (int y = yOffset; y < height; y++)
    {
        for (int x = xOffset; x < width; x++)
        {
            var pixel = bmp.GetPixel(x, y);
            mipLevel.Data[index++] = pixel.R/255f;
            mipLevel.Data[index++] = pixel.G/255f;
            mipLevel.Data[index++] = pixel.B/255f;
            mipLevel.Data[index++] = pixel.A/255f;
        }
    }
}

And the InitializeTexture method would have the following changes:

private void InitTextures(string filename)
{
    var data = LoadTexture(filename);
    GL.CreateTextures(TextureTarget.Texture2D, 1, out _texture);
    GL.BindTexture(TextureTarget.Texture2D, _texture);
    GL.TextureStorage2D(
        _texture,
        _maxMipmapLevel,             // levels of mipmapping
        SizedInternalFormat.Rgba32f, // format of texture
        data.First().Width,
        data.First().Height); 

    for (int m = 0; m < data.Count; m++)
    {
        var mipLevel = data[m];
        GL.TextureSubImage2D(_texture,
            m,                  // this is level m
            0,                  // x offset
            0,                  // y offset
            mipLevel.Width,
            mipLevel.Height,
            PixelFormat.Rgba,
            PixelType.Float,
            mipLevel.Data);
    }
            
    var textureMinFilter = (int)All.LinearMipmapLinear;
    GL.TextureParameterI(_texture, All.TextureMinFilter, ref textureMinFilter);
    var textureMagFilter = (int)All.Linear;
    GL.TextureParameterI(_texture, All.TextureMagFilter, ref textureMagFilter);
    // data not needed from here on, OpenGL has the data
}

First off we tell GL.TextureStorage2D that we will have mipmaps by supplying the _maxMipmapLevel variable. We then proceed to iterate over the levels provided by our load method and initialize them with GL.TextureSubImage2D with the level.
At the end, we tell OpenGL that we want to use mipmapping for the minimizing filter by sending in the LinearMipmapLinear constant to the GL.TextureParameterI method

Let OpenGL generate the mipmap

Ok, so maybe we are lazy and don't want to create this by ourselves. Luckily, there is a shortcut. We can tell OpenGL to generate mipmap levels based on the input texture that you load into level 0 with the following command.

GL.GenerateTextureMipmap(_texture);

So, lets create another class called MipmapGeneratedRenderObject as a copy from the original TextureRenderObject and change it to generate the mipmapping by iteself.
We need only change the InitTextures method in this case

private void InitTextures(string filename)
{
    int width, height;
    var data = LoadTexture(filename, out width, out height);
    GL.CreateTextures(TextureTarget.Texture2D, 1, out _texture);
    GL.TextureStorage2D(
        _texture,
        _maxMipmapLevel,             // levels of mipmapping
        SizedInternalFormat.Rgba32f, // format of texture
        width, 
        height); 

    GL.BindTexture(TextureTarget.Texture2D, _texture);
    GL.TextureSubImage2D(_texture,
        0,                  // this is level 0
        0,                  // x offset
        0,                  // y offset
        width,   
        height, 
        PixelFormat.Rgba,
        PixelType.Float,
        data);
            
    GL.GenerateTextureMipmap(_texture);
    GL.TextureParameterI(_texture, All.TextureBaseLevel, ref _minMipmapLevel);
    GL.TextureParameterI(_texture, All.TextureMaxLevel, ref _maxMipmapLevel);
    var textureMinFilter = (int)TextureMinFilter.LinearMipmapLinear;
    GL.TextureParameterI(_texture, All.TextureMinFilter, ref textureMinFilter);
    var textureMagFilter = (int)TextureMinFilter.Linear;
    GL.TextureParameterI(_texture, All.TextureMagFilter, ref textureMagFilter);
    // data not needed from here on, OpenGL has the data
}

Call the GL.GenerateTextureMipmap and then set the rendering filters and levels needed to get it working. It may not be the best quality and may differ between different graphics cards, but it does the trick. If you want the best quality, you must use the manual creation above.

Change to the Fragment Shader

For this to work we need to change from texelFetch that took exact texture coordinates and mipmap level to texture that takes a coordinate between 0 to 1 and does the interpolation and level changing automatically.

#version 450 core
in vec2 vs_textureCoordinate;
uniform sampler2D textureObject;
out vec4 color;

void main(void)
{
 color = texture(textureObject, vs_textureCoordinate);
}

It is nice to see that my questioning of the solution in the Texture post now found its answer. We need to change the calls to

RenderObjectFactory.CreateTexturedCube(1, 1, 1)

To send in 1 as width and height instead of the pixel sizes before.

In action

So how does this work in action. Lets initialize 3 objects with the same texture. One without mipmap (to the left) one with auto generated mipmap (right lower) and one with manually created mipmap texture (right upper) and see how it looks like when the object moves farther away and then back close. I added the level numberings as I did not see any difference with this particular texture between the manual and generated one, but it is interesting to see how OpenGL fluidly changes between the levels as needed.

The manually created object has the following texture so that we can see the mipmap level that is used at the moment.

Asteroid texture with mipmap levels numbered

For the complete source code for the game at the end of this part, go to: https://github.com/eowind/dreamstatecoding

So there, thank you for reading. Hope this helps someone out there : )

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment or share a link, not required but appreciated! :)

4 comments:

ZimmerSeptember 25, 2019 at 8:25 PM
On the GL.TextureParameterI(_texture, All.TextureMaxLevel, ref _maxMipmapLevel); lines I get errors. Can not convert All.TextureMaxLevel to TextureParameterName and _maxMipMapLevel may not be passed by ref keyword. After I remove the ref it says can not convert int to int[].
Could use some help here, as I'm learning OpenGL.
AnonymousAugust 12, 2021 at 5:54 PM
FYI: The wooden texture in the repository is laid out incorrectly.

Dreamstate Coding

Pages

Sunday, February 12, 2017

OpenGL 4 with OpenTK in C# Part 11: Mipmap