Sunday, February 12, 2017

OpenGL 4 with OpenTK in C# Part 11: Mipmap


This post we will go through how to setup Mipmaps in OpenGL 4.5 with the help of OpenTK.

This is part 11 of my series on OpenGL4 with OpenTK.
For other posts in this series:
OpenGL 4 with OpenTK in C# Part 1: Initialize the GameWindow
OpenGL 4 with OpenTK in C# Part 2: Compiling shaders and linking them
OpenGL 4 with OpenTK in C# Part 3: Passing data to shaders
OpenGL 4 with OpenTK in C# Part 4: Refactoring and adding error handling
OpenGL 4 with OpenTK in C# Part 5: Buffers and Triangle
OpenGL 4 with OpenTK in C# Part 6: Rotations and Movement of objects
OpenGL 4 with OpenTK in C# Part 7: Vectors and Matrices
OpenGL 4 with OpenTK in C# Part 8: Drawing multiple objects
OpenGL 4 with OpenTK in C# Part 9: Texturing
OpenGL 4 with OpenTK in C# Part 10: Asteroid Invaders
OpenGL 4 with OpenTK in C# Part 11: Mipmap
OpenGL 4 with OpenTK in C# Part 12: Basic Moveable Camera
OpenGL 4 with OpenTK in C# Part 13: IcoSphere
OpenGL 4 with OpenTK in C# Part 14: Basic Text

As stated in the previous post, I am in no way an expert in OpenGL. I write these posts as a way to learn and if someone else finds these posts useful then all the better :)
If you think that the progress is slow, then know that I am a slow learner :P
This part will build upon the game window and shaders from part 10.

Mipmap what?

Mipmapping is an optimization tool that lets OpenGL pick a texture that is closer to the needed size depending on the size of the geometry being rendered. This lets OpenGL sample from an image that has the data needed closer together, i.e. the stride between texels is not very far. Also, this helps preventing scintillation or grid illusions when the object is far away. All this to a small penalty of memory overhead.
An example of a manually created mipmap texture below:
Wood texture with mipmap levels included
Wood texture with mipmap levels included

Manual mipmap loading

So, lets see how to load a manually created mipmap texture with OpenTK.
Lets first create a new class, basically a copy from the existing TexturedRenderObject, called MipmapManualRenderObject.
    public class MipMapManualRenderObject : ARenderable
    {
        private int _minMipmapLevel = 0;
        private int _maxMipmapLevel;
        private int _texture;
        public MipMapManualRenderObject(TexturedVertex[] vertices, int program, string filename, int maxMipmapLevel)
            : base(program, vertices.Length)
Note the addition of the maxMipmapLevel parameter in the constructor. This will help us load the texture.
Now lets modify the LoadTexture method a bit to allow for it to load all the levels and return them to the InitializeTextures method
private List<MipLevel> LoadTexture(string filename)
{
    var mipmapLevels = new List<MipLevel>();
    using (var bmp = (Bitmap)Image.FromFile(filename))
    {
        int xOffset = 0;
        int width = bmp.Width;
        int height = (bmp.Height/3)*2;
        var originalHeight = height;
        for (int m = 0; m < _maxMipmapLevel; m++)
        {
            xOffset += m == 0 || m == 1 ? 0 : width*2;
            var yOffset = m == 0 ? 0 : originalHeight;

            MipLevel mipLevel;
            mipLevel.Level = m;
            mipLevel.Width = width;
            mipLevel.Height = height;
            mipLevel.Data = new float[mipLevel.Width * mipLevel.Height * 4];
            int index = 0;
            ExtractMipmapLevel(yOffset, mipLevel, xOffset, bmp, index);
            mipmapLevels.Add(mipLevel);

            if (width == 1 || height == 1)
            {
                _maxMipmapLevel = m;
                break;
            }

            width /= 2;
            height /= 2;
        }
    }
    return mipmapLevels;
}
Basically what we do here is to load the image from disk, iterate over it foreach mipmap level and create a MipLevel object that holds the data for that level.
For every iteration, the width and height varibles are halved, for example: 256, 128, 64, 32, 16, 8, 4, 2, 1 would yield in 0 as the original texture followed by 8 mipmaps.
If the width or height goes to 1 before the other, we will continue until both reach 1. If they reach 1 before we reach the max levels provided as input, we just reset the variable to whatever level we are on at the moment. The MipLevel struct looks like following.
public struct MipLevel
{
    public int Level;
    public int Width;
    public int Height;
    public float[] Data;
}

For each level the following method will extract the data from the bitmap. The assumed structure is as the example images in this post.
private static void ExtractMipmapLevel(int yOffset, MipLevel mipLevel, int xOffset, Bitmap bmp, int index)
{
    var width = xOffset + mipLevel.Width;
    var height = yOffset + mipLevel.Height;
    for (int y = yOffset; y < height; y++)
    {
        for (int x = xOffset; x < width; x++)
        {
            var pixel = bmp.GetPixel(x, y);
            mipLevel.Data[index++] = pixel.R/255f;
            mipLevel.Data[index++] = pixel.G/255f;
            mipLevel.Data[index++] = pixel.B/255f;
            mipLevel.Data[index++] = pixel.A/255f;
        }
    }
}

And the InitializeTexture method would have the following changes:
private void InitTextures(string filename)
{
    var data = LoadTexture(filename);
    GL.CreateTextures(TextureTarget.Texture2D, 1, out _texture);
    GL.BindTexture(TextureTarget.Texture2D, _texture);
    GL.TextureStorage2D(
        _texture,
        _maxMipmapLevel,             // levels of mipmapping
        SizedInternalFormat.Rgba32f, // format of texture
        data.First().Width,
        data.First().Height); 

    for (int m = 0; m < data.Count; m++)
    {
        var mipLevel = data[m];
        GL.TextureSubImage2D(_texture,
            m,                  // this is level m
            0,                  // x offset
            0,                  // y offset
            mipLevel.Width,
            mipLevel.Height,
            PixelFormat.Rgba,
            PixelType.Float,
            mipLevel.Data);
    }
            
    var textureMinFilter = (int)All.LinearMipmapLinear;
    GL.TextureParameterI(_texture, All.TextureMinFilter, ref textureMinFilter);
    var textureMagFilter = (int)All.Linear;
    GL.TextureParameterI(_texture, All.TextureMagFilter, ref textureMagFilter);
    // data not needed from here on, OpenGL has the data
}
First off we tell GL.TextureStorage2D that we will have mipmaps by supplying the _maxMipmapLevel variable. We then proceed to iterate over the levels provided by our load method and initialize them with GL.TextureSubImage2D with the level.
At the end, we tell OpenGL that we want to use mipmapping for the minimizing filter by sending in the LinearMipmapLinear constant to the GL.TextureParameterI method

Let OpenGL generate the mipmap

Ok, so maybe we are lazy and don't want to create this by ourselves. Luckily, there is a shortcut. We can tell OpenGL to generate mipmap levels based on the input texture that you load into level 0 with the following command.
GL.GenerateTextureMipmap(_texture);

So, lets create another class called MipmapGeneratedRenderObject as a copy from the original TextureRenderObject and change it to generate the mipmapping by iteself.
We need only change the InitTextures method in this case
private void InitTextures(string filename)
{
    int width, height;
    var data = LoadTexture(filename, out width, out height);
    GL.CreateTextures(TextureTarget.Texture2D, 1, out _texture);
    GL.TextureStorage2D(
        _texture,
        _maxMipmapLevel,             // levels of mipmapping
        SizedInternalFormat.Rgba32f, // format of texture
        width, 
        height); 

    GL.BindTexture(TextureTarget.Texture2D, _texture);
    GL.TextureSubImage2D(_texture,
        0,                  // this is level 0
        0,                  // x offset
        0,                  // y offset
        width,   
        height, 
        PixelFormat.Rgba,
        PixelType.Float,
        data);
            
    GL.GenerateTextureMipmap(_texture);
    GL.TextureParameterI(_texture, All.TextureBaseLevel, ref _minMipmapLevel);
    GL.TextureParameterI(_texture, All.TextureMaxLevel, ref _maxMipmapLevel);
    var textureMinFilter = (int)TextureMinFilter.LinearMipmapLinear;
    GL.TextureParameterI(_texture, All.TextureMinFilter, ref textureMinFilter);
    var textureMagFilter = (int)TextureMinFilter.Linear;
    GL.TextureParameterI(_texture, All.TextureMagFilter, ref textureMagFilter);
    // data not needed from here on, OpenGL has the data
}
Call the GL.GenerateTextureMipmap and then set the rendering filters and levels needed to get it working. It may not be the best quality and may differ between different graphics cards, but it does the trick. If you want the best quality, you must use the manual creation above.

Change to the Fragment Shader

For this to work we need to change from texelFetch that took exact texture coordinates and mipmap level to texture that takes a coordinate between 0 to 1 and does the interpolation and level changing automatically.
#version 450 core
in vec2 vs_textureCoordinate;
uniform sampler2D textureObject;
out vec4 color;

void main(void)
{
 color = texture(textureObject, vs_textureCoordinate);
}
It is nice to see that my questioning of the solution in the Texture post now found its answer. We need to change the calls to
RenderObjectFactory.CreateTexturedCube(1, 1, 1)
To send in 1 as width and height instead of the pixel sizes before.

In action

So how does this work in action. Lets initialize 3 objects with the same texture. One without mipmap (to the left) one with auto generated mipmap (right lower) and one with manually created mipmap texture (right upper) and see how it looks like when the object moves farther away and then back close.  I added the level numberings as I did not see any difference with this particular texture between the manual and generated one, but it is interesting to see how OpenGL fluidly changes between the levels as needed.
The manually created object has the following texture so that we can see the mipmap level that is used at the moment.
Asteroid texture with mipmap levels numbered
Asteroid texture with mipmap levels numbered
For the complete source code for the game at the end of this part, go to: https://github.com/eowind/dreamstatecoding

So there, thank you for reading. Hope this helps someone out there : )

Until next time: Work to Live, Don’t Live to Work

No comments:

Post a Comment