Friday, July 21, 2017

Expression evaluation over time - Was?


A closer look at the Was query to determine if something has been, builds heavily on the previous articles in this series.

Other articles in the AI Knowledge Based Reasoning series on this site:
Knowledge based reasoning in .net c#
Reasoning in Open or Closed models
Logic Expression evaluation with open-world assumption
Expression evaluation on object based models
Expression evaluation over time
Expression evaluation over time - Was?

Definition of Was

So lets start with defining what we mean when we say Was.
Was Mikaela angry?
Is there a previous frame where the logical expression Feeling(Angry) results in true?
There is also a meaning of this to not be the case at some point after it was true.

Lets look at this in a table form to get our bearings straight.
FrameNameOccupationEye-colorHair-color
1MikaelaStudentBlueBrown
2MikaelaPearlescent
3MikaelaPink
4MikaelaJunior Software DeveloperBlue
5MikaelaSoftware DeveloperBlonde
6MikaelaSenior Software DeveloperPearlescent
7MikaelaSoftware ArchitectBrown

At this point was could ask,

WAS Occupation(Student)
Would be true, as there is a frame where Mikaela was a student. and after that something else.

WAS Name(Mikaela)
Would not be true, as she is Mikaela and has never had any other name

WAS Occupation(Software Architect)
Would not be true, as she currently is and there is no previous frame where she was

WAS HairColor(Brown)
Would be true, she still is in the current frame, but there was other hair colors for at least one frame and all the way back there was Brown again.

Unit tests

Lets look at this in unit test form

[TestClass]
public class RelativeWasTest
{
    [TestMethod]
    public void RelativeWasTest_True()
    {
        Context target;
        var obj = CreateTarget(out target);
            
        var expr2 = new ExpressionIs(new KnowledgeRelation { Subject = "Student", Relation = "Occupation of", Target = obj.ToString() });
        var relative = new RelativeWas(expr2);
        var actual = target.Evaluate(relative);
        Assert.AreEqual(EvaluationResult.True, actual);
    }
    [TestMethod]
    public void RelativeWasTest_False()
    {
        Context target;
        var obj = CreateTarget(out target);

        var expr2 = new ExpressionIs(new KnowledgeRelation { Subject = "Lawyer", Relation = "Occupation of", Target = obj.ToString() });
        var relative = new RelativeWas(expr2);
        var actual = target.Evaluate(relative);
        Assert.AreEqual(EvaluationResult.False, actual);
    }
    [TestMethod]
    public void RelativeWasTest_True_PeriodOfWasNot()
    {
        Context target;
        var obj = CreateTarget(out target);

        var expr2 = new ExpressionIs(new KnowledgeRelation { Subject = "Black", Relation = "HairColor of", Target = obj.ToString() });
        var relative = new RelativeWas(expr2);
        var actual = target.Evaluate(relative);
        Assert.AreEqual(EvaluationResult.True, actual);
    }
    [TestMethod]
    public void RelativeWasTest_NotSure()
    {
        Context target;
        var obj = CreateTarget(out target);

        var expr2 = new ExpressionIs(new KnowledgeRelation { Subject = "Student", Relation = "Lunch of", Target = obj.ToString() });
        var relative = new RelativeWas(expr2);
        var actual = target.Evaluate(relative);
        Assert.AreEqual(EvaluationResult.NotSure, actual);
    }

    private static Person CreateTarget(out Context target)
    {
        var obj = new Person("Alice")
        {
            Occupation = "Student",
            HairColor = "Black"
        };
        target = new Context(obj.ToString());
        target.AddFrame(FrameFactory.Create(obj));

        obj.Occupation = string.Empty;
        obj.HairColor = "Pearlescent";
        target.AddFrame(FrameFactory.Create(obj));

        obj.Occupation = string.Empty;
        obj.HairColor = "Pink";
        target.AddFrame(FrameFactory.Create(obj));

        obj.Occupation = "Lawyer";
        obj.HairColor = "Black";
        target.AddFrame(FrameFactory.Create(obj));
        return obj;
    }
}

And just to make things interesting its not the exact example as in this post. Just noticed it now, but anyway. The idea is the same.

How to implement this based on the IRelative interface and Context class from the previous post.

public class RelativeWas : IRelative
{
 private readonly AExpression _expression;

 public RelativeWas(AExpression expression)
 {
  _expression = expression;
 }

 public EvaluationResult Evaluate(Context context)
 {
  var wasNot = false;
  for (int frameIndex = context.Frames.Count - 1; frameIndex >= 0; frameIndex--)
  {
   var result = context.Evaluate(_expression, frameIndex);

   if (result == EvaluationResult.NotSure)
    return EvaluationResult.NotSure;
   if (wasNot)
   {
    if (result == EvaluationResult.True)
     return EvaluationResult.True;
   }
   else
   {
    if (result == EvaluationResult.False)
     wasNot = true;
   }
  }
  return EvaluationResult.False;
 }
}


Thank you for reading! Here comes a video of our cat Prime trying fresh catnip for the first time!
All code provided as-is. This is copied from my own code-base, May need some additional programming to work.

Monday, July 3, 2017

Expression evaluation over time

We have looked at different aspects of knowledge representation during this series and this time its time to look at time itself. How to model logical expressions that incorporate time? I.e. how to store knowledge and query it with time based queries?

Other articles in the AI Knowledge Based Reasoning series on this site:
Knowledge based reasoning in .net c#
Reasoning in Open or Closed models
Logic Expression evaluation with open-world assumption
Expression evaluation on object based models
Expression evaluation over time
Expression evaluation over time - Was?

Introduction

To be able to model logic expressions that incorporate time we first have to ask ourselves, what do we mean by time? Is it abstract ideas like before, after, now and then? Or is it exact number of ticks since we began to measure? Or is it a combination of both?

This part will be quite heavy on ideas rather then code..
Turns out that I provided code for the examples as well. Enjoy : )

Discrete Time

Lets first look at some definitions of discrete time.
Discrete time views values of variables as occurring at distinct, separate "points in time", or equivalently as being unchanged throughout each non-zero region of time ("time period")—that is, time is viewed as a discrete variable -wikipedia
So basically what we say is that even though time is continuous, we will look at it in a discrete way just to simplify our models.

Events

Lets say that we receive knowledge in the form of events, all data recorded in the event will be added to our Context as one frame. For example:
Time Frame Name Occupation Eye-color Hair-color
1 Kate Student Blue Brown
2 Kate Pearlescent
3 Kate Pink
4 Kate Student Blue
5 Kate Intern Blonde
6 Kate
Pearlescent
7 Kate Lawyer Brown
Time frame can be incremental number if we don't care about actual time passed or a time stamp of when the even occurred.
So lets store all our data in this form.

Full frame from incomplete frames

How to query something like the above example?
One idea is to see each event as its own frame of context. I.e. all expressions written in previous posts would now be transferred to refer only to one frame at a time. But that would be kind of incomplete as all events do not record data on all possible parameters. One way to work around this would be to always start from the latest frame and search backwards in the frame-stack until that parameter gets hit. For example in the example above, a full 'latest' frame of information would be the blue highlighted information as seen in the following table:
Time Frame Name Occupation Eye-color Hair-color
4 Kate Student Blue
5 Kate Intern Blonde
6 Kate
Pearlescent
7 Kate Lawyer Brown

Occupation(Lawyer) AND Eye-color(Blue) AND Hair-color(Brown)
Would return True

Occupation(Intern) AND Eye-color(Blue) AND Hair-color(Brown)
Would evaluate to False, as it is not the last full aggregate frame. See Occupation column has newer entries then Intern.

Occupation(Lawyer) AND Eye-color(Blue) AND Drives(Volvo XC90)
Would evaluate to Not Sure as there is no event recording any knowledge about what car Kate drives.

Lets look at some C# code to do just that. Lets convert our Evaluate method in the Context class to the following instead:
public EvaluationResult Evaluate(AExpression expression, int frameStart = -1)
{
    if (frameStart == -1)
        frameStart = _frames.Count - 1;
    var facts = new Dictionary<ExpressionLeaf, EvaluationResult>();
    var leafNodes = expression.GetLeafNodes();
    foreach (var node in leafNodes)
    {
        var leaf = node.Leaf;
        var attr = leaf as KnowledgeAttribute;
        var rel = leaf as KnowledgeRelation;
        if (!(attr != null | rel != null))
            continue;
        for (int i = frameStart; i >= 0; i--)
        {
            var frame = _frames[i];
            var result = attr != null ? frame.Evaluate(attr) : frame.Evaluate(rel);

            if (result == EvaluationResult.NotSure)
                continue;
            facts.Add(node, result);
            break;
        }
    }
    if (!facts.Any())
        return EvaluationResult.NotSure;
    if (facts.Values.Any(x => x == EvaluationResult.NotSure))
        return EvaluationResult.NotSure;
    return expression.TransformEvaluation(facts);
}
And the frame/event store looks something like this (still in the Context class):
private readonly List<KnowledgeStore> _frames;
public IReadOnlyCollection<KnowledgeStore> Frames => new ReadOnlyCollection<KnowledgeStore>(_frames);
public KnowledgeStore CurrentFrame => _frames.Last();
public void AddFrame(KnowledgeStore frame)
{
    _frames.Add(frame);
}

Relative queries

So now we have covered how to determine the current state from this event based knowledge store. Next step is to start determining how knowledge relates to each other based on relative time queries.

For this we will need to add an Evaluate method in the Context class for just relative queries:
public EvaluationResult Evaluate(IRelative expression)
{
    return expression.Evaluate(this);
}

All keyword

Time Frame Name Occupation Eye-color Hair-color
1 Kate Student Blue Brown
2 Kate Pearlescent
3 Kate Pink
4 Kate Student Blue
5 Kate Intern Blonde
6 Kate
Pearlescent
7 Kate Lawyer Brown
This one is pretty simple, it just states that All knowledge frames in the store should evaluate the expression to true.
I.e. ALL Name(Kate) AND Eye-color(Blue)
would be True.
But ALL Occupation(Lawyer) would evaluate to False as there exists frames where this statement is false.
And as before, including a fact from a column that has no recorded data in the query would result in Not Sure.

public class RelativeAll : IRelative
{
    private readonly AExpression _expression;

    public RelativeAll(AExpression expression)
    {
        _expression = expression;
    }

    public EvaluationResult Evaluate(Context context)
    {
        var result = EvaluationResult.NotSure;
        for (int frameIndex = context.Frames.Count - 1; frameIndex >= 0; frameIndex--)
        {
            result = context.Evaluate(_expression, frameIndex);
            if(result == EvaluationResult.False)
                return EvaluationResult.False;
        }
        return result;
    }
}

Some unit tests to see how it behaves:
[TestMethod]
public void RelativeAllTest_AddFrame_CurrentUpdated()
{
    Context target;
    var obj = CreateTarget(out target);

    var expr = new ExpressionIs(new KnowledgeRelation { Subject = "Lawyer", Relation = "Occupation of", Target = obj.ToString() });
    var actual = target.Evaluate(expr);
    Assert.AreEqual(EvaluationResult.True, actual);
}

[TestMethod]
public void RelativeAllTest_All_True()
{
    Context target;
    var obj = CreateTarget(out target);

    var expr = new ExpressionIs(new KnowledgeRelation
    {
        Subject = "Alice",
        Relation = "Name of",
        Target = obj.ToString()
    });
    var rel = new RelativeAll(expr);
    var actual = target.Evaluate(rel);
    Assert.AreEqual(EvaluationResult.True, actual);
}

[TestMethod]
public void RelativeAllTest_All_False()
{
    Context target;
    var obj = CreateTarget(out target);

    var expr = new ExpressionIs(new KnowledgeRelation
    {
        Subject = "Lawyer",
        Relation = "Occupation of",
        Target = obj.ToString()
    });
    var rel = new RelativeAll(expr);
    var actual = target.Evaluate(rel);
    Assert.AreEqual(EvaluationResult.False, actual);
}

[TestMethod]
public void RelativeAllTest_All_NotSure()
{
    Context target;
    var obj = CreateTarget(out target);

    var expr = new ExpressionIs(new KnowledgeRelation
    {
        Subject = "Undercut",
        Relation = "Hair style of",
        Target = obj.ToString()
    });
    var rel = new RelativeAll(expr);
    var actual = target.Evaluate(rel);
    Assert.AreEqual(EvaluationResult.NotSure, actual);
}

private static Person CreateTarget(out Context target)
{
    var obj = new Person("Alice");
    obj.Occupation = "Student";
    target = new Context(obj.ToString());
    target.AddFrame(FrameFactory.Create(obj));
    obj.Occupation = "Lawyer";
    target.AddFrame(FrameFactory.Create(obj));
    return obj;
}

Before keyword

Time Frame Name Occupation Eye-color Hair-color
4 Kate Student Blue
5 Kate Intern Blonde
6 Kate
Pearlescent
7 Kate Lawyer Brown

Syntax: Expression1 Before Expression2
A little harder, trying to determine if Expression1 evaluates true in a full frame before Expression2 evaluates true.
For example:
(Name(Kate) AND Occupation(Intern) AND Hair-color(Blonde)) Before (Name(Kate) AND Occupation(Intern) AND Hair-color(Pearlescent))
Would evaluate to True.

(Name(Kate) AND Occupation(Intern) AND Hair-color(Pearlescent)) Before (Name(Kate) AND Occupation(Intern) AND Hair-color(Blonde))
Would evaluate to False.

And as before, including a fact from a column that has no recorded data in the query would result in Not Sure.

Tricky one is if only one of the expressions evaluate but the other comes back Not Sure, I think for sake of consistency we should evaluate that as Not Sure also.
public class RelativeBefore : IRelative
{
    private readonly AExpression _left;
    private readonly AExpression _right;

    public RelativeBefore(AExpression left, AExpression right)
    {
        _left = left;
        _right = right;
    }

    public EvaluationResult Evaluate(Context context)
    {
        int rightFrameIndex;
        var rightResult = EvaluateExpression(context, _right, out rightFrameIndex);
        int leftFrameIndex;
        var leftResult = EvaluateExpression(context, _left, out leftFrameIndex);

        if (leftResult == EvaluationResult.NotSure || rightResult == EvaluationResult.NotSure)
            return EvaluationResult.NotSure;

        if (leftResult == EvaluationResult.True && rightResult == EvaluationResult.True)
        {
            return leftFrameIndex < rightFrameIndex ? EvaluationResult.True : EvaluationResult.False;
        }
        return EvaluationResult.False;
    }

    private EvaluationResult EvaluateExpression(Context context, AExpression expression, out int frameIndex)
    {
        var result = EvaluationResult.NotSure;
        for (frameIndex = context.Frames.Count - 1; frameIndex >= 0; frameIndex--)
        {
            result = context.Evaluate(expression, frameIndex);
            if (result == EvaluationResult.True)
            {
                return EvaluationResult.True;
            }
        }
        return result;
    }
}

Some tests to clarify things:
[TestMethod]
public void RelativeBeforeTest_True()
{
    Context target;
    var obj = CreateTarget(out target);

    var expr1 = new ExpressionIs(new KnowledgeRelation { Subject = "Student", Relation = "Occupation of", Target = obj.ToString() });
    var expr2 = new ExpressionIs(new KnowledgeRelation { Subject = "Lawyer", Relation = "Occupation of", Target = obj.ToString() });
    var relative = new RelativeBefore(expr1, expr2);
    var actual = target.Evaluate(relative);
    Assert.AreEqual(EvaluationResult.True, actual);
}
[TestMethod]
public void RelativeBeforeTest_False()
{
    Context target;
    var obj = CreateTarget(out target);

    var expr1 = new ExpressionIs(new KnowledgeRelation { Subject = "Student", Relation = "Occupation of", Target = obj.ToString() });
    var expr2 = new ExpressionIs(new KnowledgeRelation { Subject = "Lawyer", Relation = "Occupation of", Target = obj.ToString() });
    var relative = new RelativeBefore(expr2, expr1);
    var actual = target.Evaluate(relative);
    Assert.AreEqual(EvaluationResult.False, actual);
}
[TestMethod]
public void RelativeBeforeTest_LeftNotSure_NotSure()
{
    Context target;
    var obj = CreateTarget(out target);

    var expr1 = new ExpressionIs(new KnowledgeRelation { Subject = "Undercut", Relation = "Hair style of", Target = obj.ToString() });
    var expr2 = new ExpressionIs(new KnowledgeRelation { Subject = "Lawyer", Relation = "Occupation of", Target = obj.ToString() });
    var relative = new RelativeBefore(expr1, expr2);
    var actual = target.Evaluate(relative);
    Assert.AreEqual(EvaluationResult.NotSure, actual);
}
[TestMethod]
public void RelativeBeforeTest_RightNotSure_NotSure()
{
    Context target;
    var obj = CreateTarget(out target);

    var expr1 = new ExpressionIs(new KnowledgeRelation { Subject = "Undercut", Relation = "Hair style of", Target = obj.ToString() });
    var expr2 = new ExpressionIs(new KnowledgeRelation { Subject = "Lawyer", Relation = "Occupation of", Target = obj.ToString() });
    var relative = new RelativeBefore(expr2, expr1);
    var actual = target.Evaluate(relative);
    Assert.AreEqual(EvaluationResult.NotSure, actual);
}
private static Person CreateTarget(out Context target)
{
    var obj = new Person("Alice");
    obj.Occupation = "Student";
    target = new Context(obj.ToString());
    target.AddFrame(FrameFactory.Create(obj));
    obj.Occupation = "Lawyer";
    target.AddFrame(FrameFactory.Create(obj));
    return obj;
}


Conclusions and future work

The method to determine a 'full' frame by aggregating each column backwards gives room for uncertainty as we don't really consider the time between two events that are aggregated together into one frame, if we only have a few records of knowledge and the time between those events is years, then maybe we should be a little bit more uncertain.. a little bit more fuzzy in our determination. Or perhaps return a probability value together with the result. I.e.
True: 0.95
False: 0.1
NotSure: 0.4
And that way be able to build a little more smooth evaluations. But for now this suites my needs.
Another improvement could be to add variance per knowledge column. I.e. if the name is always Kate, then variance should be 0, but for the hair color in the example above it should be way higher as it changes for each event..
Addition of more types of relative expressions could be useful. Exists, Followed by, X years ago etc. But here I think the requirements for each type of project needs to guide the way.

For example source code, head over to my github repository and play around for yourself:


All code provided as-is. This is copied from my own code-base, May need some additional programming to work.

Saturday, May 13, 2017

Expression evaluation on object based models


Let's look at how to mix object based models with logical expression evaluation.

Other articles in the AI Knowledge Based Reasoning series on this site:
Knowledge based reasoning in .net c#
Reasoning in Open or Closed models
Logic Expression evaluation with open-world assumption
Expression evaluation on object based models
Expression evaluation over time
Expression evaluation over time - Was?

Many systems work on an object model of some kind, to be able to integrate logical expression evaluation to an existing system you need some way to translate from that object model to a knowledge model. This is one way to do it, there are probably better ways out there. But this works for me : )

To be able to evaluate logical expressions on an object model, you first have to translate the object model into a Knowledge model. The idea here is to scope the amount of knowledge to a certain object (or objects), lets call that scope a Context, in the case of objects lets call it an ObjectContext.
public class Context
{
 public Guid Id { get; }
 public string Name { get; }
   
 public KnowledgeStore KnowledgeStore { get; }

 public Context(string name)
 {
  Id = Guid.NewGuid();
  Name = name;
  KnowledgeStore = new KnowledgeStore(this);
 }

 public EvaluationResult Evaluate(AExpression expression)
 {
  var facts = new Dictionary<ExpressionLeaf, EvaluationResult>();
  var leafNodes = expression.GetLeafNodes();
  foreach (var node in leafNodes)
  {
   var leaf = node.Leaf;
   if(leaf is KnowledgeNon)
    continue;
   var attr = leaf as KnowledgeAttribute;
   if (attr != null)
   {
    facts.Add(node, KnowledgeStore.Evaluate(attr));
   }
   var rel = leaf as KnowledgeRelation;
   if (rel != null)
   {
    facts.Add(node, KnowledgeStore.Evaluate(rel));
   }
  }
  if (facts.Values.Any(x => x == EvaluationResult.NotSure))
   return EvaluationResult.NotSure;
  return expression.TransformEvaluation(facts);
 }
}
So, the base Context object lets us Evaluate an Expression, as we assume an open world we also allow for the response NotSure.

And for the ObjectContext, we will use reflection to create the Knowledge from the objects that we throw at it.In my case, my model inherits from a BaseObject class, but it could be any object.
public class ObjectContext : Context
{
 public BaseObject Root { get; set; }
 public ObjectContext(BaseObject root) : base(root.ToString())
 {
  Root = root;
  KnowledgeStore.AddAttribute(new KnowledgeAttribute
  {
   Attribute = root.GetType().Name,
   Subject = Name
  }, Name);
  var fields = GetAllProperties(root);
  foreach (var field in fields)
  {
   KnowledgeStore.AddRelation(new KnowledgeRelation
   {
    Subject = field.Value,
    Relation = $"{field.Key}Of",
    Target = Name
   }, Name);
  }
 }

 private Dictionary<string, string> GetAllProperties(object obj)
 {
  var d = new Dictionary<string, string>();
  var properties = obj.GetType().GetProperties();
  foreach (var prop in properties)
  {
   var val = prop.GetValue(obj);
   if (val == null)
    val = string.Empty;
   d.Add(prop.Name, val.ToString());
  }
  return d;
 }
}

So basically we will take a snapshot of the model and allow for expression evaluation on that snapshot. If the underlying model changes, the context object would need to be recreated.
As you can see we add a suffix to each attribute name, so if the object has a field called Name, the knowledge attribute would be NameOf.

Lets look at some unit tests to see how this works.
[TestClass]
public class ObjectContextTest
{
 [TestMethod]
 public void ContextTest_Evaluate_Not_False()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionNot(new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() });
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.False, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_Not_True()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionNot(new KnowledgeRelation { Relation = "NameOf", Subject = "T", Target = obj.ToString() });
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.True, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_And_BothTrue()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionAnd(
   new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() },
   new KnowledgeRelation { Relation = "NameOf", Subject = "Charlize", Target = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.True, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_And_RightFalse()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionAnd(
   new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() },
   new KnowledgeRelation { Relation = "NameOf", Subject = "T", Target = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.False, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_And_LeftFalse()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionAnd(
   new KnowledgeRelation { Relation = "NameOf", Subject = "T", Target = obj.ToString() },
   new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.False, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_And_BothFalse()
 {
  var obj = new Person("Charlize") { Gender = Person.GenderType.Female };
  var target = new ObjectContext(obj);
  var expr = new ExpressionAnd(
   new KnowledgeRelation { Relation = "NameOf", Subject = "T", Target = obj.ToString() },
   new KnowledgeRelation { Relation = "GenderOf", Subject = "male", Target = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.False, actual);
 }
 [TestMethod]
 public void ContextTest_Evaluate_Or_BothTrue()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionOr(
   new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() },
   new KnowledgeRelation { Relation = "NameOf", Subject = "Kate", Target = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.True, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_Or_RightFalse()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionOr(
   new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() },
   new KnowledgeRelation { Relation = "NameOf", Subject = "Kate", Target = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.True, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_Or_LeftFalse()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionOr(
   new KnowledgeRelation { Relation = "NameOf", Subject = "Kate", Target = obj.ToString() },
   new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.True, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_Or_BothFalse()
 {
  var obj = new Person("Charlize") { Gender = Person.GenderType.Female };
  var target = new ObjectContext(obj);
  var expr = new ExpressionOr(
   new KnowledgeRelation { Relation = "NameOf", Subject = "Kate", Target = obj.ToString() },
   new KnowledgeRelation { Relation = "GenderOf", Subject = "Male", Target = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.False, actual);
 }


 [TestMethod]
 public void ContextTest_Evaluate_Xor_BothTrue()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionXor(
   new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() },
   new KnowledgeRelation { Relation = "NameOf", Subject = "Charlize", Target = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.False, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_Xor_RightFalse()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionXor(
   new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() },
   new KnowledgeRelation { Relation = "NameOf", Subject = "Kate", Target = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.True, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_Xor_LeftFalse()
 {
  var obj = new Person("Charlize");
  var target = new ObjectContext(obj);
  var expr = new ExpressionXor(
   new KnowledgeRelation { Relation = "NameOf", Subject = "Kate", Target = obj.ToString() },
   new KnowledgeAttribute { Attribute = "Person", Subject = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.True, actual);
 }

 [TestMethod]
 public void ContextTest_Evaluate_Xor_BothFalse()
 {
  var obj = new Person("Charlize") { Gender = Person.GenderType.Female };
  var target = new ObjectContext(obj);
  var expr = new ExpressionXor(
   new KnowledgeRelation { Relation = "NameOf", Subject = "T", Target = obj.ToString() },
   new KnowledgeRelation { Relation = "GenderOf", Subject = "female", Target = obj.ToString() }
   );
  var actual = target.Evaluate(expr);
  Assert.AreEqual(EvaluationResult.False, actual);
 }
}

So, just some thoughts on how to do this. This is far from completed and is still a work in progress used by one of my home projects. Any comments are appreciated.
If you wonder why this is not on my github repository, I feel that it doesn't really have that kind of quality yet. But with time it will get there.

All code provided as-is. This is copied from my own code-base, May need some additional programming to work.


Sunday, April 9, 2017

DIY Game Financial System Part 2: Trade Algorithm


This is part of my series on DIY Game Financial System
For other posts in this series check out the following links:
DIY Game Financial System Part 1: Intro and Accounts
DIY Game Financial System Part 2: Trade Algorithm

For the full source code for the financial system discussed in this series, look in the repository.
This post we will look at the actual trade algorithm used. It is a quite naive implementation but works for me. Feel free to extend it for your own needs.

In the Marketplace class, we have an Update method that iterates through all the buy and sell orders available and tries to match them with each other. This simple implementation tries to find an exact match but there is potential to extend it to match multiple sell to one buy, or one sell to multiple sell.
In the demo application this is triggered by a timer.

public void Update()
{
 bool updated = false;
 lock(_sellOrders)
 {
  lock (_buyOrders)
  {
   foreach (var buyName in _buyOrders)
   {
    List<SellOrder> sellOrders;
    if (!_sellOrders.TryGetValue(buyName.Key, out sellOrders))
     continue;
    // naive
    foreach (var buyOrder in buyName.Value)
    {
     foreach (var sellOrder in sellOrders)
     {
      if (buyOrder.Quantity == sellOrder.Quantity
       && buyOrder.PricePerItem == sellOrder.PricePerItem)
      {
       updated = true;
       buyOrder.Execute(sellOrder);
      }
     }
    }
   }
   foreach (var order in _sellOrdersToBeRemoved)
   {
    _sellOrders[order.SecurityName].Remove(order);
   }
   foreach (var order in _buyOrdersToBeRemoved)
   {
    _buyOrders[order.SecurityName].Remove(order);
   }

  }
 }
 if (updated)
 {
  Updated?.Invoke(this, this);
 }
}

The key lines above are the
if (buyOrder.Quantity == sellOrder.Quantity
 && buyOrder.PricePerItem == sellOrder.PricePerItem)
{
 updated = true;
 buyOrder.Execute(sellOrder);
}
This tries to exactly match each buy with a sell and then trigger the Execute method:
public void Execute(SellOrder sellOrder)
{
    OrderAccount.DepositInto(sellOrder.OrderAccount, sellOrder.Quantity * sellOrder.PricePerItem);
    Security = sellOrder.ForSale;
    foreach (var action in _executeActions)
    {
        action(this);
    }
    sellOrder.Execute();
}
This is where the money is transferred between the two orders, all actions connected to this BuyOrder are triggered and then finally the sellOrder is executed as well.

When the buy order was added to the Marketplace, it associated a Cancel and Execute action with it to be triggered when the order was successful or canceled.
public void Buy(BuyOrder order)
{
 lock (_buyOrders)
 {
  if (_buyOrders.ContainsKey(order.SecurityName))
   _buyOrders[order.SecurityName].Add(order);
  else
   _buyOrders.Add(order.SecurityName, new List<BuyOrder> { order });
  order.AddCancelAction(RemoveBuy);
  order.AddExecuteAction(RemoveBuy);
 }
}
For both cases, we are just interested in removing the order from the list of available orders. The same goes for the Sell method.
private void RemoveBuy(BuyOrder order)
{
 _buyOrdersToBeRemoved.Add(order);
}
As this is triggered from an iteration of the _buyOrders collection, we need to deffer the removal to after the iteration has completed, hence the addition to a secondary removal list.
Just writing this I spotted a bug in the code where the order needs to be flagged as executed so that the same order isn't executed multiple times. I will not fix this in the demo application for now. 


Last in the Update method, a call to the Updated event handlers associated with this marketplace object.
if (updated)
{
 Updated?.Invoke(this, this);
}
This way a GUI can update itself if there was an update. The Updated event is also fired each time an order is added to the marketplace.

So there, thank you for reading. Hope this helps someone out there : )

Until next time: Work to Live, Don’t Live to Work

Friday, March 10, 2017

DIY Game Financial System Part 1: Intro and Accounts

picture of commerce on river banks in the evening

I'm quite interested in finance as can be seen on my other blog dreamstateliving, and thus I decided to start working on a small but powerful in game financial system that can be used in my future games.
The goal for this system is to allow for
  • trading at a marketplace, much like the stock market
  • buying and selling stuff from vendors
Note that the code provided with this post is intended to be used in games, a lot of shortcuts have been taken. This is not a fool-proof system, more like a base to start building your own in game financial system on.

In this part we will introduce the scenario and go through the Account and TradeAccount classes.

This is part of my series on DIY Game Financial System
For other posts in this series check out the following links:
DIY Game Financial System Part 1: Intro and Accounts
DIY Game Financial System Part 2: Trade Algorithm

Introduction, a textual description

Every player, vendor etc. has their private TradeAccount consisting of a BalanceAccount and a list of Securities (items, shares whatever) owned at his current state.
Each TradeAccount is able to generate a BuyOrder based on a name of the Security to buy, the Quantity of Securities we are shopping for and the PricePerItem we are willing to pay. The BuyOrder  also contains a BalanceAccount with the amount of money needed to make the purchase. It is not possible to generate a BuyOrder if the TradeAccounts' BalanceAccount does not hold the amount of money needed to close the transaction.
In a similar fashion, each TradeAccount is able to generate a SellOrder for Securities currently held by that TradeAccount. The SellOrder also consists of the PricePerItem, Quantity and the Security for sale.

Both the SellOrder and the BuyOrder are sent to the Marketplace where the trade should be made. The Marketplace should be possible to instance as many times as needed. For example each vendor would have their own with things for sale, but the global Marketplace would allow for players to trade.

If there is an exact match on Security, Quantity and PricePerItem we will Execute the BuyOrder and transfer the money to the sellers TradeAccount and the securities to the buyers TradeAccount.
This will be our first naive algorithm to match orders.

Lets look at some code.


BalanceAccount

This is just a holder of money. Lets look at the interface.
public interface IAccount
{
    Guid Id { get; }
    ulong Balance { get; }
    bool CanAfford(ulong x);
    bool DepositInto(IAccount dest, ulong amount);
    void AddBalance(ulong amount);
}
So first we have an identifier, Id, just so that we can identify this account uniquely. Then we will store the actual current Balance of the account. I chose an unsigned long to store because of three reasons:

  • I don't really want to handle negative numbers, i.e. if you cannot afford it you can't buy it.
  • to be able to handle very large sums, I don't really know the applications for this yet, but don't want to limit someone for buying something for a ridiculous amount of money (18,446,744,073,709,551,615 to be exact)
  • I don't want to handle decimal numbers. Keep it simple, its a game :)
After that we have some functions, CanAfford checks if the account could afford to buy something costing x. 
public bool CanAfford(ulong x)
{
    return Balance >= x;
}
DepositInto withdraws money from this account and inserts it into the destination account.
public bool DepositInto(IAccount dest, ulong amount)
{
    lock (this)
    {
        if (!CanAfford(amount))
            return false;
        SubtractBalance(amount);
    }
    lock (dest)
    {
        dest.AddBalance(amount);
    }
    return true;
}
And lastly AddBalance creates money out of the blue (or you handle it somewhere else) and inserts into this account.
public void AddBalance(ulong amount)
{
    lock (this)
    {
        Balance += amount;
    }
}
private void SubtractBalance(ulong amount)
{
    Balance -= amount;
}
As you can see we add an extra lock in the AddBalance function as it is a public one and could be called from outside the DepositInto lock. For the private SubtractBalance method, we have full control and don't need the additional lock overhead.
There is a number of unit tests to verify this implementation in the source code in the repository. It gets quite long and thus not included in the post.

TradeAccount

public interface ITradeAccount
{
    event EventHandler<TradeAccount> Updated;
    IAccount BalanceAccount { get; }
    ulong ValueOfActiveBuyOrders { get; }
    Dictionary<string, Security> Securities { get; }
    List<BuyOrder> BuyOrders { get; }
    List<SellOrder> SellOrders { get; }

    SellOrder CreateSellOrder(string name, ulong quantity, ulong pricePerItem);
    BuyOrder CreateBuyOrder(string name, ulong quantity, ulong pricePerItem);
}

First off, we will expose an event, Updated, that will be triggered whenever the contents of this Trade Account change. This to be able to create event based updates of GUI elements at a later stage.
After that we have the BalanceAccount associated with this TradeAccount, for storage of money that can be used to buy stuff.
ValueOfActiveBuyOrders will return the amount of money bound in buy orders created by this TradeAccount.
public ulong ValueOfActiveBuyOrders
{
    get
    {
        ulong sum = 0;
        foreach (var item in BuyOrders)
        {
            sum += item.OrderAccount.Balance;
        }
        return sum;
    }
}
Securities is a dictionary of securities held by this account followed bu the currently active BuyOrders and SellOrders.

Lastly we expose functions to create both Sell and Buy orders.

CreateSellOrder

public SellOrder CreateSellOrder(string name, ulong quantity, ulong pricePerItem)
{
    Security security; 
    if (!Securities.TryGetValue(name, out security))
        return null;
    var split = security.Split(quantity);
    if (split == null)
        return null;
    var order = new SellOrder(split, pricePerItem);
    order.AddCancelAction(CancelSellOrder);
    order.AddExecuteAction(ExecuteSellOrder);
    lock (SellOrders)
    {
        SellOrders.Add(order);
    }
    Updated?.Invoke(this, this);
    return order;
}
To createa a SellOrder we first need to check if we actually own any securities with the name provided, if we don't we will stop directly. After that we split our owned security into a new one with the quantity we want to sell. We expect the Split method to return null if the split is invalid (tries to sell more then owned) and we stop there as well. If all is well, we setup the SellOrder, connect both Cancel and Execute handlers to this TradeAccount so that it will know its way home when the mission is accomplished or canceled. Lastly, we add it to the list of active SellOrders and invoke the Updated event.
private void ExecuteSellOrder(SellOrder order)
{
    order.OrderAccount.DepositInto(BalanceAccount, order.OrderAccount.Balance);
    lock (SellOrders)
    {
        SellOrders.Remove(order);
    }
    Updated?.Invoke(this, this);
}
The ExecuteSellOrder is responsible to transfer money to the selling TradeAccount when a trade is completing. As seen, the SellOrder has already received the money from the buying account so this is just a final put the money where it belongs and cleanup things handler.
private void CancelSellOrder(SellOrder order)
{
    Security security;
    if (!Securities.TryGetValue(order.SecurityName, out security))
    {
        Securities.Add(order.SecurityName, order.ForSale);
    }
    else
    {
        Securities[order.SecurityName].Merge(order.ForSale);
    }
    lock (SellOrders)
    {
        SellOrders.Remove(order);
    }
    Updated?.Invoke(this, this);
}
The CancelSellOrder is a little more complicated. We are not receiving any money here but we need to put the Securities back where they belong. So find the security we were trying to sell and either merge with the existing one or create a new entry (if it has been cleaned up for any reason). After that some cleanup and update event invoking.

CreateBuyOrder

The CreateBuyOrder basically follows the same pattern as the sell order creator but here we check that we have enough money to make the purchase that we want to.
We do this by creating a new account and trying to deposit the needed amount of money into it. If it fails, we stop there. If all is good, we create the BuyOrder and supply it the just created account and what to look for in the marketplace. Just as for the SellOrder, we connect handlers for both the Execute and Cancel outcomes and add it to the list of active buy orders.
public BuyOrder CreateBuyOrder(string name, ulong quantity, ulong pricePerItem)
{
    var orderAccount = new Account(0);
    if (!BalanceAccount.DepositInto(orderAccount, quantity*pricePerItem))
        return null;
    var order = new BuyOrder(name, pricePerItem, quantity, orderAccount);
    order.AddCancelAction(CancelBuyOrder);
    order.AddExecuteAction(ExecuteBuyOrder);
    lock (BuyOrders)
    {
        BuyOrders.Add(order);
    }
    Updated?.Invoke(this, this);
    return order;
}
The ExecuteBuyOrder looks very similar to the CancelSellOrder above, instead of putting back a canceled sell order, we will put in a newly purchased Security. It needs to find the already owned security and merge with it if it exists or create a new entry in the owned Securites dictionary. After that a cleanup is done.
private void ExecuteBuyOrder(BuyOrder order)
{
    Security security;
    if (!Securities.TryGetValue(order.SecurityName, out security))
    {
        Securities.Add(order.SecurityName, order.Security);
    }
    else
    {
        Securities[order.SecurityName].Merge(order.Security);
    }
    lock (BuyOrders)
    {
        BuyOrders.Remove(order);
    }
    Updated?.Invoke(this, this);
}
And surprise, surprise the CancelBuyOrder is similar to the ExecuteSellOrder. Here we put back the pocket-money we received when going to the market and then cleaning up.
private void CancelBuyOrder(BuyOrder order)
{
    order.OrderAccount.DepositInto(BalanceAccount, order.OrderAccount.Balance);
    lock (BuyOrders)
    {
        BuyOrders.Remove(order);
    }
    Updated?.Invoke(this, this);
}

So just like with the Account, there are a number of unit tests in the repository  that I will not cover here. Actually the repository contains the code for the first working version with a test GUI. We will cover the rest of the code in future posts in this series, but if you are interested you can look at the code already now :)

So there, thank you for reading. Hope this helps someone out there : )

Until next time: Work to Live, Don’t Live to Work

Monday, February 27, 2017

OpenGL 4 with OpenTK in C# Part 14: Basic Text


In this post we will look at how to get basic text on screen so that we can display the score of the game to the player.

This is part 14 of my series on OpenGL4 with OpenTK.
For other posts in this series:
OpenGL 4 with OpenTK in C# Part 1: Initialize the GameWindow
OpenGL 4 with OpenTK in C# Part 2: Compiling shaders and linking them
OpenGL 4 with OpenTK in C# Part 3: Passing data to shaders
OpenGL 4 with OpenTK in C# Part 4: Refactoring and adding error handling
OpenGL 4 with OpenTK in C# Part 5: Buffers and Triangle
OpenGL 4 with OpenTK in C# Part 6: Rotations and Movement of objects
OpenGL 4 with OpenTK in C# Part 7: Vectors and Matrices
OpenGL 4 with OpenTK in C# Part 8: Drawing multiple objects
OpenGL 4 with OpenTK in C# Part 9: Texturing
OpenGL 4 with OpenTK in C# Part 10: Asteroid Invaders
OpenGL 4 with OpenTK in C# Part 11: Mipmap
OpenGL 4 with OpenTK in C# Part 12: Basic Moveable Camera
OpenGL 4 with OpenTK in C# Part 13: IcoSphere
OpenGL 4 with OpenTK in C# Part 14: Basic Text

As stated in the previous post, I am in no way an expert in OpenGL. I write these posts as a way to learn and if someone else finds these posts useful then all the better :)
If you think that the progress is slow, then know that I am a slow learner :P
This part will build upon the game window and shaders from part 13.

Generate font texture

First off, we need to generate a texture for the font that we want to use. For simplicity, all letters that we will need are put on a single line and each letter will get a fixed size rectangle in the texture.
private const string Characters = @"qwertyuiopasdfghjklzxcvbnmQWERTYUIOPASDFGHJKLZXCVBNM0123456789µ§½!""#¤%&/()=?^*@£€${[]}\~¨'-_.:,;<>|°©®±¥";
public Bitmap GenerateCharacters(int fontSize, string fontName, out Size charSize)
{
    var characters = new List<Bitmap>();
    using (var font = new Font(fontName, fontSize))
    {
        for (int i = 0; i < Characters.Length; i++)
        {
            var charBmp = GenerateCharacter(font, Characters[i]);
            characters.Add(charBmp);
        }
        charSize = new Size(characters.Max(x => x.Width), characters.Max(x => x.Height));
        var charMap = new Bitmap(charSize.Width * characters.Count, charSize.Height);
        using (var gfx = Graphics.FromImage(charMap))
        {
            gfx.FillRectangle(Brushes.Black, 0, 0, charMap.Width, charMap.Height);
            for (int i = 0; i < characters.Count; i++)
            {
                var c = characters[i];
                gfx.DrawImageUnscaled(c, i * charSize.Width, 0);

                c.Dispose();
            }
        }
        return charMap;
    }
}

private Bitmap GenerateCharacter(Font font, char c)
{
    var size = GetSize(font, c);
    var bmp = new Bitmap((int)size.Width, (int)size.Height);
    using (var gfx = Graphics.FromImage(bmp))
    {
        gfx.FillRectangle(Brushes.Black, 0, 0, bmp.Width, bmp.Height);
        gfx.DrawString(c.ToString(), font, Brushes.White, 0, 0);
    }
    return bmp;
} 
private SizeF GetSize(Font font, char c)
{
    using (var bmp = new Bitmap(512, 512))
    {
        using (var gfx = Graphics.FromImage(bmp))
        {
            return  gfx.MeasureString(c.ToString(), font);
        }
    }
}


The GenerateCharacters method takes the name of the font that we want to use, the size of the font and outputs the size of a single character in the returned texture. So white on black.
output from font texture generator
Example output from above algorithm. Original texture was 3432x48 pixels
This texture will be used to calculate the alpha channel of the color that we want to render on screen. I.e. whatever is white on the texture will be rendered as the chose color and the black will be rendered as transparent.

Render Objects

Basically we want to render each character in a quad, so we need to generate a model with 2 triangles that can be reused. So into our RenderObjectFactory we add the following:
public static TexturedVertex[] CreateTexturedCharacter()
{
    float h = 1;
    float w = RenderText.CharacterWidthNormalized;
    float side = 1f / 2f; // half side - and other half

    TexturedVertex[] vertices =
    {
        new TexturedVertex(new Vector4(-side, -side, side, 1.0f),    new Vector2(0, h)),
        new TexturedVertex(new Vector4(side, -side, side, 1.0f),     new Vector2(w, h)),
        new TexturedVertex(new Vector4(-side, side, side, 1.0f),     new Vector2(0, 0)),
        new TexturedVertex(new Vector4(-side, side, side, 1.0f),     new Vector2(0, 0)),
        new TexturedVertex(new Vector4(side, -side, side, 1.0f),     new Vector2(w, h)),
        new TexturedVertex(new Vector4(side, side, side, 1.0f),      new Vector2(w, 0)),
    };
    return vertices;
}
It is the front facing quad from the generate cube method already there.
The RenderText.CharacterWidthNormalized is returning the 1/number of characters to get the correct alignment of the x-axis.

We will be needing 2 new render objects to accomplish putting text on the screen. RenderText that handles the whole string to be rendered, and RenderCharacter that handles each individual character in the string.
public class RenderText : AGameObject
{
    private readonly Vector4 _color;
    public const string Characters = @"qwertyuiopasdfghjklzxcvbnmQWERTYUIOPASDFGHJKLZXCVBNM0123456789µ§½!""#¤%&/()=?^*@£€${[]}\~¨'-_.:,;<>|°©®±¥";
    private static readonly Dictionary<char, int> Lookup;
    public static readonly float CharacterWidthNormalized;
    // 21x48 per char, 
    public readonly List<RenderCharacter> Text;
    static RenderText()
    {
        Lookup = new Dictionary<char, int>();
        for (int i = 0; i < Characters.Length; i++)
        {
            if (!Lookup.ContainsKey(Characters[i]))
                Lookup.Add(Characters[i], i);
        }
        CharacterWidthNormalized = 1f / Characters.Length;
    }
    public RenderText(ARenderable model, Vector4 position, Color4 color, string value)
        : base(model, position, Vector4.Zero, Vector4.Zero, 0)
    {
        _color = new Vector4(color.R, color.G, color.B, color.A);
        Text = new List<RenderCharacter>(value.Length);
        _scale = new Vector3(0.02f);
        SetText(value);
    }
    public void SetText(string value)
    {
        Text.Clear();
        for (int i = 0; i < value.Length; i++)
        {
            int offset;
            if (Lookup.TryGetValue(value[i], out offset))
            {
                var c = new RenderCharacter(Model,
                    new Vector4(_position.X + (i * 0.015f),
                        _position.Y,
                        _position.Z,
                        _position.W),
                    (offset*CharacterWidthNormalized));
                c.SetScale(_scale);
                Text.Add(c);
            }
        }
    }
    public override void Render(ICamera camera)
    {
        _model.Bind();
        GL.VertexAttrib4(3, _color);
        for (int i = 0; i < Text.Count; i++)
        {
            var c = Text[i];
            c.Render(camera);
        }
    }
}
From the top.
Characters string containing all the characters in our font texture in the correct order.
Static constructor that initializes the lookup table, mapping each character to its index. Dictionary for faster lookup then doing the index of operation during for each string we want to show.
Instance constructor, just decomposes the Color struct to a Vector4, I still don't know why the GL.VertexAttrib4 doesn't support the Color struct out of the box.
SetText, allows for changing the contents of this RenderText object. This is a naive implementation that empties all content and then adds new. Optimization could be to try re-use whatever objects that already are in the collection. But for now, this works for me.
Render, just sets the color attribute and calls render for all RenderCharacters.

Next, all characters in the string
public class RenderCharacter : AGameObject
{
    private float _offset;

    public RenderCharacter(ARenderable model, Vector4 position, float charOffset)
        : base(model, position, Vector4.Zero, Vector4.Zero, 0)
    {
        _offset = charOffset;
        _scale = new Vector3(0.2f);
    }

    public void SetChar(float charOffset)
    {
        _offset = charOffset;
    }

    public override void Render(ICamera camera)
    {
        GL.VertexAttrib2(2, new Vector2(_offset, 0));
        var t2 = Matrix4.CreateTranslation(
            _position.X,
            _position.Y,
            _position.Z);
        var s = Matrix4.CreateScale(_scale);
        _modelView = s * t2 * camera.LookAtMatrix;
        GL.UniformMatrix4(21, false, ref _modelView);
        _model.Render();
    }
}
The key component for a character is the x-axis offset in the texture. The Render method just binds the offset attribute to the shader and renders the quad holding the character. At the moment the character is transformed with a Model-View matrix.


Shaders

Vertex Shader
#version 450 core
layout(location = 0) in vec4 position;
layout(location = 1) in vec2 textureCoordinate;
layout(location = 2) in vec2 textureOffset;
layout(location = 3) in vec4 color;

out vec2 vs_textureOffset;
out vec4 vs_color;
layout(location = 20) uniform  mat4 projection;
layout (location = 21) uniform  mat4 modelView;

void main(void)
{
 vs_textureOffset = textureCoordinate + textureOffset;
 gl_Position = projection * modelView * position;
 vs_color = color;
}
The first 2 inputs to our vertex shader are bound from buffers, and we introduce 2 new inputs that we set from the render methods. The texture offset and color.
We also need to send the color forward to the fragment shader so we need an out parameter for that.
The vs_textureOffset is the original texture coordinate plus the new offset to find a character. The original texture coordinate the X-axis was of the width of 1 character and that's why this works.

Fragment Shader
#version 450 core
in vec2 vs_textureOffset;
in vec4 vs_color;
uniform sampler2D textureObject;
out vec4 color;

void main(void)
{
 vec4 alpha = texture(textureObject, vs_textureOffset);
 color = vs_color;
 color.a = alpha.r;
}
The vertex shader reads the texture, as it is black and white the red, green and blue channels should have the same values, hence we can look at just one of them and set the alpha channel of our color to get transparency. i.e. the color just sticks to the characters and we cut out everything else.

Game Window

OnLoad
We need to setup some stuff in the OnLoad method of our game window. First out is to setup the new shaders needed to render our text objects.
_textProgram = new ShaderProgram();
_textProgram.AddShader(ShaderType.VertexShader, @"Components\Shaders\1Vert\textVert.c");
_textProgram.AddShader(ShaderType.FragmentShader, @"Components\Shaders\5Frag\textFrag.c");
_textProgram.Link();

Nothing fancy there, next to load our texture and add it to the list of models (for correct disposal)
var textModel = new TexturedRenderObject(RenderObjectFactory.CreateTexturedCharacter(), _textProgram.Id, @"Components\Textures\font singleline.bmp");
models.Add("Quad", textModel);

As we do this in our Asteroid Invaders game, we will be displaying the score. So we need a variable to store this in.
_text = new RenderText(models["Quad"], new Vector4(-0.2f, 0.1f, -0.4f, 1),  Color4.Red, "Score");

And lastly, we need to enable transparency. Otherwise the alpha blending wouldn't bite and we would have a black background instead.
GL.Enable(EnableCap.Blend);
GL.BlendFunc(BlendingFactorSrc.SrcAlpha, BlendingFactorDest.OneMinusSrcAlpha);

OnRenderFrame
The major changes to our OnRenderFrame method consists of updating our score and adding a second render step after all normal game objects for rendering of our transparent objects. This because we need to have whatever we want to show in the transparent areas to be drawn before our text. Otherwise we would get the dark blue background color as a box for all characters.
protected override void OnRenderFrame(FrameEventArgs e)
{
 Title = $"{_title}: FPS:{1f / e.Time:0000.0}, obj:{_gameObjects.Count}, score:{_score}";
 _text.SetText($"Score: {_score}");
 GL.ClearColor(_backColor);
 GL.Clear(ClearBufferMask.ColorBufferBit | ClearBufferMask.DepthBufferBit);

 int lastProgram = -1;
 foreach (var obj in _gameObjects)
 {
  lastProgram = RenderOneElement(obj, lastProgram);
 }
 // render after all opaque objects to get transparency right
 RenderOneElement(_text, lastProgram);
 SwapBuffers();
}

private int RenderOneElement(AGameObject obj, int lastProgram)
{
 var program = obj.Model.Program;
 if (lastProgram != program)
  GL.UniformMatrix4(20, false, ref _projectionMatrix);
 lastProgram = obj.Model.Program;
 obj.Render(_camera);
 return lastProgram;
}

Further work

This is not a fully featured way to render text, but enough to get a score on the screen. There's a lot of things that could be done to improve it and here comes a few:

As the solution is based on a bitmap, it scales kind of bad. There exists a lot of material on how to make that smooth and something that I will look at some time in the future. Distance fields is an area that seems promising and that takes advantage of the GPU. More information in the following links if you want to give it a try:

Positioning on the screen. Today the solution uses world coordinates, works for this static camera scene but not so well with a moving camera if you still want to have the text visible to the user.

Text wrapping, multi row text. Better scaling.
The list goes on and I'll probably end up coding some of the stuff on the list whenever I feel a need for it.

End results

For the complete source code for the tutorial at the end of this part, go to: https://github.com/eowind/dreamstatecoding

So there, thank you for reading. Hope this helps someone out there : )

Until next time: Work to Live, Don’t Live to Work


Sunday, February 19, 2017

OpenGL 4 with OpenTK in C# Part 13: IcoSphere


This time we will look at how to generate a sphere in code, I got tired of the cubes and wanted a little variation. The decision fell on IcoSpheres, mostly as they seem to be more flexible in the long run.

This is part 13 of my series on OpenGL4 with OpenTK.
For other posts in this series:
OpenGL 4 with OpenTK in C# Part 1: Initialize the GameWindow
OpenGL 4 with OpenTK in C# Part 2: Compiling shaders and linking them
OpenGL 4 with OpenTK in C# Part 3: Passing data to shaders
OpenGL 4 with OpenTK in C# Part 4: Refactoring and adding error handling
OpenGL 4 with OpenTK in C# Part 5: Buffers and Triangle
OpenGL 4 with OpenTK in C# Part 6: Rotations and Movement of objects
OpenGL 4 with OpenTK in C# Part 7: Vectors and Matrices
OpenGL 4 with OpenTK in C# Part 8: Drawing multiple objects
OpenGL 4 with OpenTK in C# Part 9: Texturing
OpenGL 4 with OpenTK in C# Part 10: Asteroid Invaders
OpenGL 4 with OpenTK in C# Part 11: Mipmap
OpenGL 4 with OpenTK in C# Part 12: Basic Moveable Camera
OpenGL 4 with OpenTK in C# Part 13: IcoSphere
OpenGL 4 with OpenTK in C# Part 14: Basic Text

As stated in the previous post, I am in no way an expert in OpenGL. I write these posts as a way to learn and if someone else finds these posts useful then all the better :)
If you think that the progress is slow, then know that I am a slow learner :P
This part will build upon the game window and shaders from part 12.


Introduction

So I take no credit for this algorithm to generate the sphere. I found a WPF version over at catch 22, Andreas Kahlers blog. I ported it to work with our OpenTK implementation of vertex buffers and texturing.
Texturing fix for the common artefact of a funny looking stripe that goes from pole to pole is based on a solution described over at sol.gfxil.net.
So, now that credit is where it belongs, lets look at the code.

Code

private struct Face
{
    public Vector3 V1;
    public Vector3 V2;
    public Vector3 V3;

    public Face(Vector3 v1, Vector3 v2, Vector3 v3)
    {
        V1 = v1;
        V2 = v2;
        V3 = v3;
    }
}
First we need a struct where we will store each face of our sphere. Basically just contains 3 vectors for each point in a triangle.

The algorithm from Khalers blog. As noted above, this does not use vertex indexing. Meaning that we have a little memory overhead as every vertex is doubled instead of reused.

Basic algorithm is to generate the initial points manually and then for each iteration split each face into 4 new faces and project them into the unit sphere by normalizing them.

public class IcoSphereFactory
{
    private List<Vector3> _points;
    private int _index;
    private Dictionary<long, int> _middlePointIndexCache;

    public TexturedVertex[] Create(int recursionLevel)
    {
        _middlePointIndexCache = new Dictionary<long, int>();
        _points = new List<Vector3>();
        _index = 0;
        var t = (float)((1.0 + Math.Sqrt(5.0)) / 2.0);
        var s = 1;

        AddVertex(new Vector3(-s, t, 0));
        AddVertex(new Vector3(s, t, 0));
        AddVertex(new Vector3(-s, -t, 0));
        AddVertex(new Vector3(s, -t, 0));

        AddVertex(new Vector3(0, -s, t));
        AddVertex(new Vector3(0, s, t));
        AddVertex(new Vector3(0, -s, -t));
        AddVertex(new Vector3(0, s, -t));

        AddVertex(new Vector3(t, 0, -s));
        AddVertex(new Vector3(t, 0, s));
        AddVertex(new Vector3(-t, 0, -s));
        AddVertex(new Vector3(-t, 0, s));

        var faces = new List<Face>();

        // 5 faces around point 0
        faces.Add(new Face(_points[0], _points[11], _points[5]));
        faces.Add(new Face(_points[0], _points[5], _points[1]));
        faces.Add(new Face(_points[0], _points[1], _points[7]));
        faces.Add(new Face(_points[0], _points[7], _points[10]));
        faces.Add(new Face(_points[0], _points[10], _points[11]));

        // 5 adjacent faces 
        faces.Add(new Face(_points[1], _points[5], _points[9]));
        faces.Add(new Face(_points[5], _points[11], _points[4]));
        faces.Add(new Face(_points[11], _points[10], _points[2]));
        faces.Add(new Face(_points[10], _points[7], _points[6]));
        faces.Add(new Face(_points[7], _points[1], _points[8]));

        // 5 faces around point 3
        faces.Add(new Face(_points[3], _points[9], _points[4]));
        faces.Add(new Face(_points[3], _points[4], _points[2]));
        faces.Add(new Face(_points[3], _points[2], _points[6]));
        faces.Add(new Face(_points[3], _points[6], _points[8]));
        faces.Add(new Face(_points[3], _points[8], _points[9]));

        // 5 adjacent faces 
        faces.Add(new Face(_points[4], _points[9], _points[5]));
        faces.Add(new Face(_points[2], _points[4], _points[11]));
        faces.Add(new Face(_points[6], _points[2], _points[10]));
        faces.Add(new Face(_points[8], _points[6], _points[7]));
        faces.Add(new Face(_points[9], _points[8], _points[1]));



        // refine triangles
        for (int i = 0; i < recursionLevel; i++)
        {
            var faces2 = new List<Face>();
            foreach (var tri in faces)
            {
                // replace triangle by 4 triangles
                int a = GetMiddlePoint(tri.V1, tri.V2);
                int b = GetMiddlePoint(tri.V2, tri.V3);
                int c = GetMiddlePoint(tri.V3, tri.V1);

                faces2.Add(new Face(tri.V1, _points[a], _points[c]));
                faces2.Add(new Face(tri.V2, _points[b], _points[a]));
                faces2.Add(new Face(tri.V3, _points[c], _points[b]));
                faces2.Add(new Face(_points[a], _points[b], _points[c]));
            }
            faces = faces2;
        }


        // done, now add triangles to mesh
        var vertices = new List<TexturedVertex>();

        foreach (var tri in faces)
        {
            var uv1 = GetSphereCoord(tri.V1);
            var uv2 = GetSphereCoord(tri.V2);
            var uv3 = GetSphereCoord(tri.V3);
            vertices.Add(new TexturedVertex(new Vector4(tri.V1, 1), uv1));
            vertices.Add(new TexturedVertex(new Vector4(tri.V2, 1), uv2));
            vertices.Add(new TexturedVertex(new Vector4(tri.V3, 1), uv3));
        }

        return vertices.ToArray();
    }

    private int AddVertex(Vector3 p)
    {
        _points.Add(p.Normalized());
        return _index++;
    }

    // return index of point in the middle of p1 and p2
    private int GetMiddlePoint(Vector3 point1, Vector3 point2)
    {
        long i1 = _points.IndexOf(point1);
        long i2 = _points.IndexOf(point2);
        // first check if we have it already
        var firstIsSmaller = i1 < i2;
        long smallerIndex = firstIsSmaller ? i1 : i2;
        long greaterIndex = firstIsSmaller ? i2 : i1;
        long key = (smallerIndex << 32) + greaterIndex;

        int ret;
        if (_middlePointIndexCache.TryGetValue(key, out ret))
        {
            return ret;
        }

        // not in cache, calculate it

        var middle = new Vector3(
            (point1.X + point2.X) / 2.0f,
            (point1.Y + point2.Y) / 2.0f,
            (point1.Z + point2.Z) / 2.0f);

        // add vertex makes sure point is on unit sphere
        int i = AddVertex(middle);

        // store it, return index
        _middlePointIndexCache.Add(key, i);
        return i;
    }

}

Get sphere coordinate is my own addition for calculating the texture coordinate for each vertex.
public static Vector2 GetSphereCoord(Vector3 i)
{
 var len = i.Length;
 Vector2 uv;
 uv.Y = (float)(Math.Acos(i.Y / len) / Math.PI);
 uv.X = -(float)((Math.Atan2(i.Z, i.X) / Math.PI + 1.0f) * 0.5f);
 return uv;
}

At this point it looks like the following:
icosphere showing the texturing glitch that goes from pole to pole
Icosphere that shows the texturing glitch from pole to pole
And then the solution for the pole to pole stripe as described on sols site. Had to do the correction twice to the expected result.
private static void FixColorStrip(ref Vector2 uv1, ref Vector2 uv2, ref Vector2 uv3)
{
    if ((uv1.X - uv2.X) >= 0.8f)
        uv1.X -= 1;
    if ((uv2.X - uv3.X) >= 0.8f)
        uv2.X -= 1;
    if ((uv3.X - uv1.X) >= 0.8f)
        uv3.X -= 1;

    if ((uv1.X - uv2.X) >= 0.8f)
        uv1.X -= 1;
    if ((uv2.X - uv3.X) >= 0.8f)
        uv2.X -= 1;
    if ((uv3.X - uv1.X) >= 0.8f)
        uv3.X -= 1;
}

And call it from here in the Create function
foreach (var tri in faces)
{
    var uv1 = GetSphereCoord(tri.V1);
    var uv2 = GetSphereCoord(tri.V2);
    var uv3 = GetSphereCoord(tri.V3);
    FixColorStrip(ref uv1, ref uv2, ref uv3);
    vertices.Add(new TexturedVertex(new Vector4(tri.V1, 1), uv1));
    vertices.Add(new TexturedVertex(new Vector4(tri.V2, 1), uv2));
    vertices.Add(new TexturedVertex(new Vector4(tri.V3, 1), uv3));
}

Now the texture should look OK like this:
Icosphere that shows the texturing glitch fixed from pole to pole
Icosphere that shows the texturing glitch fixed
Lastly, change the generation of the initial models in GameWindow class to use these spheres instead of the cubes.
models.Add("Wooden", new MipMapGeneratedRenderObject(new IcoSphereFactory().Create(3), _texturedProgram.Id, @"Components\Textures\wooden.png", 8));
models.Add("Golden", new MipMapGeneratedRenderObject(new IcoSphereFactory().Create(3), _texturedProgram.Id, @"Components\Textures\golden.bmp", 8));
models.Add("Asteroid", new MipMapGeneratedRenderObject(new IcoSphereFactory().Create(3), _texturedProgram.Id, @"Components\Textures\moonmap1k.jpg", 8));
models.Add("Spacecraft", new MipMapGeneratedRenderObject(RenderObjectFactory.CreateTexturedCube6(1, 1, 1), _texturedProgram.Id, @"Components\Textures\spacecraft.png", 8));
models.Add("Gameover", new MipMapGeneratedRenderObject(RenderObjectFactory.CreateTexturedCube6(1, 1, 1), _texturedProgram.Id, @"Components\Textures\gameover.png", 8));
models.Add("Bullet", new MipMapGeneratedRenderObject(new IcoSphereFactory().Create(3), _texturedProgram.Id, @"Components\Textures\dotted.png", 8));

End result should be as in the following video:

Known issues that still need to be fixed:

Texturing at the poles is still glitchy. I was unable to get the solution described at sol.gfxil.net for it to work.. Yet.
Maybe implement the vertex index solution as soon as I figure it out. As this currently fits my needs that might take a while.

For the complete source code for the tutorial at the end of this part, go to: https://github.com/eowind/dreamstatecoding

So there, thank you for reading. Hope this helps someone out there : )

Until next time: Work to Live, Don’t Live to Work

Tuesday, February 14, 2017

OpenGL 4 with OpenTK in C# Part 12: Basic Movable Camera


In this post we will create some basic cameras in OpenGL with the help of OpenTK.

This is part 12 of my series on OpenGL4 with OpenTK.
For other posts in this series:
OpenGL 4 with OpenTK in C# Part 1: Initialize the GameWindow
OpenGL 4 with OpenTK in C# Part 2: Compiling shaders and linking them
OpenGL 4 with OpenTK in C# Part 3: Passing data to shaders
OpenGL 4 with OpenTK in C# Part 4: Refactoring and adding error handling
OpenGL 4 with OpenTK in C# Part 5: Buffers and Triangle
OpenGL 4 with OpenTK in C# Part 6: Rotations and Movement of objects
OpenGL 4 with OpenTK in C# Part 7: Vectors and Matrices
OpenGL 4 with OpenTK in C# Part 8: Drawing multiple objects
OpenGL 4 with OpenTK in C# Part 9: Texturing
OpenGL 4 with OpenTK in C# Part 10: Asteroid Invaders
OpenGL 4 with OpenTK in C# Part 11: Mipmap
OpenGL 4 with OpenTK in C# Part 12: Basic Moveable Camera
OpenGL 4 with OpenTK in C# Part 13: IcoSphere
OpenGL 4 with OpenTK in C# Part 14: Basic Text

As stated in the previous post, I am in no way an expert in OpenGL. I write these posts as a way to learn and if someone else finds these posts useful then all the better :)
If you think that the progress is slow, then know that I am a slow learner :P
This part will build upon the game window and shaders from part 11, including homing bullets from this post.

Camera

At this point, it is quite easy to implement a movable camera, it is actually just another level of matrix multiplication that we do when we setup the ModelView matrix for each object. So lets start there by modifying the Render method of the AGameObject class to take a camera as input.
public virtual void Render(ICamera camera)
{
    _model.Bind();
    var t2 = Matrix4.CreateTranslation(_position.X, _position.Y, _position.Z);
    var r1 = Matrix4.CreateRotationX(_rotation.X);
    var r2 = Matrix4.CreateRotationY(_rotation.Y);
    var r3 = Matrix4.CreateRotationZ(_rotation.Z);
    var s = Matrix4.CreateScale(_scale);
    _modelView = r1*r2*r3*s*t2*camera.LookAtMatrix;
    GL.UniformMatrix4(21, false, ref _modelView);
    _model.Render();
}

In this case our cameras should be able to provide a LookAtMatrix and be able to update themselves. So the interface looks like this:
public interface ICamera
{
    Matrix4 LookAtMatrix{ get; }
    void Update(double time, double delta);
}
The update method is called from the OnFrameUpdate override in the GameWindow and the look at matrix is used for each object rendered in the OnRenderFrame override.

So, lets look at some cameras

Default: Static Camera


This camera is basically what we have out of the box and been using so far. It is located at origin (0, 0, 0) and is pointed towards the negative Z axis. I.e. into the screen (remember right handed coordinate system).
Lets create a camera that implements this camera so that we can change back to it whenever we want to.
public class StaticCamera : ICamera
{
    public Matrix4 LookAtMatrix { get; }
    public StaticCamera()
    {
        Vector3 position;
        position.X = 0;
        position.Y = 0;
        position.Z = 0;
        LookAtMatrix = Matrix4.LookAt(position, -Vector3.UnitZ, Vector3.UnitY);
    }
    public StaticCamera(Vector3 position, Vector3 target)
    {
        LookAtMatrix = Matrix4.LookAt(position, target, Vector3.UnitY);
    }
    public void Update(double time, double delta)
    {}
}
Also added a constructor that makes this camera a little bit more useful, it can initialize to any position and look at any static target. Note that we are using the OpenTK Matrix4 method LookAt to create out camera look at matrix.

First Person Camera

Next camera is the First Person Camera. We send in a AGameObject that the camera should follow and it should give us a feed following the path of the object.
public class FirstPersonCamera : ICamera
{
    public Matrix4 LookAtMatrix { get; private set; }
    private readonly AGameObject _target;
    private readonly Vector3 _offset;

    public FirstPersonCamera(AGameObject target)
        : this(target, Vector3.Zero)
    {}
    public FirstPersonCamera(AGameObject target, Vector3 offset)
    {
        _target = target;
        _offset = offset;
    }

    public void Update(double time, double delta)
    {
        LookAtMatrix = Matrix4.LookAt(
            new Vector3(_target.Position) + _offset,  
            new Vector3(_target.Position + _target.Direction) + _offset, 
            Vector3.UnitY);
    }
}

Here as well we have an overloaded constructor that takes an offset. Still looking in the direction that the object is moving, but from an offset to the position variable, maybe from the cockpit of an airplane instead of the origin of the model.

Third Person Camera

Our third person camera looks at the object that we are tracking from an offset baside it.
public class ThirdPersonCamera : ICamera
{
    public Matrix4 LookAtMatrix { get; private set; }
    private readonly AGameObject _target;
    private readonly Vector3 _offset;

    public ThirdPersonCamera(AGameObject target)
        : this(target, Vector3.Zero)
    {}
    public ThirdPersonCamera(AGameObject target, Vector3 offset)
    {
        _target = target;
        _offset = offset;
    }

    public void Update(double time, double delta)
    {
        LookAtMatrix = Matrix4.LookAt(
            new Vector3(_target.Position) + (_offset * new Vector3(_target.Direction)),  
            new Vector3(_target.Position), 
            Vector3.UnitY);
    }
}

Demo in the video of the three basic movable cameras:


For the complete source code for the tutorial at the end of this part, go to: https://github.com/eowind/dreamstatecoding

So there, thank you for reading. Hope this helps someone out there : )

Until next time: Work to Live, Don’t Live to Work