Tuesday, June 5, 2018

Functional Adventures in F# - Type-safe identifiers


In this post we will look at type-safe identifiers for things.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers

So, working on my game I just used to define identifiers like for example ShipId like the following:
type ShipId = Guid
type Ship = 
 {
  Id : ShipId
  Name : string
 }
let ship = { Id : Guid.NewGuid(); Name : "Nebuchadnezzar" }
Turns out this was not the best way to do things. From C# code you can just send in any Guid, nothing is typed. And you can assign a 'ShipId' to a 'UserId' in F# as well, as it is just a type alias. I found out that a hard way while tracking down a bug that turned out to be a copy paste issue.

So, now that I have a bunch of stuff already written, I had to change this to a single item discriminated union. Took a while but it seems to work much better with the type-safety.. The way I thought that I thought it already did.

So, lets redefine the ShipId as a single item discriminated union like
type ShipId = ShipId of Guid
The code to use it from code is like:
let ship = { Id : ShipId (Guid.NewGuid()); Name : "Nebuchadnezzar" }


Usage from C#

My game is partially written in C# as it just works better for some tasks. So interfacing with the new typed ids will be like:

Getting the underlying Guid
var shipIdGuid = ship.Id.Item,
Creating a new ShipId from a Guid
var shipId = ShipId.NewShipId(Guid.NewGuid());

So a little bit more hassle from C#, but not that much, and most of my code is just service layer code creating F# records from external input. And the little extra overhead is worth it for the type-safety!

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work


Friday, May 18, 2018

Functional Adventures in F# - Adding Snapshot Persisting



In this post we will look at how to add snapshots to our event source engine from previous posts.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers


Quickest way to get started for me was to just persist a snapshot every X actions that were processed. Also for my needs, it is OK to persist the snapshot on a separate thread, as long as the actions are persisted by the AppHolder MailboxProcessor, then the snapshot can be written by another MailboxProcessor. This way, as the state grows it will not affect the main processing...

Changes needed to the AppPersister

We will be adding a new function called Persist that will be called from the AppHolder instead of the current PersistAction.
    let Persist (nextAction:obj) (state:obj) (actionCounter:int)  =
        PersistAction nextAction nextAction?ActionId
        PersistSnapshot actionCounter state nextAction?ActionId
        ()
This in turn will call 2 functions, the old PersistAction from previous part and the new PersistSnapshot.
Also, note the nextAction?ActionId, this is part of the FSharp.Interop.Dynamic NuGet package that adds the support for Dynamics, i.e. runtime checked fields and functions on objects instead of compile time checked types. We will be assuming that all actions (that are of type object, or obj in F#) implement the field ActionId... We could solve this by forcing an interface implementation but that just gives us ugly code here, but I guess there is a better solution but for me this solved the problem and I am only to blame myself if things break.
Use them with care.

    let private Snapshotter = MailboxProcessor.Start(fun inbox ->
        let rec loop (n:int) =
            async {
                let! (actionCounter:int,state:obj, nextActionId:Guid) = inbox.Receive()
                if actionCounter % 300 = 0 then
                    writeSnapshotToDisk state nextActionId
                return! loop n
                }
        loop 0)

    let PersistSnapshot (actionCounter:int) (state:obj) (nextActionId:Guid) =
        Snapshotter.Post (actionCounter, state, nextActionId)

The PersistSnapshot function just posts the state to the MailboxProcessor called Snapshotter that in turn calls writeSnahotToDisk every 300 action. Much hardcoded here, I guess you will have to experiment a little to find the magic number that works for you (300 is just an example and I will probably end changing it in my code as well)
    let private createSnapshotFilename (nextActionId:Guid) =
        let now = DateTime.UtcNow
        let dayPath = now.ToString("yyyy-MM-dd")
        let fullPath = Path.Combine(store, "snapshots", dayPath)
        let di = new DirectoryInfo(fullPath)
        di.Create()
        let filename = now.ToString("yyyy-MM-dd hh_mm_ss_fffffff.") + nextActionId.ToString() + ".snapshot"
        let fullFilename = Path.Combine(fullPath, filename)
        fullFilename
    
    let private writeSnapshotToDisk  (state:obj) (nextActionId:Guid) =
        let fullFilename = createSnapshotFilename nextActionId
        let json = JsonConvert.SerializeObject(state)
        File.WriteAllText(fullFilename,  json)
        ()

So basically as in the PersistAction part, we add some meta-data to the filename that we can use when reading from disk later. We JSON serialize the state object just as it is. The meta-data that we are interested in is the ID of the next action to be executed.


We will also be changing the createActionFilename function to add the ID of the action to the filename as follows:
    let private createActionFilename (action:obj) (actionId:Guid) =
        let now = DateTime.UtcNow
        let hourPath = now.ToString("yyyy-MM-dd HH")
        let fullPath = Path.Combine(store, hourPath)
        let di = new DirectoryInfo(fullPath)
        di.Create()
        let t = action.GetType()
        let filename = now.ToString("yyyy-MM-dd hh_mm_ss_fffffff+") + now.Ticks.ToString() + "." + t.Name + "." + actionId.ToString() +  ".action"
        let fullFilename = Path.Combine(fullPath, filename)
        fullFilename


Next we will be adding functionality to load snapshots and actions from disk. I.e. latest snapshot and all actions persisted after that snapshot. A totality that will result in a total state.

First, a minor change to getAction
    let private getAction (json:string, filename:string) =
        let split = filename.Split('.')
        let actionName = split.[1]
        let actionNameWithNamespace = "dreamstatecoding.core.Actions+" + actionName
        let t = Assembly.GetExecutingAssembly().GetType(actionNameWithNamespace)
        (JsonConvert.DeserializeObject(json, t), Guid.Parse(split.[2]))
We will be returning a tuple, with the deserialized object as the first element and the ID as the second element

Next, we want to get all actions from a specific ID, namely the ID that we wrote into the filename of the snapshot. To do this we do the following:
    let private fileContainAction (filename:string) (actionIdString:string) =
        let split = filename.Split('.')
        split.[2] = actionIdString

    let GetAllActionsFromId (nextActionId:Guid) =
        let di = new DirectoryInfo(store)
        di.Create()
        let nextActionIdString = nextActionId.ToString()
        let actions =
            di.GetFiles("*.action", SearchOption.AllDirectories)
            |> Seq.skipWhile (fun (fi:FileInfo) -> not (fileContainAction fi.Name nextActionIdString))
            |> Seq.map (fun (fi:FileInfo) -> File.ReadAllText(fi.FullName), fi.Name)
            |> Seq.map (fun x -> (getAction x))
            |> Seq.toArray
        actions
We use the Seq.skipWhile that skips elements in the input sequence until the function returns false, after that it will return the rest of the sequence. In our case, we check if the filename contains the next action id with the help of the fileContainsAction function and negating the result by using the not operator.
Handy when the framework contians these nice functions that do exactly what you need at the moment!
    let private getSnapshot (fi:FileInfo) =
        let split = fi.Name.Split('.')
        let json = File.ReadAllText(fi.FullName)
        (JsonConvert.DeserializeObject<ApplicationState.AppliationState>(json), Guid.Parse(split.[1]))

The getSnaphot is pretty much the same as the GetAction, we parse out the ID of the next action to be executed from the filename and return a tuple of the deserialized state and the ID.

And lastly the new function to be called from outside to get the latest state from disk storage:
    let GetLatestSnapshotAndActions () =
        let di = new DirectoryInfo(Path.Combine(store, "snapshots"))
        di.Create()
        let fileInfos = di.GetFiles("*.snapshot", SearchOption.AllDirectories)
        match fileInfos with
        | a when a.Length = 0 -> (None, GetAllActions())
        | _ ->
            let (snapshot, nextActionId) =
                fileInfos
                |> Seq.last
                |> getSnapshot
            let actions = GetAllActionsFromId nextActionId
            (Some snapshot, actions)
Key things to note here:

  • We create the snapshot directory if it does not exist, this so that we don't get 'directory does not exist' exceptions from the GetFiles method.
  • If there are no files returned by the GetFiles method, we return a tuple with None as the first element and the original GetAllActions as the actions list to be executed.
  • otherwie we get the last snapshot (Seq.last) and return Some snapshot, and reminder of actions as a tuple
The None and Some way of doing things seems nicer then using null as we do in other languages, especially from the caller side as we will see in our rewritten AppHolder module:

    type Message =
        | Snapshot of AppliationState
        | Replay of obj
        | Action of obj
First we introduce a new message type to our MailboxProcessor, the Snapshot of ApplicationState.
let s' =
 match message with
 | Snapshot snapshot -> snapshot
 | Replay a -> s.HandleAction a
 | Action a -> 
  AppPersister.Persist a s c'
  s.HandleAction a
If it is a snapshot, then the s' state is the snapshot, i.e. no other processing is done
Then, lastly we add a new init function to be called from our application startup code (Program.cs or whatever you are using)

    let InitiateFromLastSnapshot () =
        let (snapshot, actions) = AppPersister.GetLatestSnapshotAndActions()
        match snapshot with
        | Some x -> Processor.Post (Snapshot x)
        | _ -> ()
        Array.map (fun x -> (HandleReplayAction x)) actions
Here we get the snapshot and actions from the AppPersister, then we use pattern matching on the snapshot to determine if it is Some, in which case we send it into the Processor as a Snapshot message. Otherwise we do nothing ()
And then just run in all the Replay actions as before.



That is it!
All code is available at github.

Known issues

Newtonsoft JSON.NET typeconverter issue

Newtonsoft JSON.NET seems to have some trouble with F# Map collections, especially when using union types as keys
type UserId = UserId of Guid.
Map<UserId, User>
Gives following nasty exception when trying to deserialize it as a key in a Map
Newtonsoft.Json.JsonSerializationException
  HResult=0x80131500
  Message=Could not convert string 'UserId 8fa6d5a6-9500-4aac-9f3e-a5b918b81c46' to dictionary key type 'dreamstatecoding.core.Model+UserId'. Create a TypeConverter to convert from the string to the key type object. Path 'UserStore.Users['UserId 8fa6d5a6-9500-4aac-9f3e-a5b918b81c46']', line 1, position 69.
  Source=Newtonsoft.Json
  StackTrace:
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateDictionary(IDictionary dictionary, JsonReader reader, JsonDictionaryContract contract, JsonProperty containerProperty, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.ResolvePropertyAndCreatorValues(JsonObjectContract contract, JsonProperty containerProperty, JsonReader reader, Type objectType)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObjectUsingCreatorWithParameters(JsonReader reader, JsonObjectContract contract, JsonProperty containerProperty, ObjectConstructor`1 creator, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonConvert.DeserializeObject(String value, Type type, JsonSerializerSettings settings)
   at Newtonsoft.Json.JsonConvert.DeserializeObject[T](String value, JsonSerializerSettings settings)
   at dreamstatecoding.core.AppPersister.getSnapshot(FileInfo fi) in C:\tfs\apps\dreamstatecoding.core\AppPersister.fs:line 104
   at dreamstatecoding.core.AppPersister.GetLatestSnapshotAndActions() in C:\tfs\apps\dreamstatecoding.core\AppPersister.fs:line 113
   at dreamstatecoding.core.AppHolder.InitiateFromLastSnapshot() in C:\tfs\apps\dreamstatecoding.core\AppHolder.fs:line 47
   at test.web.Program.Main(String[] args) in C:\tfs\apps\test\test.web\Program.cs:line 14

Inner Exception 1:
JsonSerializationException: Error converting value "UserId 8fa6d5a6-9500-4aac-9f3e-a5b918b81c46" to type 'dreamstatecoding.core.Model+UserId'. Path 'UserStore.Users['UserId 8fa6d5a6-9500-4aac-9f3e-a5b918b81c46']', line 1, position 69.

Inner Exception 2:
ArgumentException: Could not cast or convert from System.String to dreamstatecoding.core.Model+UserId.
This is fixed and a new version is on github.. Thanks to the really nice #fsharp community on twitter for the help!
Code changes to include the Fable.JsonConverter
let converters =
 [ Fable.JsonConverter () :> JsonConverter ] |> List.toArray :> IList<JsonConverter>

let settings =
 JsonSerializerSettings (
  Converters = converters,
  Formatting = Formatting.Indented,
  NullValueHandling = NullValueHandling.Ignore)
And then just using it in serializing and deserializing as follows:
let json = JsonConvert.SerializeObject(state, settings)
let newState = JsonConvert.DeserializeObject<ApplicationState.ApplicationState>(json, settings)

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work

Wednesday, May 9, 2018

Functional Adventures in F# - Persisting Application State



In this part we will look at how to persist the application state to disk. It should not be too hard to modify for usage with other persistence solutions.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers

Storing all actions

The idea is pretty simple, we will store all actions sent to the AppHolder MailboxProcessor and whenever we want to re-initiate the application state, we just load all the actions from storage and run them through the processor again and we should end up with the same application state.

This has been described by other people, for example Martin Fowler, if you have the time then head over there and read his article about event sourcing.

For this we will create a module called AppPersister. It will handle the actual storing and retrieving of the events from disk. So it will have side-effects and thus not be purely functional.... So lets just throw all those concepts out the door. But, this is a great opportunity to show some nice features of F# that will help write this kind of code a little more ... functional?
For serialization I chose to use the json format, and serializer is the JSON.NET library by Newtonsoft that can be added to your project with NuGet.

Some code for starters:
module AppPersister =
    open System
    open System.Reflection
    open System.IO
    open Newtonsoft.Json
    
    let private store = @".\store\"
Basically we just create a new module that will handle the persisting of actions... Here we also define that the store should be a subdirectory for the application.
    let private createActionFilename (action:obj) =
        let now = DateTime.UtcNow
        let hourPath = now.ToString("yyyy-MM-dd HH")
        let fullPath = Path.Combine(store, hourPath)
        let di = new DirectoryInfo(fullPath)
        di.Create()
        let t = action.GetType()
        let filename = now.ToString("yyyy-MM-dd hh_mm_ss_fffffff+") + now.Ticks.ToString() + "." + t.Name + ".action"
        let fullFilename = Path.Combine(fullPath, filename)
        fullFilename
The createActionFilename function does just that, it generates a new unique filename for the action by combining todays date and time, throwing in the total number of ticks to ensure that we always have a fresh value and lastly adding the type of action and the extension '.action'. Here we also create the directory if it does not exist already for the current date and hour (UTC), the DirectoryInfo.Create method is safe to run on an already existing directory so we do no other checking ourselves.
    let PersistAction (action:obj) =
        let fullFilename = createActionFilename action
        let json = JsonConvert.SerializeObject(action)
        File.WriteAllText(fullFilename,  json)
        ()
The PersisAction function handles the actual writing of the action to disk. We call createActionFilename to get the full filename and then use JsonConvert to serialize the action to a json string. Lastly we write the file with the nice File.WriteAllText method.

Now that we can write the actions, lets write some code to read them from disk.
    let GetAllActions () =
        let di = new DirectoryInfo(store)
        let actions =
            di.GetFiles("*.action", SearchOption.AllDirectories)
            |> Seq.map (fun (fi:FileInfo) -> File.ReadAllText(fi.FullName), fi.Name)
            |> Seq.map (fun x -> (getAction x))
            |> Seq.toArray
        actions

  • Here we use the DirectoryInfo.GetFiles built in method to find all files with the .action extension in any subdirectory in the store path. The result is an array of FileInfo objects that we pipe to a 
  • Seq.map where we return a Tuple containing the file contents and filename and pipe that tuple into
  • another Seq.map where we call getAction for all elements. This function will be tasked with deserializing the json to the correct type
  • Lastly we pipe the contents to Seq.toArray to make the result concrete. To my understanding is that the Seq constructs work a little like the IEnumerable and do lazy evaluation if you do not actually list it. 
    let private getAction (json:string, filename:string) =
        let split = filename.Split('.')
        let actionName = split.[1]
        let actionNameWithNamespace = "dreamstatecoding.core.Actions+" + actionName
        let t = Assembly.GetExecutingAssembly().GetType(actionNameWithNamespace)
        JsonConvert.DeserializeObject(json, t)
Lastly, the getAction function that is called from GetAllActions. Here we split the filename and pick out the part containing the name of the Action and then as the currently executing assembly to find the Type for that. Notice that we need to prefix the action name with the namespace name to make it work. In my solution I have all my actions in 1 module, so this works.


Rewriting AppHolder to work with persisted actions

Next step is to rewrite the AppHolder module from previous part to work with the AppPersister module.

So what we want to do here is to separate actual real time Actions from Replay actions from the persisting, mainly so that they do not get persisted again (duplicated).

So, lets put up a new discriminated union that defines what we want to do
    type Message =
        | Replay of obj
        | Action of obj

So, either we want to execute a Replay object or execute an Action object.
    let private Processor = Agent.Start(fun inbox ->
        let rec loop (s : AppliationState) =
            async {
                let! message = inbox.Receive()
                let (action:obj) = 
                    match message with
                    | Action a -> 
                        AppPersister.PersistAction a
                        a
                    | Replay a -> a

                let s' = s.HandleAction action
                state <- s'
                counter <- counter + 1
                return! loop s'
                }
        loop AppliationState.Default)
Now we rewrite our MailboxProcessor (Agent) to take a message instead of object directly. The first step is to pattern-match the received message to the type and get out the payload object.
If we are handling an Action we want to persist it as well, so lets put that function call here as well. Otherwise no change here.
    let HandleAction (action:obj) =
        Processor.Post (Action action)
The old HandleAction code called Processor.Post with the action directly. Now we must state that it is an Action, and not a Replay.
    let private HandleReplayAction (action:obj) =
        Processor.Post (Replay action)
The same goes for the Replay execution, we just add a new function that executes actions as Replays
    let InitiateFromStore () =
        AppPersister.GetAllActions()
        |> Array.map (fun x -> (HandleReplayAction x))
        |> ignore
        ()

Lastly, a function to initialize the store, we pipe all the actions loaded from persistence to the HandleReplayAction and then pipe the results from that to ignore. This means that we are not really interested in the results from this function call, in the end we just return unit from the InitiateFromStore function.
I put the call to this into my Program.cs file, it will load all the actions already executed from the store and execute them all as Replay actions.

Conclusions

In this part we have looked at how to persist the application state as a series of actions and then re-creating the application state when the application is restarted.
For applications handling lots and lots of actions, we may want to limit the number of actions needed to recreate the up to date state.. So in the next part we will look at snapshots. My plan is to upload the code to git after the snapshot part has been added.

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work

Saturday, April 28, 2018

Functional Adventures in F# - The MailboxProcessor


In this post we will look at MailboxProcessor in F#.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers

MailboxProcessor

This continues with the code from Application State post.
Mainly the GameState type
and GameState =
 {
  PlayerStore:PlayerStore
  UnitStore:UnitStore
 }
 member this.HandleAction (action:obj) (state:GameState) =
  {
   PlayerStore = state.PlayerStore.HandleAction action state.UserStore
   UnitStore = state.UnitStore.HandleAction action state.AmmunitionStore 
  }


My goal is to incorporate F# with C# in a nice way. And as described in the Application State post, the core application model and all actions that can be made in the application will be modeled in F#. But to prevent a multithreaded C# application to corrupt the state we need to serialize the commands that are processed.
In C# I would have used a background Thread and a Queue<T> and written a locks to synchronize the reads and writes. Luckily in F#, we have all that nicely packaged in the built in MailboxProcessor.

The first line redefines the MailboxProcessor to the term Agent instead, this seems to be something that a lot of people are doing, so lets stick to that for the time being.
After that we define a F# static class, i.e. an abstract and sealed class with only static members. (this is what C# does under the hood).

type Agent<'T> = MailboxProcessor<'T>     
[<AbstractClass; Sealed>]
type GameHolder private () =
 static let mutable state : GameState = 
  {
   PlayerStore = { Players = Map.empty }
   UnitStore = { Units = Map.empty }
  }
 static member State : GameState = state
 static member private Processor = Agent.Start(fun inbox ->
  let rec loop =
   async {
    let! action = inbox.Receive()
    state <- state.HandleAction action state
    return! loop
    }
  loop)
 static member HandleAction (action:obj) =
  GameHolder.Processor.Post action

In addition we will be allowing for mutation here, something that the business logic in the F# module should be unaware of, but here, we are exposing the current state to the rest of the application so that it is easily accessible.
I.e. C# we can call GameHolder.State and always get the latest state. It is exposed as immutable so there is no risk that other threads will write to it.

After that we define the MailboxProcessor<'T> or in this case the redefined Agent<'T> as a private static member of the class.
What happens in the recursive part is a little iffy for me still. To my understanding the let! (with the exclamation mark) makes the inbox.Receive (that is also an async call) an synchronous call and stores its result in the action value.
If anyone has detailed explanation of what is going on here, please feel free to comment as I was unable to find details of this seems just to be the way things are done.

The HandleAction method calls the Post method on the MailboxProcessor and thus posts the action to the MailboxProcessor that in turn calls the HandleAction method of the State and mutates the exposed State value.

Update 2018-05-02, forcing serial access

Turns out the MailboxProcessor above does not work the way I thought... In my mind it serialized the processing but it did not. I.e. when multiple actions were sent to it in parallel, it corrupted the state variable.
After looking at this for some time now, I finally had to post the question on stackoverflow.
https://stackoverflow.com/questions/50135825/f-mailboxprocessor-limit-parallelism

Turns out the bad wolf in the drama was the static member. Changing the code to a module instead got the solution to work as expected.

The end result is:
module GameHolder =
    type Agent<'T> = MailboxProcessor<'T>     

    let mutable private state = AppliationState.Default
    let mutable private counter : int = 0
    let GetCurrentState() =  state
    let GetProcessedActionsCounter() = counter
    let private Processor = Agent.Start(fun inbox ->
        let rec loop (s : AppliationState) =
            async {
                let! action = inbox.Receive()
                let s' = s.HandleAction action
                state <- s'
                counter <- counter + 1
                return! loop s'
                }
        loop AppliationState.Default)

    let HandleAction (action:obj) =
        Processor.Post action

I can still get the current application state by calling a 'static' method from C#, in this case GameHolder.GetCurrentState().
So a small change thanks to stackoverflow and we still have a small but neat interface between our C# game/application and it model/business logic that is written in F#.

Finally

In the end. We have a GameHolder class that exposes State and HandleAction to the outside world. Whoever can query the state at anytime, but they are not allowed to change it other then through the HandleAction method.

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work

Thursday, March 22, 2018

Functional Adventures in F# - Getting rid of temp variables


This time we will look at how to get rid of temporary variables when updating an object in multiple steps.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers

In my game that I am trying to write with F# (at least the core model) I have some functions that look like the following
static member HandleTick (state:UnitStore) (action:Tick) =     
 let queueUpdatedState = UnitStore.handleTickBuildQueues state action
 let unitsUpdatedState = UnitStore.handleTickUnitUpdates queueUpdatedState action
 unitsUpdatedState
A function that takes a state and an action and in turn it uses other functions to update that state with the help of the action.
I don't know, but I grew tired of naming temporary variables after a while and started googling for a better solution. Turns out there is one. By using the forward-pipe operator '|>' we can take the output of one function and send it as input to the next function.

static member HandleTick (state:UnitStore) (action:Tick) =     
 UnitStore.handleTickBuildQueues action state 
 |> UnitStore.handleTickUnitUpdates action
To make this work, the order of parameters need to be switched so that Action is the first argument followed by the State.
Turns out I have to refactor all the functions in my Stores to take the action first and state as second argument to get this working, but I guess that is a small price to pay for cleaner code.

Looking at the assembly in ILSpy gives us that in the background there will be temporary variables passed around, but our code is clean of them.
public static UnitStore HandleTick(Actions.Tick action, UnitStore state)
{
 UnitStore unitStore = UnitStore.handleTickBuildQueues(action, state);
 UnitStore state2 = unitStore;
 return UnitStore.handleTickUnitUpdates(action, state2);
}

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work

Wednesday, March 21, 2018

Functional Adventures in F# - Getting rid of loops


Solving problems without loops.....

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers

This time lets look at a real issue when coming from an imperative language and trying functional programming.
The example problem (that I crashed into when rewriting some game logic in F#) is that of spawning new Ships in a space game. We do not want to spawn 2 ships on top of each other, so we generate the default spawning coordinate and then test if it is safe to use, if not we move the ship a bit and try again....

How I solved it in C#..
private Ship FindClosestFreeCoordinate(ImmutableList<Fleet> state, Ship newShip)
{
    var shipsWithinSafetyDistance = ShipsWithinSafetyDistance(state, newShip);
    while (shipsWithinSafetyDistance.Any())
    {
        var newCoordinates = newShip.Coordinates.Position.Add(Vector3.UnitXVector);
        newShip = newShip.Modify(coordinates: newShip.Coordinates.Modify(position: newCoordinates));
        shipsWithinSafetyDistance = ShipsWithinSafetyDistance(state, newShip);
    }
    return newShip;
}

private List<Ship> ShipsWithinSafetyDistance(ImmutableList<Fleet> state, Ship newShip)
{
    return (from fleet in state
        from
            ship in fleet.Ships
        where
            ship.Distance(newShip) < 5
        select ship).ToList();
}

Translating the ShipsWithinSafetyDistance function is no problem. We just use the Map.filter function...
static member private shipsWithinSafetyDistance (unit:Unit) (state:UnitStore) =
    Map.filter (fun key value -> unit.distanceToUnit value < 5.0) state.Units
Ah, yes.. I switched from a ImmutableList in C# to a Map in F#, just to keep things interesting (and to get faster Key lookups)


So, the tricky part here is the while loop. Most loops can be rewritten with Map/List functions like map, filter and fold etc... But this time I don't see how to use them as I don't have a Collection of any sort here so my brain keeps yelling "Write a loooooop", as that is the way I have solved this kind of problems for the past 20+ years... So...
How should we do it in F#??? We have some other constructs to use that fit the functional idea better.

static member private findClosestFreeCoordinate (unit:Unit) (state:UnitStore) =
 let rec findClosestFreeCoordinateRecursive (unit:Unit) (state:UnitStore) = 
  let shipsWithinSafetyDistance = shipsWithinSafetyDistance unit state
  match shipsWithinSafetyDistance with
  | a when a.IsEmpty -> unit
  | _ ->  let newUnit = { unit with Coordinates = unit.Coordinates + Vector3.UnitX }
    findClosestFreeCoordinateRecursive newUnit state
 findClosestFreeCoordinateRecursive unit state
Recursion!
At least for this problem, it seems to be a good option. Here we define a function, findClosestFreeCoordinateRecursive with the rec keyword to let the compiler know that it will be called recursively. In the function body we match the result list with IsEmpty and return the input value or else call the recursive function with a new copy of the unit with new coordinates.

Call stack.. Is there a risk for stack overflow with this?
Lets open the dll output with ILSpy to check what actually goes on behind the scenes
ILSpy of F# Recursive Function
Turns out, that the current F# compiler in the background uses a while loop. The key here is that we do not, instead we define a short and concise function where we state what it is that we want done, how it is solved behind the scene in detail is not up to us. At some point, there is always someone that needs to write the ifs and loops, but lets keep them away from the application code that we write.

So there, lets move the last shipsWithinSafetyDistance to inner scope of our function and we are set
static member private findClosestFreeCoordinate (unit:Unit) (state:UnitStore) =
 let shipsWithinSafetyDistance (unit:Unit) (state:UnitStore) =
  Map.filter (fun _ value -> unit.distanceToUnit value < 50.0) state.Units
 let rec findClosestFreeCoordinateRecursive (unit:Unit) (state:UnitStore) = 
  let shipsWithinSafetyDistance = shipsWithinSafetyDistance unit state
  match shipsWithinSafetyDistance with
  | a when a.IsEmpty -> unit
  | _ ->  let newUnit = { unit with Coordinates = unit.Coordinates + Vector3.UnitX }
    findClosestFreeCoordinateRecursive newUnit state
 findClosestFreeCoordinateRecursive unit state

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work


Friday, March 16, 2018

Functional Adventures in F# - Types with member functions

Lets look at how to structure functions into the types that they are linked to, this helps with keeping the code base clean.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers


I started converting my small but handy 3D math library to F# and very quickly I ran into a problem. Not that unexpected as I am a beginner with F#.
So say that you have some types:
type Vector3 = 
 { 
  X:float
  Y:float
  Z:float 
 }
type Vector4 = 
 { 
  X:float
  Y:float
  Z:float 
  W:float 
 }
That in the end will have quite similar functionality like .. for example Add, Divide etc. With the knowledge that I have so far of F# I ended up with
let addVector3 (value1:Vector3) (value2:Vector3) = 
 {
  X = value1.X + value2.X
  Y = value1.Y + value2.Y
  Z = value1.Z + value2.Z
 }
let addVector4 (value1:Vector4) (value2:Vector4) = 
 {
  X = value1.X + value2.X
  Y = value1.Y + value2.Y
  Z = value1.Z + value2.Z
  W = value1.W + value2.W
 }
I quickly thought that there must be a cleaner way to write this. I like tidy code, neat code.. not code that just looks bad and will turn into a maintenance hell even before it is shipped.
Luckily it turns out you can define functions as members of types. So the above 2 functions would look like:
type Vector3 = 
 { 
  X:float
  Y:float
  Z:float 
 }
 member this.add (value2:Vector3) = 
 {
  X = this.X + value2.X
  Y = this.Y + value2.Y
  Z = this.Z + value2.Z
 }
type Vector4 = 
 { 
  X:float
  Y:float
  Z:float 
  W:float 
 }
 member this.add (value2:Vector4) = 
 {
  X = this.X + value2.X
  Y = this.Y + value2.Y
  Z = this.Z + value2.Z
  W = this.W + value2.W
 }
Here, both are called just add, and they can be invoked directly on the objects.
let value1 = { X = 1.0; Y = 1.0; Z = 1.0 }
let value2 = { X = 2.0; Y = 2.0; Z = 2.0 }
let added = value1.add value2
A little cumbersome syntax if you want to do more complex calculations. So lets look at operator overloading in F# to so that we can get nice arithmetic operators into play for our new types
type Vector3 = 
 { 
  X:float
  Y:float
  Z:float 
 }
 static member (+) (value1:Vector3, value2:Vector3) = 
  {
   X = value1.X + value2.X
   Y = value1.Y + value2.Y
   Z = value1.Z + value2.Z
  }
 static member maxValue = { X = Microsoft.FSharp.Core.float.MaxValue; Y = Microsoft.FSharp.Core.float.MaxValue; Z = Microsoft.FSharp.Core.float.MaxValue; } 

 member this.add (value2:Vector3) = this + value2
This lets us write something like this instead:
let added = value1 + value2
Also note the static member maxValue that is a static function that is invoked by Vector3.maxValue, in this case it just returns the max vector but it can be any function.

So there.

Update: evidently static members should start with capital letter according to design guide-lines. Makes sense as they are easier to spot.

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work

Thursday, March 15, 2018

Functional Adventures in F# - Application State


In this article we will go through how to handle complex application state in F#. This will in practice be a rewrite of 'Functional Adventures in C# - Application State'. So I will probably skip a lot of the details and focus on the F# implementation.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers

Overview

Illustration of Application State, different Stores and their model objects. For a lot of illustrations go read the C# article!
Lets be real, nobody read the C# version first. So I'll paste the overview here.
At any given point in time, the whole application state is stored in a class named Application State, it consists of different Stores and they in turn consist of the object model for each store. I.e. Fleet Store might contain Fleets that consists of Ships. All updates to the model are routed to the store in the form of Actions. The application state is immutable, meaning that any object that you get a reference to will never mutate into something else (see previous post in this series on Immutable objects)
So. from the text above we need to define Actions, Application State, and Stores and we should be set to go.

Actions

Actions will be implemented as basic record types. There is no need for anything else fancy. An action should contain all information that is needed to create a new application state.
      newApplicationState = currentApplicationState + action
The code for this part looks like the following
module Actions =
    type InsertCashToPlayerAccount = { Player:GameModel.PlayerId; Amount:GameModel.Cash }
    type Tick = { Time:float; Delta:float }
We defined 2 actions. One that puts money into the players account and the other lets us update each store before rendering a new frame in a game.

Application State

Next up, we will look at the Application State definition, or in this example the GameState definition
module GameState =
    type GameState =
        {
            PlayerStore:PlayerStore
            UnitStore:UnitStore
        }
Here we just define that GameState is a record type that contains 2 stores, the PlayerStore and the UnitStore. Here we will also be adding a way to handle actions, so lets continue with the GameState definition
        
        static member handleAction (state:GameState) (action:obj) =
            {
                PlayerStore = PlayerStore.handleAction state.PlayerStore action
                UnitStore = UnitStore.handleAction state.UnitStore action
            }
Here we add a static member function to the handleAction type (that can be called as: 'GameState.handleAction state action'). We have defined the types here, the state is a GameState and the action is typed to obj, meaning that we can send in any type into this function and it will work. I.e. when we define new actions, we don't need to change anything here. Only if we add a new Store, then we need to add it here obviously.
Also, worth noting is that the action is in turn sent to all Store handlers together with the part of the state that is handled by that Store. Meaning that we can define an action, that is handled by multiple stores. For example the Tick action above.
So basically the handleAction function just builds a new Application State by calling each store and telling them to handle their parts.

Stores

OK, so whats so special about the stores? Nothing really, the Application State is practically a store in itself,
module PlayerStore =
    type PlayerStore =
        {
            Players: Map<PlayerId, Player>
            CurrentTime: float
        }
        static member handleInsertCashToPlayerAccount (state:PlayerStore) (action:InsertCashToPlayerAccount) =
            let player = state.Players.[action.Player]
            let newPlayer = { player with Cash = player.Cash + action.Amount }
            { state with Players = state.Players.Add(player.Id, newPlayer) }            
        static member handleTick (state:PlayerStore) (action:Tick) =
            { state with CurrentTime = action.Time }
        static member handleAction (state:PlayerStore) (action:obj) =
            match action with
            | :? InsertCashToPlayerAccount as a -> PlayerStore.handleInsertCashToPlayerAccount state a
            | :? Tick as a -> PlayerStore.handleTick state a
            | _ -> state

PlayerStore is a record type, just as the GameState type, with a static member handleAction function that has a little different implementation. The key here is the pattern matching on the action, we use the type of the action to determine what handler function should be called to perform whatever it is that the action needs done. As a last state, we have the wild card pattern '_' that returns the old state as it was sent in. I.e. if this actionHandler can't handle the action, nothing is done.
Also notable in the handleInsertCashToPlayerAccount function, the new state is built by taking the current state and applying the action. In this example, finding the player object and constructing a new player object with the added cash in it.

Lastly, lets look at the UnitStore so that we have all the code for our example
module UnitStore =
    type UnitStore =
        {
            Units: Map<GameObjectId, Unit>
            CurrentGameObjectId: GameObjectId
            CurrentTime: float
        }
        static member handleTick (state:UnitStore) (action:Tick) =
            { state with CurrentTime = action.Time }
        static member handleAction (state:UnitStore) (action:obj) =
            match action with
            | :? Tick as a -> UnitStore.handleTick state a
            | _ -> state
Same thing here, the handleAction function takes the UnitStore and the action and then matches an action handler function if one exists to create a new state, otherwise it will just hand back the original state.

Tests
[<TestClass>]
type StoreTests () =
    member this.createDefaultStore =
        { 
            PlayerStore = 
                { 
                    Players = Map.empty.Add(1, { Id = 1; Name = "Player1"; Cash = 0; CurrentSelection = [] })
                    CurrentTime = 0.0
                }
            UnitStore =
                {
                    Units = Map.empty
                    CurrentGameObjectId = 0
                    CurrentTime = 0.0 
                }
        }
For starters, we define a function that creates the initial GameState that will be used by all of our tests.
    [<TestMethod>]
    member this.StoreTests_GameState_UnknownType () =
        let originalState = this.createDefaultStore
        let action = 1
        let newState = GameState.handleAction originalState action
        Assert.AreEqual(originalState, newState)
First test, just check that if we send in an unknown action, we get back the original state object, nothing changed, the exact same object.
    [<TestMethod>]
    member this.StoreTests_GameState_UpdatePlayerStore () =
        let originalState = this.createDefaultStore
        let action = { Player = 1; Amount = 10 }
        let newState = GameState.handleAction originalState action
        Assert.AreNotEqual(originalState, newState)
        Assert.AreEqual(originalState.UnitStore, newState.UnitStore)
        Assert.AreEqual(10, newState.PlayerStore.Players.[1].Cash)
Secondly we test that an action that should update 1 store, does that, but leaves the other stores unchanged.
    [<TestMethod>]
    member this.StoreTests_GameState_UpdateMultipleStores () =
        let originalState = this.createDefaultStore
        let action = { Time = 22.0; Delta = 0.015 }
        let newState = GameState.handleAction originalState action
        Assert.AreNotEqual(originalState, newState)
        Assert.AreEqual(22.0, newState.PlayerStore.CurrentTime)
        Assert.AreEqual(22.0, newState.UnitStore.CurrentTime)
The last test just verifies that an action can be handled by multiple Stores.

Finally

There are probably a lot of things that could be written in a better way, I am still learning F# and finding new ways to do things daily. Overall I find that this language has nice constructs for many of the things I tried to do with C# when I tried functional constructs there. A lot less plumbing here as the language has built in support for many of the things, lets one focus on actually writing functionality instead of plumbing code. I am glad that I tried this out : )

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work

Tuesday, March 13, 2018

Functional Adventures in F# - Using Map Collections


This time we will look at how we can manipulate Map collections to store data.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers

I am no expert in F#, writing these posts is a way for me to learn and hopefully they can help someone else as well.

Graph of symbols

OK, so I want to create a symbol graph (influenced by some ideas in the book 'Gödel, Escher, Bach: An Eternal Golden Braid', a book I really recommend everyone to read)
This is just a very naive implementation but it raised some issues when dealing with F# in general.
Basic concept is that you have a bunch of symbols and each time they are triggered together with another symbol you either add an edge or increase the edges weight. And when 2 symbols are totally interlinked, they may become a new symbol of their own. But in this example I will just focus on the graph bit.

Defining types with and keyword

For example: In F#, everything that you use need to be defined before you use them. Meaning that if you have type X that references type Y, and then type Y reference type X.. You have a problem.
type X = { name:string; hits:int; outgoingEdges:Y}
type Y = { target:X; hits:int;  }
This would result in a "The type 'Y' is not defined." error. To solve it, we can change the second type definition to use the keyword and instead of type.
type X = { name:string; hits:int; outgoingEdges:Y}
and Y = { target:X; hits:int;  }

Lets start with defining some types.
module Symbolism =
    type Edge = { targetName:string; hits:int;  }
    type Symbol = { name:string; hits:int; outgoingEdges:Edges}
    and Edges = Map<string, Edge>
    type Symbols = Map<string, Symbol>
So here we see the usage of the and keyword as Symbol forward references Edges.

Using Map Collections

As you can see above, we have defined Edges and Symbols to be of the type Map, basically Map works like a Dictionary in C#, there are probably differences (other then the immutability part), but for my purposes I like to think that they are about the same. They both store objects as Key/Value pairs.

So, our first function will be getSymbol that takes a Symbols collection and the name of the symbol to get. Here I want to return the symbol from the Map if it exists, otherwise return a new empty symbol that has never been seen before. For this we will use pattern matching mechanism of F#, the task is quite trivial and thus a straight forward place to figure out how pattern matching works.
let getSymbol (symbols:Symbols) (key:string) =
    match symbols.ContainsKey key with
    | true -> symbols.[key]
    | _ -> { name = key; hits = 0; outgoingEdges = Map.empty }
We start by checking if they Symbols Map contains the key with the match [expression] with part.
The next line we define what happens if the result is true. We return the object that is in the Map, notice the dot (.) before the indexer. In C# you would access a dictionary as 'symbols[key]' but in F# you need the extra dot.
The second pattern line means, anything else.... So in this case it would be if the result is false we return a brand new record. The outgoingEdges is of the type Edges that is a Map<string, Edge>. I spent some time trying to initialize it with Edges.empty until I figured out that it is the Map.empty that should be used to initialize an empty map of any type.

Next, we want to add or update a value in the Map.
let addEdgeToSymbol (symbol:Symbol) (target:Symbol) = 
    let edge = getEdge symbol target
    let weightedEdge = increaseEdgeWeigth edge
    let updatedSymbol:Symbol = { symbol with outgoingEdges = symbol.outgoingEdges.Add(target.name, weightedEdge) }
    updatedSymbol
This took me a while as well. Trying to google how to update a value in a Map collection. Evidently it is built into the Add function.
If the key does not exist in the Map: a new Map with the key/value pair added to it is returned
If the key exists already in the Map: a new Map with the value in the key location updated is returned.
Pretty neat to not have to bloat the code with all the checks just to add or update, just call the Add function.
Here also, we want to update one field in the symbol record. We could define a new record and manually copy all the fields but that is just code debt waiting to blow. Instead there is a nice way to do this with the following line.
{ symbol with outgoingEdges = symbol.outgoingEdges.Add(target.name, weightedEdge) }
Here we say, that we want a new record based on the values in the record 'symbol' but with the following things updated. In this case the outgoingEdges field.

Full source code

module Symbolism =
    type Edge = { targetName:string; hits:int;  }
    type Symbol = { name:string; hits:int; outgoingEdges:Edges}
    and Edges = Map<string, Edge>
    type Symbols = Map<string, Symbol>
        
    let getSymbol (symbols:Symbols) (key:string) =
        match symbols.ContainsKey key with
        | true -> symbols.[key]
        | _ -> { name = key; hits = 0; outgoingEdges = Map.empty }
    
    let getEdge (symbol:Symbol) (target:Symbol) = 
        match symbol.outgoingEdges.ContainsKey target.name with
        | true -> symbol.outgoingEdges.[target.name]
        | _ -> { targetName = target.name; hits = 0; }

    let increasSymboleWeigth symbol:Symbol =
        let updatedSymbol:Symbol = { symbol with hits = symbol.hits + 1 }
        updatedSymbol

    let increaseEdgeWeigth edge:Edge =
        let updatedEdge:Edge = { edge with hits = edge.hits + 1 }
        updatedEdge
        
    let addEdgeToSymbol (symbol:Symbol) (target:Symbol) = 
        let edge = getEdge symbol target
        let weightedEdge = increaseEdgeWeigth edge
        let updatedSymbol:Symbol = { symbol with outgoingEdges = symbol.outgoingEdges.Add(target.name, weightedEdge) }
        updatedSymbol

    let addEdge (symbols:Symbols) (from:string) (target:string) = 
        let symbol = getSymbol symbols from
        let targetSymbol = getSymbol symbols target
        let edgedSymbol = addEdgeToSymbol symbol targetSymbol
        let weightedSymbol = increasSymboleWeigth edgedSymbol
        let weightedTargetSymbol = increasSymboleWeigth targetSymbol
        symbols
            .Add(from, weightedSymbol)
            .Add(target, weightedTargetSymbol)

Unit tests

[<TestClass>]
type SymbolismTest () =

    [<TestMethod>]
    member this.Symbolism_getSymbol_NoExistingSymbols () =
        let target : Symbols = Map.empty
        let actual = getSymbol target "ewan mcgregor"
        Assert.AreEqual("ewan mcgregor", actual.name)
        Assert.AreEqual(0, actual.outgoingEdges.Count)
  
  
    [<TestMethod>]
    member this.Symbolism_getSymbol_ExistingSymbol () =
        let target : Symbols = Map.empty.Add("ewan mcgregor", { name = "test item"; hits = 0; outgoingEdges = Map.empty })
        let actual = getSymbol target "ewan mcgregor"
        Assert.AreEqual("test item", actual.name)
        Assert.AreEqual(0, actual.outgoingEdges.Count)
        
    [<TestMethod>]
    member this.Symbolism_addEdgeToSymbol () =
        let from = { name = "daisy ridley"; hits = 0; outgoingEdges = Map.empty }
        let target = { name = "star wars"; hits = 0; outgoingEdges = Map.empty }
        let actual = addEdgeToSymbol from target
        Assert.AreEqual("daisy ridley", actual.name)
        Assert.AreEqual(1, actual.outgoingEdges.Count)
        Assert.IsTrue(actual.outgoingEdges.ContainsKey("star wars"))
        Assert.AreEqual(1, actual.outgoingEdges.["star wars"].hits)
        
    [<TestMethod>]
    member this.Symbolism_addEdge_NoExistingSymbols () =
        let target : Symbols = Map.empty
        let symbols = addEdge target "ewan mcgregor" "star wars"
        let actualewan = getSymbol symbols "ewan mcgregor"
        let actualStarWars = getSymbol symbols "star wars"
        Assert.AreEqual("ewan mcgregor", actualewan.name)
        Assert.AreEqual(1, actualewan.hits)
        Assert.AreEqual(1, actualewan.outgoingEdges.Count)
        Assert.AreEqual("star wars", actualStarWars.name)
        Assert.AreEqual(1, actualStarWars.hits)
        Assert.AreEqual(0, actualStarWars.outgoingEdges.Count)
        Assert.AreEqual(2, symbols.Count)

Here in the Symbolism_getSymbol_ExistingSymbol test we can see how to initialize a Map with a starting key value. I.e. by adding an item to an empty Map.



Merging 2 Map collections

Added 2018-03-21, did not want to create a new post for this.
At some point you may come across a problem that needs to be solved by merging 2 different Map collections that have the same type if Key and Value. To use this, we will use Map.fold to create a new Map collection that has the contents of both input Maps
let newUnits = Map.fold (fun acc key value -> Map.add key value acc) state.Units builtUnits




So there it is. We have gone through how to
  • Check if a key exists in a Map
  • Retrieve the value stored in the Map for a particular key
  • Add values to a Map
  • Update values in a Map
  • Initialize an empty Map
  • Initialize a Map with key/value pairs from the start
  • Merge 2 Map collections
All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work

Sunday, March 11, 2018

Functional Adventures in F# - Calling F# from C#


I have quite a lot of code lying around that is written in C# and going into a new project, the most natural is to start with C# as the core application. Now when I am exploring the functional paradigm with F#, I still want to be able to call all the F# code from C#. In other words, the most natural is to add new functionality to my existing applications using F# and not try to accomplish the 'everything shall be written in F#' utopia.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers

So, first off. We need to give our C# project the knowledge of F# types. So lets Nuget the FSharp.Core package and we are all set!

After that, calling the F# code from C# is easy. Lets use the code that we wrote in the last post with the Planner module.

We had 2 different types described in the module, the ScheduledAction that was a record type and the Schedule that was a list of ScheduledActions.

Lets look at some code:
using System;
using Microsoft.FSharp.Collections;

namespace app.Components
{
    public class Main
    {
        private FSharpList<Planner.ScheduledAction> _schedule;
        public bool Shutdown { get; set; }

        public void MainLoop()
        {
            while (!Shutdown)
            {
                var executableActions = Planner.getExecutableActions(DateTime.UtcNow, _schedule);
                foreach (var action in executableActions)
                {
                    Execute(action);
                    _schedule = Planner.removeFromSchedule(action, _schedule);
                }
            }
        }

        private void Execute(Planner.ScheduledAction action)
        {
            // execute business logic here
        }
    }
}

As you an see, the Schedule type did not follow through as I expected. It is exposed a FSharpList<ScheduledAction>, i.e. we need to write the whole thing instead of just using Schedule. Other then that there seems to be nothing strange here. Planner is identified by intellisense as a class.

If we try to modify the name of the ScheduledAction that we have received from the F# code, the code will not compile but instead show you the error
Property or indexer 'Planner.ScheduledAction.name' cannot be assigned to -- it is read only
This is due to that everything defined in F# is immutable (well almost everything).
So that is that, it just reference the FSharp.Core and your F# project from your C# project and you are good to go.

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time: Work to Live, Don’t Live to Work

Wednesday, March 7, 2018

OpenGL 4 with OpenTK in C# Part 15: Object picking by mouse


This time we will go through how to use the mouse to pick objects on the screen.

This is part 15 of my series on OpenGL4 with OpenTK.
For other posts in this series:
OpenGL 4 with OpenTK in C# Part 1: Initialize the GameWindow
OpenGL 4 with OpenTK in C# Part 2: Compiling shaders and linking them
OpenGL 4 with OpenTK in C# Part 3: Passing data to shaders
OpenGL 4 with OpenTK in C# Part 4: Refactoring and adding error handling
OpenGL 4 with OpenTK in C# Part 5: Buffers and Triangle
OpenGL 4 with OpenTK in C# Part 6: Rotations and Movement of objects
OpenGL 4 with OpenTK in C# Part 7: Vectors and Matrices
OpenGL 4 with OpenTK in C# Part 8: Drawing multiple objects
OpenGL 4 with OpenTK in C# Part 9: Texturing
OpenGL 4 with OpenTK in C# Part 10: Asteroid Invaders
Basic bullet movement patterns in Asteroid Invaders
OpenGL 4 with OpenTK in C# Part 11: Mipmap
OpenGL 4 with OpenTK in C# Part 12: Basic Moveable Camera
OpenGL 4 with OpenTK in C# Part 13: IcoSphere
OpenGL 4 with OpenTK in C# Part 14: Basic Text
OpenGL 4 with OpenTK in C# Part 15: Object picking by mouse

Introduction

As I've written in previous posts, I'm still not that good with all the math etc. writing these posts is a way for me to learn and hopefully someone else out there finds it useful as well. This time I found a great post that goes into the details of ray-casting to pick objects in 3D space over at Anton Gerdelan's blog: antongerdelan.net/opengl/raycasting.html

I've taken his base solution and made it work with the code-base that we use in this tutorial series. It did not work directly, so a lot of time was spent on what was going wrong.
So, now that credit is where it belongs, lets look at the code.

Code

First we want to calculate the ray from our camera location based on the mouse click on the screen. So first we need to subscribe to mouse events. So in our OnLoad method, we will add the following event handler.
MouseUp += OnMouseUp;

In our event handler we will check that it is the correct button and send the coordinates to the actual object picking method
private void OnMouseUp(object sender, MouseButtonEventArgs mouseButtonEventArgs)
{
    if (mouseButtonEventArgs.Button != MouseButton.Left)
        return;
    PickObjectOnScreen(mouseButtonEventArgs.X, mouseButtonEventArgs.Y);
}

This is the method that calculates the ray from the camera and into the screen. It takes the mouse coordinates and then backtracks the transformations back to world space. For more details, check antongerdelan.net/opengl/raycasting.html.
private void PickObjectOnScreen(int mouseX, int mouseY)
{
    // heavily influenced by: http://antongerdelan.net/opengl/raycasting.html
    // viewport coordinate system
    // normalized device coordinates
    var x = (2f * mouseX) / Width - 1f;
    var y = 1f - (2f * mouseY) / Height;
    var z = 1f;
    var rayNormalizedDeviceCoordinates = new Vector3(x, y, z);

    // 4D homogeneous clip coordinates
    var rayClip = new Vector4(rayNormalizedDeviceCoordinates.X, rayNormalizedDeviceCoordinates.Y, -1f, 1f);

    // 4D eye (camera) coordinates
    var rayEye = _projectionMatrix.Inverted() * rayClip;
    rayEye = new Vector4(rayEye.X, rayEye.Y, -1f, 0f);

    // 4D world coordinates
    var rayWorldCoordinates = (_camera.LookAtMatrix.Inverted() * rayEye).Xyz;
    rayWorldCoordinates.Normalize();
    FindClosestAsteroidHitByRay(rayWorldCoordinates);
}

After that we need to add a method that checks if the game object has been hit by the ray. I added this to the AGameObject class so that all game objects can be picked. Basically we begin by checking if the origin of the ray is inside the sphere that we are checking against. After that we check if the ray is pointing away from the object and lastly we perform Pythagorean method to determine if the ray is within the radius of the sphere that we are looking at. This ray method is something that I've had lying around for a while and used for various purposes.

public double? IntersectsRay(Vector3 rayDirection, Vector3 rayOrigin)
{
    var radius = _scale.X;
    var difference = Position.Xyz - rayDirection;
    var differenceLengthSquared = difference.LengthSquared;
    var sphereRadiusSquared = radius * radius;
    if (differenceLengthSquared < sphereRadiusSquared)
    {
        return 0d;
    }
    var distanceAlongRay = Vector3.Dot(rayDirection, difference);
    if (distanceAlongRay < 0)
    {
        return null;
    }
    var dist = sphereRadiusSquared + distanceAlongRay * distanceAlongRay - differenceLengthSquared;
    var result = (dist < 0) ? null : distanceAlongRay - (double?)Math.Sqrt(dist);
    return result;
}

The above method is called from the following that iterates all the objects currently active in the game and then tries to find the object that is closest (if there are many) to the origin of the ray.

private void FindClosestAsteroidHitByRay(Vector3 rayWorldCoordinates)
{
    AGameObject bestCandidate = null;
    double? bestDistance = null;
    foreach (var gameObject in _gameObjects)
    {
        if (!(gameObject is SelectableSphere) && !(gameObject is Asteroid))
            continue;
        var candidateDistance = gameObject.IntersectsRay(rayWorldCoordinates, _camera.Position);
        if (!candidateDistance.HasValue)
            continue;
        if (!bestDistance.HasValue)
        {
            bestDistance = candidateDistance;
            bestCandidate = gameObject;
            continue;
        }
        if (candidateDistance < bestDistance)
        {
            bestDistance = candidateDistance;
            bestCandidate = gameObject;
        }
    }
    if (bestCandidate != null)
    {
        switch (bestCandidate)
        {
            case Asteroid asteroid:
                _clicks += asteroid.Score;
                break;
            case SelectableSphere sphere:
                sphere.ToggleModel();
                break;
        }
    }
}

As you can see above, we have introduced a new GameObject for this part of the tutorial series, the SelectableSphere. It looks like the following:
public class SelectableSphere : AGameObject
{
    private ARenderable _original;
    private ARenderable _secondaryModel;
    public SelectableSphere(ARenderable model, ARenderable secondaryModel, Vector4 position, Vector4 direction, Vector4 rotation)
        : base(model, position, direction, rotation, 0)
    {
        _original = model;
        _secondaryModel = secondaryModel;
    }


    public override void Update(double time, double delta)
    {
        _rotation.Y = (float) ((time + GameObjectNumber) * 0.5);
        var d = new Vector4(_rotation.X, _rotation.Y, 0, 0);
        d.Normalize();
        _direction = d;
        base.Update(time, delta);
    }

    public void ToggleModel()
    {
        if (_model == _original)
            _model = _secondaryModel;
        else
            _model = _original;
    }
}
Basically an endlessly rotating sphere that will toggle its model when selected.

And lastly we generate these spheres in 2 layers in the OnLoad method in our MainWindow to have stuff on screen that can be selected.
var maxX = 2.5f;
var maxY = 1.5f;
for (float x = -maxX; x < maxX; x += 0.5f)
{
    for (float y = -maxY; y < maxY; y += 0.5f)
    {
        _gameObjects.Add(_gameObjectFactory.CreateSelectableSphere(new Vector4(x, y, -5f, 0)));
    }
}
for (float x = -maxX; x < maxX; x += 0.65f)
{
    for (float y = -maxY; y < maxY; y += 0.65f)
    {
        _gameObjects.Add(_gameObjectFactory.CreateSelectableSphereSecondary(new Vector4(x, y, -6f, 0)));
    }
}


End results


For the complete source code for the tutorial at the end of this part, go to: https://github.com/eowind/dreamstatecoding

So there, thank you for reading. Hope this helps someone out there : )

Until next time: Work to Live, Don’t Live to Work

Sunday, March 4, 2018

Functional Adventures in F# - A simple planner


Last year when I got sick I bought the openGL Bible and started writing about graphics programming with openTK in C# just to keep my mind occupied.
This year things turned bad again, so a new book... This time it is time for F# as I got really curious about functional programming during the last few months.

This post is part of a series:
Functional Adventures in F# - A simple planner
Functional Adventures in F# - Calling F# from C#
Functional Adventures in F# - Using Map Collections
Functional Adventures in F# - Application State
Functional Adventures in F# - Types with member functions
Functional Adventures in F# - Getting rid of loops
Functional Adventures in F# - Getting rid of temp variables
Functional Adventures in F# - The MailboxProcessor
Functional Adventures in F# - Persisting Application State
Functional Adventures in F# - Adding Snapshot Persisting
Functional Adventures in F# - Type-safe identifiers

So previous experience with functional ideas are from web tech like Redux and writing the core engine of my latest game project in C# with functional constructs. If you want to know more about that project you can read here:
Functional adventures in .NET C# - Part 1, Immutable Objects
Functional adventures in .NET C# - Part 2, Application State

So, next step is to actually use a language that is built for this. And luckily in .NET there is a full featured one.

At first I thought about converting a already existing piece of C# to F# to see the similarities and differences but I quickly omitted that idea. So for the next few posts I will try to write a Scheduler with F#.

If you do not have any development environment, please check out Visual Studio Community Edition, it is a great and free environment that I use for my private projects. It has support for F#.

Defining data

As a first step we need to define the different types of data that we have.

Step 1, decide if we should use CLI types or F# specific types.

CLI Types: Standard types that can be used in all .NET languages
F# Types: Specific to F#, cannot be used outside. Designed with functional paradigm in mind and does not have null and are immutable. They also have built in equality checking and pretty printing.

Our choice here will be F# types to be able to use the language to its full extent as it was designed.
As a starter, we want to be able to build a collection of planned Actions. So lets first define what we will be planning.
type ScheduledAction = { time:DateTime; name:string; }
This pretty much describes a new Record type that has a time and a name.
Next lets define the Schedule as an ImmutableSortedSet
type Schedule = ScheduledAction list
Nothing fancy here either, as you can see compared to C# the order of things is switched, where we would write list<ScheduledAction> we just write ScheduledAction litinstead. I think you can use the C# notation as well but I am trying to stick with the way my book says is the F# way.

Adding functionality

Next we will want to add a ScheduledAction to the current Schedule. As everything is immutable we will define a function that takes the new ScheduledAction and the current Schedule, and returns a new Schedule with the ScheduledAction added to it. We will not change the current Schedule, just return a new Schedule with the change made to it.
So just add a new item.
let schedule (action:ScheduledAction) (currentPlan:Schedule) =
    let newPlan = action :: currentPlan
    newPlan
Here we define the function schedule that takes 2 arguments: action and currentPlan. The parenthesis are not needed if you do not supply the type arguments. But my compiler complained (FS0613 Syntax error in labelled type argument) if I didn't add them so hence the parenthesis.
The item :: tail construct takes an item and a list and returns a new list with the item as the first item and the tail as ... the tail.

Next step is to retrieve actions that should be executed. So lets define a new function, this will not change the plan, just return a new list with items that should be executed.
let getExecutableActions (currentTime:DateTime) (currentPlan:Schedule) =
    let actions = List.filter (fun x -> x.time <= currentTime) currentPlan
    actions
Here we use the List.filter function to apply a filter to each item in the list. As Schedule is a list this works fine.

Finally we want to remove items that have been executed (in some way, outside of scope here).
let removeFromSchedule (action:ScheduledAction) (currentPlan:Schedule) =
    let newPlan = List.filter (fun x -> compare x action <> 0) currentPlan
    newPlan
Basically the method is the same as the above, but it compares each element in the list to the one we want to remove, and if it is the one we want to remove, we just do not add it to the new list. I.e. filter it away.

The complete planner module

namespace scheduler

module Planner =
    open System
    type ScheduledAction = { time:DateTime; name:string;}
    type Schedule = ScheduledAction list

    let schedule (action:ScheduledAction) (currentPlan:Schedule) =
        let newPlan = action :: currentPlan
        newPlan

    let getExecutableActions (currentTime:DateTime) (currentPlan:Schedule) =
        let actions = List.filter (fun x -> x.time <= currentTime) currentPlan
        actions

    let removeFromSchedule (action:ScheduledAction) (currentPlan:Schedule) =
        let newPlan = List.filter (fun x -> compare x action <> 0) currentPlan
        newPlan
So here we have the complete planner in 18 lines of code.

Adding unit tests

Of course we should have unit tests to ensure correct functionality. Luckily F# makes it quite easy to write tests as well.
namespace scheduler.tests

open System
open Microsoft.VisualStudio.TestTools.UnitTesting
open scheduler
open scheduler.Planner

[<TestClass>]
type PlannerTests () =

    [<TestMethod>]
    member this.Planner_schedule_AddItemToEmptySchedule () =
        let item : ScheduledAction = { time = DateTime.UtcNow; name = "first item" }
        let target : Schedule = []
        let newSchedule = schedule item target
        Assert.AreEqual(item, newSchedule.Head);
  
    [<TestMethod>]
    member this.Planner_schedule_AddItemToNonEmptySchedule () =
        let item : ScheduledAction = { time = DateTime.UtcNow; name = "first item" }
        let item2 : ScheduledAction = { time = DateTime.UtcNow; name = "second item" }
        let target : Schedule = []
        let intermediateSchedule = schedule item target
        let newSchedule = schedule item2 intermediateSchedule
        Assert.AreEqual(item2, newSchedule.Head);
        Assert.AreEqual(2, newSchedule.Length);

For the schedule function, we add 2 tests, one that adds a new item to an empty Schedule, and one that adds an extra item to a Schedule that already has items.
       
    [<TestMethod>]
    member this.Planner_getExecutableActions_EmptySchedule () =
        let target : Schedule = []
        let actual = getExecutableActions DateTime.UtcNow target
        Assert.AreEqual(0, actual.Length);

   
    [<TestMethod>]
    member this.Planner_getExecutableActions_SingleItemSchedule_NoExecutable () =
        let currentTime = DateTime.UtcNow;
        let item : ScheduledAction = { time = currentTime; name = "first item" }
        let target : Schedule = []
        let newSchedule = schedule item target
        let currentTime = currentTime.AddSeconds(-2.0);
        let actual = getExecutableActions currentTime newSchedule
        Assert.AreEqual(0, actual.Length);
 
 
    [<TestMethod>]
    member this.Planner_getExecutableActions_SingleItemSchedule_SingleExecutable () =
        let currentTime = DateTime.UtcNow;
        let item : ScheduledAction = { time = currentTime; name = "first item" }
        let target : Schedule = []
        let newSchedule = schedule item target
        let currentTime = currentTime.AddSeconds(2.0);
        let actual = getExecutableActions currentTime newSchedule
        Assert.AreEqual(1, actual.Length);

    [<TestMethod>]
    member this.Planner_getExecutableActions_MultiItemSchedule_SingleExecutable () =
        let currentTime = DateTime.UtcNow;
        let item : ScheduledAction = { time = currentTime.AddYears(1); name = "first item" }
        let item2 : ScheduledAction = { time = currentTime; name = "second item" }
        let target : Schedule = []
        let intermediateSchedule = schedule item target
        let newSchedule = schedule item2 intermediateSchedule
        let currentTime = currentTime.AddSeconds(2.0);
        let actual = getExecutableActions currentTime newSchedule
        Assert.AreEqual(item2, actual.Head);
        Assert.AreEqual(1, actual.Length);
For the getExecutableActions function we add 4 tests.

  • One that checks so that the special case of an empty Schedule is handled correctly. 
  • Secondly that the time filter is correct when we have a single item in the list that is not OK to execute yet. 
  • Thirdly a single item that is OK to execute
  • and lastly a Schedule with multiple items with 1 able to execute and 1 not able to

    [<TestMethod>]
    member this.Planner_removeFromSchedule_RemoveItem () =
        let item : ScheduledAction = { time = DateTime.UtcNow; name = "first item" }
        let item2 : ScheduledAction = { time = DateTime.UtcNow; name = "second item" }
        let target : Schedule = []
        let intermediateSchedule = schedule item target
        let newSchedule = schedule item2 intermediateSchedule
        let actual = removeFromSchedule item newSchedule
        Assert.AreEqual(item2, actual.Head);
        Assert.AreEqual(1, actual.Length);
And lastly we check that removeFromSchedule function does just that, removes the item we want to remove and leaves the rest in the list. (in the new list that is returned, the original is not changed... remember immutable)


All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment, not required but appreciated! :)

Hope this helps someone out there!
Until next time, why not try brewing some beer?