Showing posts with label Performance. Show all posts
Showing posts with label Performance. Show all posts

Saturday, September 29, 2018

Immutability in .NET

First off, what are immutable objects

Immutable is pretty much a fancy word for unchangeable. In other words, once you create an object it will have the same properties for its entire life. No way to reassign for example a name, if you want to do that you would have to create a new object with the same properties but with the new name. Immutable = Not Mutable

Why bother?

I didn't understand the point of immutable objects, I though that as long as we encapsulate and follow good object oriented guidelines then we are good to go.. Right?
Let's look at a naive example of things that can go wrong:
public class CSharpPoco
{
    private uint _value;
    public uint Value => _value;
    public void Increment()
    {
        _value++;
    }
    public uint GetNext()
    {
        Increment();
        return _value;
    }
}
        
public class ServiceLayer
{
    private CSharpPoco _poco = new CSharpPoco();
    public CSharpPoco GetPoco()
    {
        return _poco;
    }
    public uint PreviewNextValue()
    {
        return _poco.GetNext();
    }
}

In the service layer we have 2 methods to call, get the poco and preview its next value.
The following service code would throw the side effect exception:

var service = new ServiceLayer();
var poco = service.GetPoco();
var firstValue = poco.Value;
var preview = service.GetPreviewValue();
if (preview <= firstValue)
    throw new Exception("preview can't be smaller then or equal to the previus value");
if (poco.Value == preview)
    throw new Exception("side effect");

Meaning, we get an unexpected side-effect by calling the second method in the service. It manages to change the value of the first service calls result. It was zero, but after the preview call it has been set to 1. This is quite a simple example, but after having debugged scenarios like these in production code with huge code bases.. You get the idea, a unexpected side effect is often a cause for hard-to-find bugs.
So, how could immutability have helped us here? once the var poco = service.GetPoco() was called, the result would never change. Preferably we would not be able to reasign the variable poco to other references either (similar to the const keyword in TypeScript or F# let). So instead of var I would have liked a const poco = service.

What options do we have in .NET?

Readonly objects in C# with constructor lists

public class CSharpImmutableConstructorList
{
    public readonly Guid Id;
    public readonly string Name;
    public readonly string DisplayName;
    public CSharpImmutableConstructorList(Guid id, string name, string displayName)
    {
        Id = id;
        Name = name;
        DisplayName = displayName;
    }

    public CSharpImmutableConstructorList SetDisplayName(string displayName)
    {
        return new CSharpImmutableConstructorList(Id, Name, displayName);
    }
}
Basically the magic here is done by the readonly keyword that states that only the constructor can set the value of the variable. Once it has been set, it can only be read. For larger objects, it can get a little heavy on the constructor and you have to supply all the values every time you want to change. For example the SetDisplayName function, it supplies the Id and Name even though they have the same value as before.
If we add more properties to this class, we would have to change all calls to the constructor and add the new properties for them as well. So a little heavy on the maintenance side.

Readonly objets in C# with Modify pattern

public class CSharpImmutableModifyPattern
{
    public readonly Guid Id;
    public readonly string Name;
    public readonly string DisplayName;
    public static CSharpImmutableModifyPattern Default { get; } = new CSharpImmutableModifyPattern(Guid.Empty, string.Empty, string.Empty);
    private CSharpImmutableModifyPattern(Guid id, string name, string displayName)
    {
        Id = id;
        Name = name;
        DisplayName = displayName;
    }
    public CSharpImmutableModifyPattern Modify(Guid? id = null, string name = null, string displayName = null)
    {
        return new CSharpImmutableModifyPattern
            (
                id ?? Id,
                name ?? Name,
                displayName ?? DisplayName
            );
    }
    public CSharpImmutableModifyPattern SetDisplayName(string displayName)
    {
        return Modify(displayName: displayName);
    }
}
This is pretty much the same pattern as I've described in a previous post (Functional adventures in .NET C# - Part 1, immutable objects), the difference is that we use the readonly members instead of get-only fields.
Also note that the constructor is set to private to prevent usage of it from outside, instead we will use the Default static get-only field that sets up a default invariant of the class and call the new method Modify on it.
The nice thing with the Modify pattern is that you don't have to supply anything else then the changed property as shown in the SetDisplayName method. Even if we add new parameters, we don't have to change the SetDisplayName method as it specifies that it only wants to supply the displayName and nothing else.

Record types in F#

module Techdump =
    open System
    type FSharpRecord =
        {
            Id : Guid
            Name : string
            DisplayName : string
        }
        member this.SetDisplayName displayName =
            { this with DisplayName = displayName }

The F# Record version of the same type as show before. Used from C# this type feels similar to the C# Constructor List version as you have to supply all the constructor parameters when calling new. The magic happens in the SetDisplayName function that takes a new displayName value and then calls the record constructor with the current record and whatever fields have changed. I.e. the with keyword in record construction. We get the power of CSharpImmutableModifyPattern, but without having to maintain a Modify method, it is all included in the language.


Performance

In this part we will look at the run-time performance the different immutable patterns described above. 

Performance test source code


CSharpPoco
public class CSharpPoco
{
    public Guid Id { get; private set; }
    public string Name { get; private set; }
    public string DisplayName { get; private set; }
    public CSharpPoco(Guid id, string name, string displayName)
    {
        Id = id;
        Name = name;
        DisplayName = displayName;
    }

    public void SetDisplayName(string displayName)
    {
        DisplayName = displayName;
    }
}
Added a C# poco object so that we have a baseline to run against. I.e, here we just mutate the object. No new instance is created, this is the traditional way of changing a value in an object.

Performance tester
public class ImmutabilityPerformance
{
 public void Execute()
 {
  Log("--------------------------------------");
  Log("... ImmutabilityPerformance");
  ExecuteTest(1_000_000);
 }
 
 
 private void ExecuteTest(int iterations)
 {
  string s = string.Empty;
  try
  {
   var data = new List<string> { "Melissa Lewis", "Maya", "Smith", "Beverly Marsh", "Jane Vasko", "Molly Bloom" };
   var csharpPocoTimings = new RunningAverage();
   var fsharpRecordTimings = new RunningAverage();
   var csharpConstructorTimings = new RunningAverage();
   var csharpModifyTimings = new RunningAverage();
   for (int i = 0; i < iterations; i++)
   {
    var csharpPoco = new CSharpPoco(Guid.NewGuid(), "Jessica Chastain", "Jessica Chastain");
    var fsharpRecordOriginal = new Techdump.FSharpRecord(Guid.NewGuid(), "Jessica Chastain", "Jessica Chastain");
    var csharpConstructorOriginal = new CSharpImmutableConstructorList(Guid.NewGuid(), "Jessica Chastain", "Jessica Chastain");
    var csharpModifyOriginal = CSharpImmutableModifyPattern.Default.Modify(Guid.NewGuid(), "Jessica Chastain", "Jessica Chastain");

    for (int dataIndex = 0; dataIndex < data.Count; dataIndex++)
    {
     var item = data[dataIndex];


     csharpPocoTimings.Add(TimeAction(() =>
     {
      csharpPoco.SetDisplayName(item);
     }));
     if (csharpPoco != null)
      s = csharpPoco.DisplayName;

     Techdump.FSharpRecord fsharpRecordModified = null;
     fsharpRecordTimings.Add(TimeAction(() =>
     {
      fsharpRecordModified = fsharpRecordOriginal.SetDisplayName(item);
     }));
     if (fsharpRecordModified != null)
      s = fsharpRecordModified.DisplayName;

     CSharpImmutableConstructorList csharpConstructorModified = null;
     csharpConstructorTimings.Add(TimeAction(() =>
     {
      csharpConstructorModified = csharpConstructorOriginal.SetDisplayName(item);
     }));
     if (fsharpRecordModified != null)
      s = csharpConstructorModified.DisplayName;

     CSharpImmutableModifyPattern csharpModifyModified = null;
     csharpModifyTimings.Add(TimeAction(() =>
     {
      csharpModifyModified = csharpModifyOriginal.SetDisplayName(item);
     }));
     if (csharpModifyModified != null)
      s = csharpModifyModified.DisplayName;
    }
   }
   Log($"CSharpPoco\tIterations:\t{iterations}\tAverage:\t{csharpPocoTimings.Average:0.000}\tticks\tTotal:\t{csharpPocoTimings.Total:0.000}\tticks");
   Log($"FSharpRecord\tIterations:\t{iterations}\tAverage:\t{fsharpRecordTimings.Average:0.000}\tticks\tTotal:\t{fsharpRecordTimings.Total:0.000}\tticks");
   Log($"CSharpImmutableConstructorList\tIterations:\t{iterations}\tAverage:\t{csharpConstructorTimings.Average:0.000}\tticks\tTotal:\t{csharpConstructorTimings.Total:0.000}\tticks");
   Log($"CSharpImmutableModifyPattern\tIterations:\t{iterations}\tAverage:\t{csharpModifyTimings.Average:0.000}\tticks\tTotal:\t{csharpModifyTimings.Total:0.000}\tticks");
  }
  catch (Exception ex)
  {
   Log($"Fail\tDataCount\t2\tIterations:\t{iterations}\tFailed\t{ex.Message}");
  }
 }


 private float TimeAction(Action action)
 {
  var sw = Stopwatch.StartNew();
  action();
  sw.Stop();
  return sw.ElapsedTicks;
 }

 private void Log(string s)
 {
  Console.WriteLine(s);
  File.AppendAllText(@"c:\temp\enumToStringTest.txt", $"{s}{Environment.NewLine}");
 }
 
}

And the RunningAverage class from a previous post.

Results

ImmutabilityPerformance
Iterations
Average (ticks)
Total (ticks)
CSharpPoco 1000000 0.090 537402
FSharpRecord 1000000 0.121727156
CSharpImmutableConstructorList 1000000 0.120 720220
CSharpImmutableModifyPattern 1000000 0.161 965545

Here we can see that the poco baseline test is the fastest by far. Using the F# Record type and C# locked down object with constructor are pretty much equal in execution. The modify pattern in the C# code really drops the performance but gives more readable code, at least if we have long constructor lists in objects.

Rewritten increment example

Let's go back and look at the first increment example again and try to rewrite it with immutability
public class CSharpImmutable
{
    public readonly uint Value;
    public CSharpImmutable(uint value)
    {
        Value = value;
    }
    public CSharpImmutable Increment()
    {
        return new CSharpImmutable(Value + 1);
    }
    public uint GetNext()
    {
        return Value + 1;
    }
}

And in F#
type FsharpIncrement =
    {
        Value : uint32    
    }
    member this.Increment () =
        { this with Value = this.Value + 1u }
    member this.GetNext () =
        this.Value + 1u

Conclusions

In my opinion Immutable objects are the way to go if you want to write maintainable code for the long run, what technique you choose is entirely up to you. There are probably other ways to achieve the same that I don't know about, if you have ideas please post a comment!

After I started writing F# code, and started getting all the immutability features for free and especially the with keyword for records has persuaded me to start writing all my core business models in F#.
let point' = { point with X = 2; }
In my eyes, that line of code just there, is so damn beautiful.



All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Use for whatever you want, how you want! If you find this helpful, please leave a comment or share a link, not required but appreciated! :)

Friday, September 28, 2018

FSharp Enum ToString vs. C# Enum Dictionary Lookup for Localization


I was reading Tao Liu's post regarding enums with space for localization to save time and I just had to run a test if this really was faster then defining a lookup table?


On my computer
--------------------------------------
... FsharpEnumToString vs. CSharpEnumDictionaryLookup
FSharpToString  Iterations: 10000000;  Average: 0,287 ticks; Total: 14374450,000 ticks;
CSharpLookup    Iterations: 10000000;  Average: 0,134 ticks; Total: 6688315,000 ticks;

So, a simple dictionary lookup is still faster then ToString of a F# enum.
So I'll guess my team will keep the localization as data files / database table that is loaded at runtime into a variable. This way we can outsource the translations to other teams and specialists while keeping the code clean. A misspelled enum member by a tired developer does not result in a hotfix of an application with new builds etc, just a data file update in runtime.


The code used to perform this test:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using fsharp.techdump;

namespace techdump.console.Tests
{
    public class FsharpEnumToStringVsLookup
    {
        private Dictionary<CSharpEnum, string> _lookup = new Dictionary<CSharpEnum, string>
        {
            { CSharpEnum.Registration, "Registration" },
            { CSharpEnum.UnderReview, "Under Review" },
            { CSharpEnum.Approval, "Approval" },
            { CSharpEnum.Release, "Release" },
            { CSharpEnum.PostRelase, "Post Release" }
        };
        private enum CSharpEnum
        {
            Registration = 0,
            UnderReview = 1,
            Approval = 2,
            Release = 3,
            PostRelase = 4,
        }
        public void Execute()
        {
            Log("--------------------------------------");
            Log("... FsharpEnumToString vs. CSharpEnumDictionaryLookup");
            ExecuteTest(10_000_000);
        }
        
        
        private void ExecuteTest(int iterations)
        {
            string s = string.Empty;
            try
            {
                var index = new List<int> { 0, 1, 2, 3, 4 };
                var fsharpToStringTimings = new RunningAverage();
                var csharpLookupTimings = new RunningAverage();
                for (int i = 0; i < iterations; i++)
                {
                    for (int dataIndex = 0; dataIndex < index.Count; dataIndex++)
                    {
                        var item = index[dataIndex];
                        var fsharpEnumMember = (Techdump.FsharpEnum)item;
                        fsharpToStringTimings.Add(TimeAction(() =>
                        {
                            s = item.ToString();
                        }));
                        if (!string.IsNullOrEmpty(s))
                            s = string.Empty;
                        var csharpEnumMember = (CSharpEnum)item;
                        csharpLookupTimings.Add(TimeAction(() =>
                        {
                            s = _lookup[csharpEnumMember];
                        }));
                        if (!string.IsNullOrEmpty(s))
                            s = string.Empty;
                    }
                }
                Log($"FSharpToString\tIterations:\t{iterations}\tAverage:\t{fsharpToStringTimings.Average:0.000}\tticks\tTotal:\t{fsharpToStringTimings.Total:0.000}\tticks");
                Log($"CSharpLookup\tIterations:\t{iterations}\tAverage:\t{csharpLookupTimings.Average:0.000}\tticks\tTotal:\t{csharpLookupTimings.Total:0.000}\tticks");
            }
            catch (Exception ex)
            {
                Log($"Fail\tDataCount\t2\tIterations:\t{iterations}\tFailed\t{ex.Message}");
            }
        }


        private float TimeAction(Action action)
        {
            var sw = Stopwatch.StartNew();
            action();
            sw.Stop();
            return sw.ElapsedTicks;
        }

        private void Log(string s)
        {
            Console.WriteLine(s);
            File.AppendAllText(@"c:\temp\enumToStringTest.txt", $"{s}{Environment.NewLine}");
        }
        
    }
    
}

FSharp Enum defined as follows:
namespace fsharp.techdump

module Techdump =
    type public FsharpEnum =
        Registration = 0
        | ``Under Review``=1
        | Approval = 2
        | Release = 3
        | ``Post Release`` = 4


And the RunningAverage class from a previous post.


Thursday, December 29, 2016

Performance: Enum ToString or GetName


Usually I am too lazy to bother using anything else than the ordinary ToString when working with enums, but I always have that filthy feeling that I am cheating.
If there is a dedicated method in the framework to do something, there probably is a reason for it. And knowing that the Enum class contains a GetName method to get the string representation of a enum value made me want to know if there is a performance difference between the two.

First, the tested Enum.GetName method, for sake of argument wrapped up as an Extension for easier and cleaner usage.

public static class Extension
{
    public static string GetEnumName<T>(this T t) where T : struct, IConvertible
    {
        return Enum.GetName(typeof (T), t);
    }
}

Back to extensions list

The test rigg

The test rigg is basically the same that I used in Performance of different lock methods in .net.
Uses RunningAverage to calculate the average without storing each individual number.
Just cleaned up and simplified for this scenario:

Data

private enum SmallEnum
{
    Small,
    Larger
}
private enum LargerEnum
{
    a, b, c, d, e, f, g, h,
    aaaaa, bbbbb, ccccc, ddddd,
    xxxxx, yyyyy, zzzzz,
    VeryLongNameThisIs,
    OtherName, Test, oooo, olf,
    eowind, eeeee, ss, qqqqq,
    mu, miu, bach, escher,
    godel, code, number, last
}

Decided to test 2 different enums with different number of elements to see if that has any impact. The small has 2 elements and the larger has 32.

Code

private void ExecuteTestSmall(int iterations)
{
    var type = "small";
    string s = string.Empty;
    try
    {
        var testData = GenerateTestDataSmall();
        var toStringTimings = new RunningAverage();
        var getNameTimings = new RunningAverage();
        for (int i = 0; i < iterations; i++)
        {
            for (int dataIndex = 0; dataIndex < testData.Count; dataIndex++)
            {
                var item = testData[dataIndex];
                toStringTimings.Add(TimeAction(() =>
                {
                    s = item.ToString();
                }));
                if (!string.IsNullOrEmpty(s))
                    s = string.Empty;
                getNameTimings.Add(TimeAction(() =>
                {
                    s = item.GetEnumName();
                }));
                if (!string.IsNullOrEmpty(s))
                    s = string.Empty;
            }
        }
        Log($"{type}ToString\tDataCount\t2\tIterations:\t{iterations}\tAverage:\t{toStringTimings.Average:0.000}\tticks");
        Log($"{type}GetName\tDataCount\t2\tIterations:\t{iterations}\tAverage:\t{getNameTimings.Average:0.000}\tticks");
    }
    catch (Exception ex)
    {
        Log($"{type}Fail\tDataCount\t2\tIterations:\t{iterations}\tFailed\t{ex.Message}");
    }
}

Ok, I was lazy. I created 2 methods instead of putting the effort to reuse it. The above is for the SmallEnum. Exactly the same code for the LargerEnum tester with difference in GenerateTestDataSmall to return a List<LargerEnum> instead of List<SmallEnum>.

This was executed as follows:
public void Execute()
{
 Log("--------------------------------------");
 Log("... single thread");
 ExecuteTestSmall(250000);
 ExecuteTestLarger(250000);
}

250000 times each.

Results

ToString enum size 2 Iterations: 250000 Average: 1.231 ticks
GetName enum size 2 Iterations: 250000 Average: 0.662 ticks
ToString enum size 32 Iterations: 250000 Average: 1.357 ticks
GetName enum size 32 Iterations: 250000 Average: 0.692 ticks

Conclusion

Using the dedicated Enum.GetName function to get the string representation instead of the lazy object.ToString() seems to be twice faster. But in the end, we are talking around 1 tick, so maybe not the first place to start optimizing if you have a performance issue in you application. As I wrote above, it just feels a little bit lazy to not do it the correct way ;)

All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Hope this helps someone out there :)



Saturday, December 3, 2016

Performance of different lock methods in .net


Something that pops up from time to time is what kind of thread synchronization technique should be used when dealing with shared ´resources in a multi-threaded environment. And yes, I know that the best would be to not share resources at all but that is not always an option, especially when working with legacy code-bases and applications that have organically grown over time.

Shared resource access types

I've included the following operations in the test.
public interface ICacheTest
{
    void Add(CacheItem item);
    CacheItem Get(Guid id);
    CacheItem Search(Guid id);
    void Clear();
}


Add
Adds an item to the dictionary

Get
Gets an item with key lookup.

Search
There will always be scenarios when the key lookup isn't enough. Most code-bases that I've seen tend to go the LINQ way. Some have extra lookup dictionaries. For this test I wanted to include something that takes a little more time then a lookup, so LINQ query is used.

Techniques in the test

Unsafe

No thread synchronization, the test class basically just wraps an Dictionary<Guid, CacheItem>
public class CacheUnsafe : ICacheTest
{
    private readonly Dictionary<Guid, CacheItem> _cache = new Dictionary<Guid, CacheItem>();
    public void Add(CacheItem item)
    {
        _cache.Add(item.Id, item);
    }
    public CacheItem Get(Guid id)
    {
        if (_cache.ContainsKey(id))
            return _cache[id];
        return null;
    }
    public CacheItem Search(Guid id)
    {
        var item = (from x in _cache.Values where x.Id.Equals(id) select x).FirstOrDefault();
        return item;
    }
    public void Clear()
    {
        _cache.Clear();
    }
}


Full lock

using the lock key-word to just allow one thread at a time to execute within the block of code. This is usually my go-to method as it is easy to implement and usually performs good enough.
public class CacheFullLock : ICacheTest
{
    private readonly Dictionary<Guid, CacheItem> _cache = new Dictionary<Guid, CacheItem>();
    public void Add(CacheItem item)
    {
        lock (_cache)
        {
            _cache.Add(item.Id, item);
        }
    }
    public CacheItem Get(Guid id)
    {
        lock (_cache)
        {
            if(_cache.ContainsKey(id))  
                return _cache[id];
        }
        return null;
    }
    public CacheItem Search(Guid id)
    {
        lock (_cache)
        {
            var item = (from x in _cache.Values where x.Id.Equals(id) select x).FirstOrDefault();
            return item;
        }
    }
    public void Clear()
    {
        lock (_cache)
        {
            _cache.Clear();
        }
    }
}

Full lock with indexed search

Same as the full lock but also tries to optimize the search to a lookup instead. So in this simplified test the lookup looks kinda unnecessary but in the real world it could be looking up with multiple parameters instead of running the Linq query.
public CacheItem Search(Guid id)
{
    lock (_cache)
    {
        if (_index.ContainsKey(id))
            return _cache[_index[id]];
        return null;
    }
}

ReaderWriterLock

Using the ReaderWriterLock class. Allows multiple readers to access the resource at the same time, but only 1 writer at a time.
public class CacheWriteLock : ICacheTest
{
    private readonly Dictionary<Guid, CacheItem> _cache = new Dictionary<Guid, CacheItem>();
    private readonly ReaderWriterLock _lock = new ReaderWriterLock();
    public void Add(CacheItem item)
    {
        _lock.AcquireWriterLock(1000);
        try
        {
            _cache.Add(item.Id, item);
        }
        finally
        {
            _lock.ReleaseWriterLock();
        }
    }
    public CacheItem Get(Guid id)
    {
        _lock.AcquireReaderLock(1000);
        try
        {
            if (_cache.ContainsKey(id))
                return _cache[id];
            return null;
        }
        finally
        {
            _lock.ReleaseReaderLock();
        }
    }
    public CacheItem Search(Guid id)
    {
        _lock.AcquireReaderLock(1000);
        try
        {
            var item = (from x in _cache.Values where x.Id.Equals(id) select x).FirstOrDefault();
            return item;
        }
        finally
        {
            _lock.ReleaseReaderLock();
        }
    }
    public void Clear()
    {
        _lock.AcquireWriterLock(1000);
        try
        {
            _cache.Clear();
        }
        finally
        {
            _lock.ReleaseWriterLock();
        }
    }
}

ConcurrentDictionary

The built in concurrent dictionary that is thread-safe. I tried to use this a long way ago when it was first introduced, but found it under-performing compared to a simple Full lock strategy. So up to the test again. At least it looks easy to implement and you do not have to care about the thread safety yourself at it is handled internally.
public class CacheConcurrent : ICacheTest
{
    private readonly ConcurrentDictionary<Guid, CacheItem> _cache = new ConcurrentDictionary<Guid, CacheItem>();
    public void Add(CacheItem item)
    {
        _cache.AddOrUpdate(item.Id, item, (key, oldValue) => item);
    }
    public CacheItem Get(Guid id)
    {
        if (_cache.ContainsKey(id))
            return _cache[id];
        return null;
    }
    public CacheItem Search(Guid id)
    {
        var item = (from x in _cache.Values where x.Id.Equals(id) select x).FirstOrDefault();
        return item;
    }
    public void Clear()
    {
        _cache.Clear();
    }
}

Single-threaded test

Measuring ticks elapsed for add, get, search and clear methods. Number of items are increased with each test epoch. Each epoch executes the test for 1000 iterations.
So no multi-threading here, just a check of baseline differences between the solutions. I.e. time to acquire the lock, ability to cope with different number of elements etc.

The test rig:

Uses RunningAverage to calculate the average without storing each individual number.

private void ExecuteTestSingle(ICacheTest cache, int dataCount, int iterations)
{
    var type = cache.GetType().Name;
    try
    {
        var testData = GenerateTestData(dataCount);
        var addTimings = new RunningAverage();
        var clearTimings = new RunningAverage();
        var getTimings = new RunningAverage();
        var searchTimings = new RunningAverage();
        for (int i = 0; i < iterations; i++)
        {
            for (int dataIndex = 0; dataIndex < testData.Count; dataIndex++)
            {
                var item = testData[dataIndex];
                addTimings.Add(TimeAction(() =>
                {
                    cache.Add(item);
                }));
                getTimings.Add(TimeAction(() =>
                {
                    cache.Get(item.Id);
                }));
                searchTimings.Add(TimeAction(() =>
                {
                    cache.Search(item.Id);
                }));
            }
            clearTimings.Add(TimeAction(cache.Clear));
        }
        Log($"{type}\tAdd\tDataCount\t{dataCount}\tIterations:\t{iterations}\tAverage:\t{addTimings.Average:0.000}\tticks");
        Log($"{type}\tGet\tDataCount\t{dataCount}\tIterations:\t{iterations}\tAverage:\t{getTimings.Average:0.000}\tticks");
        Log($"{type}\tSearch\tDataCount\t{dataCount}\tIterations:\t{iterations}\tAverage:\t{searchTimings.Average:0.000}\tticks");
        Log($"{type}\tClear\tDataCount\t{dataCount}\tIterations:\t{iterations}\tAverage:\t{clearTimings.Average:0.000}\tticks");
    }
    catch (Exception ex)
    {
        Log($"{type}\tFail\tDataCount\t{dataCount}\tIterations:\t{iterations}\tFailed\t{ex.Message}");
    }
}

private float TimeAction(Action action)
{
    var sw = Stopwatch.StartNew();
    action();
    sw.Stop();
    return sw.ElapsedTicks;
}

Multi-threaded test

3 threads executing at the same time.

  • 1 thread Adds
  • 1 thread Gets
  • 1 thread Searches
Number of successful operations withing the allowed timespan (50 milliseconds) are counted.
The test is executed 1000 times per type and an average is calculated

The test rig:

private void ExecuteTestMulti(ICacheTest cache, int dataCount, int iterations)
{
    var type = cache.GetType().Name;
    try
    {
        var testData = GenerateTestData(dataCount);
        var getAverage = new RunningAverage();
        var searchAverage = new RunningAverage();
        var writeAverage = new RunningAverage();
        for (int i = 0; i < iterations; i++)
        {
            foreach (var data in testData)
                cache.Add(data);

            var d = testData.GetRandom();
            var startTime = DateTime.UtcNow.AddMilliseconds(8);
            var getter = CreateThread(startTime, () => { cache.Get(d.Id); });
            var searcher = CreateThread(startTime, () => { cache.Search(d.Id); });
            var writer = CreateThread(startTime, () => {cache.Add(CacheItem.Create());});

            getter.Thread.Start(startTime);
            writer.Thread.Start(startTime);
            searcher.Thread.Start(startTime);
            getter.Thread.Join();
            searcher.Thread.Join();
            writer.Thread.Join();
            if (!getter.Verify(type, Log)
                || !searcher.Verify(type, Log)
                || !writer.Verify(type, Log))
            {
                return;
            }
            getAverage.Add(getter.WorkIterations);
            searchAverage.Add(searcher.WorkIterations);
            writeAverage.Add(writer.WorkIterations);
            cache.Clear();
        }
        Log($"{type}\tGets\tDataCount\t{dataCount}\tIterations:\t{iterations}\tAverage:\t{getAverage.Average:0}\tgets");
        Log($"{type}\tSearch\tDataCount\t{dataCount}\tIterations:\t{iterations}\tAverage:\t{searchAverage.Average:0}\tsearches");
        Log($"{type}\tWrites\tDataCount\t{dataCount}\tIterations:\t{iterations}\tAverage:\t{writeAverage.Average:0}\twrites");
    }
    catch (Exception ex)
    {
        Log($"{type}\tFail\tDataCount\t{dataCount}\tIterations:\t{iterations}\tFailed\t{ex.Message}");
    }
}

private WorkerResult CreateThread(DateTime start, Action action )
{
    var work = new WorkerResult();
    work.Thread = new Thread((context =>
    {
        try
        {
            while (DateTime.UtcNow < start)
                Thread.Sleep(0);
            var sw = Stopwatch.StartNew();
            while (sw.ElapsedMilliseconds < 50)
            {
                action();
                work.Increment();
            }
        }
        catch (Exception ex)
        {
            work.Exception = ex;
        }
    }))
    {
        IsBackground = true
    };
    return work;
}

private class WorkerResult
{
    public Thread Thread { get; set; }
    public Exception Exception { get; set; }
    public float WorkIterations => _workIterations;
    private float _workIterations;

    public void Increment()
    {
        _workIterations++;
    }

    public bool Verify(string type, Action<string> logger)
    {
        if (Exception != null)
        {
            logger($"{type}\tFail\tDataCount\t-\tIterations:\t-\tFailed\t{Exception.Message}");
            return false;
        }
        return true;
    }
}

Results

This post only contains the summary..
If you are interested in all the timings, I've posted the table in a previous post.

Starting with the single threaded results





For the 16 items test, the ConcurrentDictionary clearly is a looser. The other strategies are at the same level and for this small number of items it seems to not be so important what strategy to chose.






So not much surprising here, the ConcurrentDictionary lost the search category and unsurprisingly the for the cause optimized lookup was the fastest that became clear with a little more data.
The Add and Get categories was won by the unsafe lookup, not really interesting for this test but
good for a baseline. The ConcurrentDictionary seems to have lost most categories.

Multi-threaded results

The unsafe did not make it here, due to the 'Collection was modified; enumeration operation may not execute' exception. The main reason we use locks in a multi-threaded environment ;)
So here we only look at 4 different contestants.



The full lock strategy seems to fail here, probably as they all lock out each other. I.e. a long duration search locks out other readers. The switch to full lock with a lookup dictionary instead of a linq query won the searches count by far, but that tends to be the case most times when writing a non-generic bit of code to optimize your scenario. So no surprise there.
The ConcurrentDictionary actually holds its place quite well, if you use it as a dictionary only. Doing searches in it seems to go very slow.

Conclusions

The numbers seem to jump a little and the conclusion I can draw is that if performance is key to your success, do your own performance tests for your exact scenario. For the general case, I think that I'll stick with the Full-Lock and extra indexes if needed strategy for now. The extra complexity needed to get a ReaderWriterLock is in my mind not worth the risk of me or someone else screwing it up, at least not for a 1 writer, 2 readers scenario.


All code provided as-is. This is copied from my own code-base, May need some additional programming to work. Hope this helps someone out there :)



Saturday, March 16, 2013

DateTime.Now vs DateTime.UtcNow

Please note that this post does not take into account any other perspective of when to use DateTime.Now or DateTime.UtcNow other than performance.

This morning I tried to find numbers that compare DateTime.Now and DateTime.UtcNow but all I found was long discussions on the subject of when to use them. So I decided to write a quick sample myself to see.

Last value was 2013-03-16 12:50:18
DateTime.Now:           00:00:06.6733481 for 10000000 iterations
Last value was 2013-03-16 11:50:18
DateTime.UtcNow:        00:00:00.0741262 for 10000000 iterations
DateTime.UtcNow is 1,11% of DateTime.Now




   1:  class Program
   2:  {
   3:      static void Main(string[] args)
   4:      {
   5:          const int iterations = 10000000;
   6:          DateTime dt = DateTime.UtcNow;
   7:          Stopwatch sw = Stopwatch.StartNew();
   8:          for (int i = 0; i < iterations; i++)
   9:              dt = DateTime.Now;
  10:          sw.Stop();
  11:          Console.WriteLine("Last value was {0}", dt);
  12:          Console.WriteLine("DateTime.Now:\t\t{0} for {1} iterations", sw.Elapsed, iterations);
  13:          double nowMs = sw.ElapsedMilliseconds;
  14:          sw = Stopwatch.StartNew();
  15:          for (int i = 0; i < iterations; i++)
  16:              dt = DateTime.UtcNow;
  17:          sw.Stop();
  18:          Console.WriteLine("Last value was {0}", dt);
  19:          Console.WriteLine("DateTime.UtcNow:\t{0} for {1} iterations", sw.Elapsed, iterations);
  20:          double utcNowMs = sw.ElapsedMilliseconds;
  21:          Console.WriteLine("DateTime.UtcNow is {0:0.00}% of DateTime.Now", CalcPercentage(utcNowMs, nowMs));
  22:          Console.ReadLine();
  23:      }
  24:      private static double CalcPercentage(double value, double max)
  25:      {
  26:          double p = (100.0 / max);
  27:          return p * value;
  28:      }
  29:  }