Work on my DuoVia.Net and DuoVia.MpiVisor projects has been progressing well. I now have an opportunity to use the libraries in a significant use case at Ancestry.com, my day job, which has been very useful in finding ways to improve the libraries. (Disclaimer: Ancestry.com does not endorse the DuoVia library and I am not a spokesman for Ancestry.)

In its first incarnation, the ProxyFactory created a new dynamic assembly each time it created a proxy. This was expensive in terms of creation time but more so in terms of memory once many thousands of proxies had been created. Assemblies are kept in memory for the life of the process.

First I tried a Dictionary of ProxyBuilder objects, the container class for the objects needed to create an instance of the dynamically generated assembly’s proxy type that implements the target interface. I used a lock on this Dictionary but of course that created a bottleneck and a many threaded application trying to create many proxy connections would run into that bottleneck.

Next I tried a ThreadStatic instance of that Dictionary, keeping a ProxyBuilder for each key type. This eliminated the bottle neck but necessitated the creation of many more ProxyBuilder objects than was necessary and no guarantee could be made that these objects would ever be utilized more than once. In a multithreaded client using thread pool threads or its own threads, over time, memory usage and performance would be negatively impacted.

Pooling was to the answer. But how can you pool objects of the same base type by key rather than type? There are many object pool examples to be found but all those that I found were based on type alone. Time to roll my own. And PooledDictionary<TKey, TValue> is what I came up with.

First the code that uses it so you can get a feel for how easy it is to use. Note that the Request method’s Func<TValue> parameter called CreateProxyBuilder. The CreateProxyBuilder is the costly and complex method that creates the dynamic assembly and collects the necessary objects into the ProxyBuilder object that will be required to create an instance of the proxy for the target interface. The function is used to create a new ProxyBuilder if the pool is depleted.

Using the PooledDictionary<TKey, TValue>

private static PooledDictionary<string, ProxyBuilder> _proxies = 
  new PooledDictionary<string, ProxyBuilder>();

public static TInterface CreateProxy<TInterface>(Type channelType, 
  Type ctorArgType, object channelCtorValue) where TInterface : class
{
  if (!channelType.InheritsFrom(typeof(Channel))) 
  {
    throw new ArgumentException("channelType does not inherit from Channel");
  }
  Type interfaceType = typeof(TInterface);
  var proxyName = interfaceType.FullName + channelType.FullName + ctorArgType.FullName;

  //get pooled proxy builder
  var localChannelType = channelType;
  var localCtorArgType = ctorArgType;
  ProxyBuilder proxyBuilder = _proxies.Request(proxyName, () => 
    CreateProxyBuilder(proxyName, interfaceType, localChannelType, localCtorArgType));

  //create proxy
  var proxy = CreateProxy<TInterface>(proxyBuilder, channelCtorValue);

  //return builder to the pool
  _proxies.Release(proxyName, proxyBuilder);

  return proxy;
}

And now the code that makes the magic happen. Note the use of the System.Collections.Concurrent namespace. While sometimes heavy, these collections really do have their place on the parallel programmer’s palette.

PooledDictionary<TKey, TValue>

public class PooledDictionary<TKey, TValue> 
{
  private readonly ConcurrentDictionary<TKey, ConcurrentQueue<TValue>> _dq;
  private readonly int _concurrencyLevel;
  private readonly int _size;

  public PooledDictionary()
  {
    _concurrencyLevel = Environment.ProcessorCount * 8;
    _size = _concurrencyLevel * _concurrencyLevel;
    _dq = new ConcurrentDictionary<TKey, ConcurrentQueue<TValue>>(_concurrencyLevel, _size);
  }

  public void Add(TKey key, TValue value)
  {
    if (!_dq.ContainsKey(key)) _dq.TryAdd(key, new ConcurrentQueue<TValue>());
    ConcurrentQueue<TValue> q;
    if (_dq.TryGetValue(key, out q))
    {
      q.Enqueue(value);
    }
    else
    {
      throw new ArgumentException("Unable to add value");
    }
  }

  public int Count(TKey key)
  {
    if (!_dq.ContainsKey(key)) _dq.TryAdd(key, new ConcurrentQueue<TValue>());
    ConcurrentQueue<TValue> q;
    if (_dq.TryGetValue(key, out q))
    {
      return q.Count;
    }
    return 0;
  }

  public TValue Request(TKey key, Func<TValue> creator = null)
  {
    if (!_dq.ContainsKey(key)) _dq.TryAdd(key, new ConcurrentQueue<TValue>());
    ConcurrentQueue<TValue> q;
    if (_dq.TryGetValue(key, out q))
    {
      TValue v;
      if (q.TryDequeue(out v)) return v;
      if (null != creator) return creator();
    }
    return default(TValue);
  }

  public void Release(TKey key, TValue value)
  {
    Add(key, value); //just adds it back to key's queue
  }
}

I hope this little gem is as useful to you as it has been to me. In tests and production, I have found it to be as fast or faster than the ThreadStatic collection approach. And memory consumption in production has returned to satisfactory levels because we are now creating only the number of ProxyBuilders that we need and using those efficiently.