Dave Bechberger

Random Musings on Software and Development

.NET – A Viable Microservices Platform?

For the past 10+ years I have been doing full stack development on data intensive applications most of that time using the .NET stack.  For the past 18 -24 months I have been focused on building out next generation distributed data platforms using Java and Scala.  I am partial to .NET, but as of the last time I looked, 24 months ago, .NET was just not a viable platform to build highly distributed/microservices based applications on.  It was optimized for desktop or monolithic server applications and was lacking support for many of the technologies that make distributed/microservices a workable architecture (containerization, deployment management, simple configuration orchestration, distributed monitoring/logging, etc.).

I work in a primarily .NET shop architecting and developing our next generation data platform.  Due to the scale, complexity and types of data we work with we have decided to use microservices and event sourcing for the basis of our architecture.  The majority of the developers are experienced in .NET so we are trying to use .NET wherever reasonable to simplify the learning curve and provide higher maintainability.  My job is to figure out the best path to accomplish this and I was pleasantly surprised with how viable a microservices platform .NET has become since my last investigation.   

So what is it that makes .NET so much more viable now?  Well there are many different things but I am going to just touch on two of them here.

.NET Core/Standard

I am not going to go into tons of detail on .NET Core/Standard as there are tons of articles out there explaining all the details. (.NET Standard, .NET Core, am explanation of the differences)

TLDR;

.NET Core is a development platform released in June 2016 that enables cross-platform development of applications that can run on Windows, Linux and macOS.

In the old .NET, creating an application required creating a project, registering that project with IIS, then configuring IIS to properly run it.  Often times .NET versions differed between machines or web.config files worked on one machine and not another, or IIS was configured one way on machine A but not on machine B.  This process was the bane of many development teams and led to more frustration than probably anything else in the development process.

Using .NET Core to build out applications is vastly simplified, which simplifies the entire development process.  In .NET Core there is no longer the requirement to run inside IIS and each application can use a specified version of the runtime.  .NET core applications are configured by default to run in a self hosted mode which minimizes the development headaches across team members.  As an additional bonus I can build and run .NET Core applications using either Windows (Visual Studio or VS Code), Linux (VS Code), or Mac (Visual Studio for Mac or VS Code).

Docker Support

The addition of Docker support was a huge leap forward for .NET, not only from a technology standpoint but also from a mindset shift.  If you had told me 5 years ago that .NET would be open sourced, running on linux, and providing solid support for containerization such as Docker I would not have believed you, and I think you might be hard pressed to find many that would have.  Well times have changed and .NET now has solid support for containerization in Docker and Windows Containers.  Visual Studio also provides the slickest integration with Docker of any IDE I have used.  I am not sure of another IDE where adding Docker support is as easy as:

  1. Right clicking on a project
  2. Selecting “Add Docker Support”
  3. Clicking “Debug”

With those simple steps (Docker must be installed, instructions here) you can now spin up and debug your project inside a running Docker container on your system.  When you combine that with Visual Studio’s ability to start multiple projects it gives you a powerful ability to orchestrate the  debugging of multiple microservices.

Conclusion

Due to the community support and flexibility of the platform to be augmented by third parties, Java has been the go to platform of choice when building distributed/microservice systems.  While this is likely to remain true for the foreseeable future the recent changes to the .NET platform have given it the ability to start to be part of the conversation in that area.  While there are many other things to take under consideration (e.g. Drivers for 3rd party tools like Kafka tend to be behind, lacking, or non-existent) the open sourcing of the .NET framework gives me some hope that a robust ecosystem will spring up like exists for Java.   

Dipping my Toe in CosmosDB Graph API

For the inaugural post of my new blog I am going to discuss something else which is the brand new CosmosDB.  CosmosDB was a major announcement at Microsoft’s Build 2017 conference held last week.  For those of you who missed the annoncement you can find it here.  

TL;DR Summary on CosmosDB 

CosmosDB is the next generation of Azure’s DocumentDB which supports globally distributed multi-model data including Key/Value, Wide Row (Table), Document and Graph data models.  I am not going to repeat all the details here but if you want them I suggest you read this post.

My Interest

While there are a lot of new features included as part of CosmosDB (global replication , data partitioning , tunable consistency levels , …) that each are worthy of their own post but what really caught my eye was the CosmosDB’s Graph API.  This new feature provides support the Apache Tinkerpop Gremlin query language.  I was particularly interested in this because part of my current project at work has been evaluating Tinkerpop enabled graph databases for use in upcoming projects.  I currently work at a .NET development shop and in general the .NET drivers and support for the major graph databases lag behind that of the Java, NodeJS and Python counterparts.  With CosmosDB being a Microsoft project the .NET driver is really a first class citizen in the ecosystem and that makes it an intriguing prospect.  

First Impression

My first experience using CosmosDB’s Graph API was to setup the initial database using the Azure web portal.  As shown in their docs (click here)  create the initial graph was the sort of point and click experience that you would expect from a managed service.  A nice additional feature in the web portal was the ability to download either a customized .NET solution or a pre-configured version of the gremlin console with all the proper connection information configured for you.  I initially missed this and I struggled to get the gremlin console connecting to the graph due to my inability to figure out the correct username and password.  It was in the documentation provided online but I missed it.  In case you need it the docs on how to manually configure gremlin console are available here.

Second Step – Migrate a Real Use Case

Since my first experience was so painless I thought, why not take this a step further and try porting my current project over to use CosmosDB.  Since this application was a .NET Core project this the migration was a rather straightforward process that took me <1 hour.  I just replaced my current driver with the .NET CosmosDB driver from Nuget (currently in Preview so make sure to check that box) here.

Once the driver was installed there was a bit of coding required to migrate to the CosmosDB method of executing Gremlin traversals.  The changes required were minimal and were easily copied by following their sample project.  I was able to get it to compile and run.  Unfortunately I ran into a few hiccups with my traversals due to some gremlin features/steps that are not yet supported.  The list of currently supported steps is available here.

The two specific issues I ran into were:

  • Recurring.V() in a traversal are not currently supported such asg.V().has(‘titan’, ‘name’, ‘saturn’).as(‘s’).V().has(‘god’, ‘name’, ‘neptune’).as(‘n’).addE(‘father’).from(‘n’).to(‘s’) 

While this was an annoyance it only took a bit of reworking my traversal to get my insert queries working to add vertices and edges.

  • Subgraph steps are not currently supported.  This was a bit of a show stopper for me as my current project relies heavily on the use of subgraphs when retrieving data.  

I contacted the CosmosDB team about these issues and they quickly responded  that both of features are currently under development or on the near term roadmap.  

Using CosmosDB

In addition to the application drivers, you are provided with two additional options to interact with CosmosDB.  

The first is to use the gremlin console to connect to the remote gremlin server to send your traversals, but don’t forget to submit your command with :> as I did at first.  This provides you the ability to run all your gremlin traversals in your standard terminal window.

The second is to use the Data Explorer, which is a visualization tool that is built into the Azure Portal.  It provides a way to visually interact with your data, which I find very helpful when working with graph models.  

It does have a few quirks about it, specifically the default node layout can start out a bit strange.

In addition to that I was not able to figure out how to see edge properties.  With that said it is a really nice tool to help you out when trying to visualize your data.

I haven’t yet tried any of the other 3rd party tools to connect but I suspect/hope that as long as they work with the gremlin console that they would.

Summary

I think it is a really great addition to the graph database ecosystem to have another Gremlin enabled graph database on the market, especially one with strong support for .NET.   I am interested in taking some time to explore their different consistency models (read about them here and here ) as well as looking at the performance of the Graph API.  

As with other current offerings, CosmosDB has a few rough edges.  Given that it was only announced as a preview last week it is a strong initial offering worth taking notice of.   The fact that it is a newly released a globally distributed multi-model datastore supporting 4 different types of data model is a huge accomplishment that the entire CosmosDB team should be proud of.  

I will be watching to see where this goes to next.

Previous Blog Posts

If you would like to read any of my previous blog posts you can find them all at:

Graph Databases

Big Data

DevOps