Going cloud-native with Terraform, Consul and Nomad at the Dutch National Police

Microservices development is common practice for quite some time at the Cloud, Big data and Internet department of the Dutch National Police. But about a year ago we saw areas where we could improve.

At the time our microservices were running on VM’s that were purpose-built for each particular microservice. There was ad-hoc automation around creating these VM’s. Furthermore each microservice was deployed on a single instance and thus not high available (HA). We set out to change this, because we want 24/7 availability, zero-downtime deployments, high scalability, and a more self-service infrastructure.

In this talk we’ll share how we gradually migrated our dozens of microservices to a high available cloud-native container platform. The key here is ‘gradual migration’. We wanted to move fast but avoid a big bang change on both the technology and process side.

With the help of Hashicorp’s Terraform, Consul and Nomad plus a few inhouse developed components (in Go, Java and Typescript) we managed to address our concerns. You will leave this talk with an understanding how these tools provide value when used on their own, and when composed together.

We start by describing the situation a year ago: a large scale private cloud, with continuous delivery and microservices already being common practice.

Then we describe (in this order):

Why and how we used terraform to rebuild all our machines and offer self-service infrastructure. And the challenges we encountered and how we solved them.

Why and how used Consul for service discovery and high availability (HA).
And the challenges we encountered and how we solved them.

Why and how Nomad for zero-downtime deployments and scalability.
And the challenges we encountered and how we solved them.

We highlight how these tools can be used separately or combined. And we finish by looking into future work.