A few months back, I wrote an article which got a bit of interest around the issues configuring and maintaining multiple clusters, and keeping the components required to make them useful in sync. Essentially, the missing piece of the puzzle was that there was no cluster aware configuration management tool.
Internally, we created an excellent tool at
$work to solve this using jsonnet, which has been very nice because it’s meant we get to use actual code to solve this problem. The only issue is that the code we have to write in is relatively niche!
When Pulumi was announced back in June, I got very excited. After seeing the that Pulumi now supports Kubernetes natively, I wanted to dig in and see if it would help with the configuration management problem.
Before I speak about the Kubernete–specific components, I want to do a brief introduction into what Pulumi actually is.
Pulumi is a tool which allows you to create cloud resources using real programming languages. In my personal opinion, it is essentially terraform with full programming languages on top of it, instead of HCL.
Pulumi uses the concept of a stack to segregate things like environments or feature branches. For example, you may have a stack for your development environment and one for your production environment.
These stacks store their state in a similar fashion to terraform. You can either store the state in:
- The public pulumi web backend
- locally on the filesystem
There is also an enterprise hosted stack storage provider. Hopefully it’ll be possible to have an open source stack storage some time soon.
Components allow you to share reusable modules, in a similar manner to terraform modules. This means you can write reusable code and create boilerplate for regularly reused resources, like S3 buckets.
Finally, there’s configuration ability within Pulumi stacks and components. What this allows you to do is differentiation configuration in the code depending on the stack you’re using. You specify these configuration values like so:
pulumi config set name my_name
And then you can reference that within your code!
Kubernetes Configuration Management
If you remember back to my previous post, the issue I was trying to solve was being able to install components that every cluster needs (as an example, an ingress controller) to all the clusters but with often differing configuration values (for example, the path to a certificate arn in AWS ACM). Helm helps with this in that it allows you to specify values when installing, but then managing, maintaining and storing those values for each cluster becomes difficult, and applying them also becomes hard.
There are two main reasons Pulumi is helping here. Firstly, it allows you to differentiate kubernetes clusters within stacks. As an example, let’s say I have two clusters - one in GKE and one in Digital Ocean. Here you can see them in my kubernetes contexts:
I can initiate a stack for each of these clusters with Pulumi, like so:
Obviously, you’d repeat the stack and config steps for each cluster!
Now, if I want to actually deploy something to the stack, I need to write some code. If you’re not familiar with typescript (I wasn’t, until I wrote this) you’ll need to generate a
package.json and a
Fortunately, Pulumi automates this for us!
If you’re doing a Kubernetes thing, you’ll need to select kubernetes-typescript from the template prompt.
Now we’re ready to finally write some code!
Deploying a standard Kubernetes resource
When you ran
pulumi new, you got an
index.ts file. In it, it imports pulumi:
You can now write standard typescript and generate Kubernetes resources. Here’s an example nginx pod as a deployment:
Here you can already see the power that writing true code has - defining a constant for defaults and allowing us to use those values in the declaration of the resource means less duplicated code and less copy/pasting.
The real power comes when using the config options we set earlier. Assuming we have two Pulumi stacks,
and these stacks are mapped to different contexts, like so:
you can now also set different configuration options and keys which can be used in the code.
This will write out these values into a
Pulumi.<stackname>.yaml in the project directory:
and we can now use this in the code very easily:
Now, use Pulumi from your project’s root to see what would happen:
Obviously you can specify whicever stack you need as required!
Let’s verify what happened…
Okay, so as you can see here, I’ve deployed an nginx deployment to two different clusters, with two different images with very little effort and energy. Awesome!
Going further - Helm
What makes this really really awesome is that Pulumi already supports Helm charts!
In my previous post, I made the comment that Helm has lots of community supported charts which have done a whole load of configuration for you. However, Helm suffers from 2 main problems (in my humble opinion)
- Templating yaml with Go templates is extremely painful when doing complex tasks
- The helm chart community can be slow to merge pull requests. This means if you find an issue with a chart, you unfortunately have to fork it, and host it yourself.
Pulumi really helps and solves this problem. Let me show you how!
First, let’s create a Pulumi component using a helm chart:
And now preview what would happen on this stack:
As you can see, we’re generating the resources automatically because Pulumi renders the helm chart for us, and then creates the resources, which really is very awesome.
However, it gets more awesome when you see there’s a callback called
transformations. This allows you to patch and manipulate the generated resources! For example:
Of course, you can combine this with the configuration options from before as well, so you can override these as needed.
I think it’s worth emphasising that this improves the implementation time for any Kubernetes deployed resource dramatically. Before, if you wanted to use something other than helm to deploy something, you had to write it from scratch. Pulumi’s ability to import and them manipulate rendered helm charts is a massive massive win for the operator and the community!
I think Pulumi is going to change the way we deploy things to Kubernetes, but it’s also definitely in the running to solve the configuration management problem. Writing resources in code is much much better than writing yaml or even jsonnet, and having the ability to be flexible with your deployments and manifests using the Pulumi concepts is really exciting.
I’ve put the code examples from the blog post in a github repo for people to look at and improve. I really hope people try out Pulumi!