Understanding Mutable vs Immutable Infrastructure

The world of computer technology, especially the space that governs modern internet applications, has evolved rapidly. The journey to the cloud and compute-level virtualisation in the last decade has advanced the way in which operational management of compute resources are handled.

Obviously hardware will always exist but (mostly) gone are the days of enterprise tech staff minting servers, running them down to a private data centre, racking them and configuring physical networks for access. This concept has not disappeared, as hardware existing in data centres will always be necessary, it has just become more specialised.  

As a relatively old school operational person, I don't miss those days. The sheer overhead of the tasks involved certainly didn't lend itself to productivity, especially if the objective was to simply have an application accessible on the internet.

Want to upgrade a server? Buy a new one, mint it with an OS, configure it in house, pack it in the van, drive it down to the data centre (sign in with all the identification needed), rack it up (don't forget your screw drivers at the office!) and plug in the network cable (hold thumbs that your network configurations work).

Now think about doing all of this at scale. With all the intricacies of operating system, application and data requirements (which change over time), compute performance, monitoring, network access and the need for precise and exact configurations, one could very quickly get disheartened, question your life choices and wish you had kept that job as a beach barman.

Those less sophisticated times, where things moved slightly slower than today, could be understood to be truly mutable, where infrastructure component replacement was slow, cumbersome, prone to failure, risky, one dimensional and most significant of all, peripheral changes were depressingly inevitable.

Getting a Grip of the Definitions

Understanding the definitions of mutable and immutable in the context of compute infrastructure is important since it is not intuitive and glaringly apparent at first glance.

The adjective "Mutable", has the meaning of "liable to change". In other words anything that is mutable is likely to change. In turn, the adjective "Immutable" has the meaning of "unchanging over time or unable to be changed." The reverse understanding can then be applied to anything that is immutable, it will not or cannot change.

How do we apply these definitions in the context of compute infrastructure? The best way to understand it is to view the definition concepts as a desired end state of stable infrastructure.

The desired state of stable infrastructure should be current and error free. Any peripheral changes made to this desired state introduces risk, inconsistency and complexity, all of which can compromise stability and should be avoided.

But how does one avoid change? One needs to discern the correct action to take. In the context of modern compute infrastructure this means that the action of replacement should be valued higher than the action of peripheral change.

Change vs Replace

Continuing with the idea of definitions in an infrastructure context, to change something means to modify that existing thing and make it different in some way. To replace is to move something in place of something else.

With this in mind, mutable infrastructure means that existing components are modified. With immutable infrastructure, these components will never be modified, they will simply be replaced.

With the advancement of public cloud virtualisation and elasticity, the ability to run immutable infrastructure has increased exponentially across a host of application requirements.

One can consider immutable infrastructure, once configured, stable and in place never to change. If the existing infrastructure component is changed in any way, outside of predefined immutable boundaries, it is no longer immutable and can be considered compromised, resulting in a host of unanticipated, mostly negative results which in turn impact productivity, reliability and security.

Alternatively, replacement should be considered the desired action in an immutable infrastructure environment. This can apply to compute resources, application configurations and a host of other infrastructure related components.

The action of replacement can be done in very controlled fashion whereas changes often happen in a more risky, haphazard and ad-hoc manner. This makes the action of peripheral change undesirable.

Great. But how does all of this look in practice?

Immutable Infrastructure in Practice

The complexities of modern compute infrastructure can range from rudimentary to the mind boggling insane, depending on requirements. One could be running a simple web application on a single virtual machine instance servicing minimal requests or running an enterprise application requiring sophisticated containerised orchestration servicing millions of requests a second. In either case, and anything in between, the ideal would be to implement a strategy of immutable infrastructure.

To keep it simple, let's examine a more rudimentary example. The example will cover application configurations with an infrastructure-as-code approach and server replacement as an infrastructure upgrade approach.

Configuration Updates

The above graphic represents a simple infrastructure. The infrastructure details are that it is a single virtual machine in a cloud enabled environment running the Apache web server, serving a website. This infrastructure can be considered version one in terms of the compute resource and the server configuration.

A common scenario would be that the configuration for the Apache web server needs to be modified. If the initial configuration is considered version 1 of the configuration, the aim would be to implement version 2 with the required modifications.

A mutable approach would be to log onto the server, make the required modifications to the configuration and manually reload the Apache service. This action, most likely, could introduce error and inconsistency that, over time, will become problematic. Also, the simple action of logging onto a system with elevated privileges to make configuration changes establishes an insecure operational model.

The immutable method would be to replace the configuration in a controlled fashion using an infrastructure-as-code procedure. The advantage is many fold. The modification can be rolled out automatically and orderly across multiple environments, ensuring consistency. The modification will exist as a persistent record through source control. This enables a smoother rollback, if necessary and keeps a track on the change that has been implemented. Security is heightened in the sense that privileges are allocated to the procedure rather than an individual user.

Infrastructure Updates

What would an infrastructure update look like in the sense of a major update requirement? It's not uncommon that infrastructure update requirements arise relatively often. These update requirements can take many forms but the overall strategy of replace over change should be the preferred method. In our example, the requirement to update web server technology has arisen. This translates into replacing the Apache web server with an Nginx web server.

The mutable approach would be an attempt to decommission the Apache web server then install and configure the Nginx web server on the same virtual machine. The risk and complexity that this  action would introduce simply does not serve a method of infrastructure stability. How would one keep the application running and avoid disruption during the upgrade? How would we reliably test that the application is working as expected after an upgrade? How would we possibly manage this at scale with all the inconsistencies that would likely arise? What other unknown "noise" could appear that would compromise stability? All of these questions should deter one from this method and to rather implement one of immutability.

The immutable approach would be to replace the entire virtual machine in a controlled fashion. One would bring up and configure the version 2 of the infrastructure in parallel with version 1. This allows for a measured and orderly deployment of the new infrastructure that can be thoroughly tested to ensure the stability of the application. While the new infrastructure is deployed, configured and tested the old infrastructure is still available and serving the application to the end user. Once the new infrastructure is deemed stable, traffic is simply switched to the new infrastructure without end user awareness resulting in a smooth, beneficial transition. If a rollback is required, the traffic can be switched back. Once the new infrastructure is satisfactorily serving the application, the old infrastructure can be decommissioned in a controlled fashion.

Conclusion

Although the example we have examined is rudimentary, the overall approach of immutability can and should certainly be applied to more complex and sophisticated use cases. With the advancement of cloud elasticity, infrastructure-as-code, virtualisation and containerisation the options available to deploy immutable infrastructure are myriad.

That being said, deploying complex immutable infrastructure is ambitious, filled with challenge and demands a solid commitment from modern cloud engineers and organisations that want to maintain an advantage in the space of modern internet applications.