Canary Deployments with Kubernetes and Containers

Continuous deployment is a main goal for many large developer teams that need more rapid code deployments of new features to an application. Container technology facilitates rapid code deployment and offers advantages that traditional production environments can’t offer. Not only can developers rapidly deploy code, but they can also test code in a production environment with little downtime (e.g., fewer reboot and process restart interruptions). Canary deployments implemented with continuous integration tools such as Kubernetes can ensure that applications are always available even during scheduled updates.

Canary Deployments vs. Blue/Green Deployments

The term “canary deployment” is a newer term, but its methodology is traditional. Traditionally, developers deploy applications to production and then log any issues to ensure quick bug fixes. Of course, promoting bugless code is the goal of every developer, but mistakes happen even after testing where an unforeseen bug is introduced into production.

In a traditional deployment, new code is deployed to all production machines and the assumption (or hope) is that no bugs crash the system, but bugs are common especially in a large codebase. These bugs could be low priority and don’t interfere with user experiences, or they could be severe revenue-impacting incidents. Any interruptions of service could result in a full rollback of the updated code as all production servers contain the newer version.

With canary deployments, developers deploy to a small subset of production machines, which only serves a subset of users who connect to the application. It’s not full A/B testing but the activity is similar. Only a small portion of your users get the new version of software, so developers can test a variety of features and user experiences while still having the old software version on other production environments.

Canary releases have several benefits:

• Identify user experience patterns and get feedback from user actions
• Test resources and performance of the application
• Capture metrics on the new software version to identify where changes should be made either on the environment or within the application code

Canary deployments are an incremental way to deliver software to users. It’s useful if you have several features that you want to test live with users. Let’s say that you have an application with one layout designed by UX staff. They aren’t sure if the new release will be better or worse for sales. By deploying to only a portion of container nodes, some users will get the old UX layout and others will get the newly updated one. Both applications run simultaneously and load balancers will direct a small segment of your users to the new version. Log user patterns and any errors and you can identify if users are more likely to buy products with the new UX layout or perhaps the new UX needs revisions to better increase revenue.

Canary deployments give you a few other benefits:

• Testing can be done in production. That’s right – in production. Only a portion of your containers have the new code, so not every customer will receive an error if there are errors, and you can roll back only these containers should a critical crash occur.
• Traffic sent to the canary release can be defined to limit risk of user frustrations and errors.
• A small amount of resources is used to run the test environment.
• Since multiple containers can be used, several different new features can be tested at the same time in production.

While there are several benefits to canary deployments, you should be aware of the risks as well. These include:

• Changing deployment procedures could lead to inefficiencies.
• Customized automation scripts might be needed to deal with feature releases.
• Tracking releases can be a pain if you have several versions of your software running in parallel.
• As several features are being tested at the same time, it could put a lot of overhead on developers who need to fix issues and bugs for quick release to production.
• Sticky sessions are essential as a client could hit a production server with one request and your testing environment in another.

How Kubernetes Can Help

Kubernetes is an orchestration tool used for continuous integration (CI) and delivery in a container environment. If you use a traditional environment and use continuous integration, you may be familiar with CI tools such as Jenkins CI or Travis CI. These tools automate testing and delivery of new code in a monolithic coding environment. Kubernetes is a CI tool for a container environment. If you decide to switch to containers running on virtual machines or even physical servers, Kubernetes can help reduce overhead and eliminate some of the disadvantages of traditional deployments.

You still need an official testing environment before sending your code to production. Although canary deployments let you test experiences and features in production, you still must test for bugs before pushing your code to production. Kubernetes can perform CI for you by promoting code to a testing and staging environment as well before deploying to production.

With Kubernetes, you can create “phases” by building a canary phase for your production test environment in the Kubernetes Service setup. With this strategy, you build a new controller for each version of your microservice and then deploy and provision containers for each deployment. Kubernetes tracks and lets you activate multiple versions of your software and control the resources used for each version. This strategy eliminates several of the disadvantages of canary deployments so that developers can stay more organized and reduce resource waste.

Kubernetes also provides a rollback strategy should any critical issue crash the program. With traditional code promotions, any environment variables and configurations must also be rolled back. This can be a stressful tedious issue for the developer in charge of fixing a failure using rollbacks. With Kubernetes, you can automate rollbacks so that failures don’t persist for long on any given container.

Conclusion

Canary deployments are a great way to incrementally promote code to test features in an A/B testing manner. Although you’re testing in production, you still can’t afford buggy code that could crash the application. Using Kubernetes, you can configure the orchestration tool to provision resources and deploy canary versions of your code in an efficient manner. Should you need a rollback, Kubernetes can perform a safe rollback as well. If you have software with several features and you want to test user satisfaction and experience, canary deployments, containers, and Kubernetes for automation is the strategy you’re looking for.