14-C.4: Orchestration

Orchestration and Automation: What is the Difference?

For the longest time, it seemed the only thing any sysadmin cared about was automation. Recently, though, the mantra seems to have changed from automation to orchestration, leading many puzzled admins to wonder: "What's the difference?"

The difference between automation and orchestration is primarily in intent and tooling. Technically, automation can be considered a subset of orchestration. While orchestration suggests many moving parts, automation usually refers to a singular task or a small number of strongly related tasks. Orchestration works at a higher level and is expected to make decisions based on changing conditions and requirements.

However, this view shouldn't be taken too literally because both terms—automation and orchestration—do have implications when they're used. The results of both are functionally the same: things happen without your direct intervention. But the way you get to those results, and the tools you use to make them happen, are different, or at least the terms are used differently depending on what tools you've used.

For instance, automation usually involves scripting, often in Bash or Python or similar, and it often suggests scheduling something to happen at either a precise time or upon a specific event. However, orchestration often begins with an application that's purpose-built for a set of tasks that may happen irregularly, on demand, or as a result of any number of trigger events, and the exact results may even depend on a variety of conditions.

Decision Making and IT Orchestration

Automation suggests that a sysadmin has invented a system to cause a computer to do something that would normally have to be done manually. In automation, the sysadmin has already made most of the decisions on what needs to be done, and all the computer must do is execute a "recipe" of tasks.

Orchestration suggests that a sysadmin has set up a system to do something on its own based on a set of rules, parameters, and observations. In orchestration, the sysadmin knows the desired end result but leaves it up to the computer to decide what to do.

Consider Ansible and Bash. Bash is a popular shell and scripting language used by sysadmins to accomplish practically everything they do during a given workday. Automating with Bash is straightforward: Instead of typing commands into an interactive session, you type them into a text document and save the file as a shell script. Bash runs the shell script, executing each command in succession. There's room for some conditional decision-making, but usually, it's no more complex than simple if-then statements, each of which must be coded into the script.

Ansible, on the other hand, uses playbooks in which a sysadmin describes the desired state of the computer. It lists requirements that must be met before Ansible can consider the job done. When Ansible runs, it takes action based on the current state of the computer compared to the desired state, based on the computer's operating system, and so on. A playbook doesn't contain specific commands, instead leaving those decisions up to Ansible itself.

Of course, it's particularly revealing that Ansible is referred to as an automation—not an orchestration—tool. The difference can be subtle, and the terms definitely overlap.

Orchestration and the Cloud

Say you need to convert a file type that's regularly uploaded to your server by your users.

The manual solution would be to check a directory for uploaded content every morning, open the file, and then save it in a different format. This solution is slow, inefficient, and probably could happen only once every 24 hours because you're a busy person.

You could automate the task. Were you to do that, you might write a PHP or a Node.js script to detect when a file has been uploaded. The script would perform the conversion and send an alert or make a log entry to confirm the conversion was successful. You could improve the script over time to allow users to interact with the upload and conversion process.

Were you to orchestrate the process, you might instead start with an application. Your custom app would be designed to accept and convert files. You might run the application in a container on your cloud, and using OpenShift, you could launch additional instances of your app when the traffic or workload increases beyond a certain threshold.

Learning Automation and Orchestration

There isn't just one discipline for automation or orchestration. These are broad practices that are applied to many different tasks across many different industries. The first step to learning, though, is to become proficient with the technology you're meant to orchestrate and automate. It's difficult to orchestrate (safely) the scaling a series of web servers if you don't understand how a web server works, or what ports need to be open or closed, or what a port is. In practice, you may not be the person opening ports or configuring the server; you could be tasked with administrating OpenShift without really knowing or caring what's inside a container. But basic concepts are important because they broadly apply to usability, troubleshooting, and security.

Infrastructure as Code

Infrastructure as code (IaC) is the process of managing and provisioning computer data centers through machine-readable definition files, rather than physical hardware configuration or interactive configuration tools. The IT infrastructure managed by this process comprises both physical equipment, such as bare-metal servers, as well as virtual machines, and associated configuration resources. The definitions may be in a version control system. It can use either scripts or declarative definitions, rather than manual processes, but the term is more often used to promote declarative approaches.

Orchestration Tools

Tool	Description
Ansible	Ansible is a radically simple IT automation engine that automates cloud provisioning, configuration management, application deployment, intra-service orchestration, and many other IT needs. Designed for multi-tier deployments since day one, Ansible models your IT infrastructure by describing how all of your systems inter-relate, rather than just managing one system at a time.
Chef	Chef operates in a client-server architecture. An agent, the Chef Client, runs on each managed node, periodically connecting to a Chef Server to download and evaluate configuration code, known as recipes. If no changes are necessary, Chef does not modify the system.
Kubernetes	Kubernetes is a portable, extensible, open-source platform for managing containerized workloads and services that facilitates both declarative configuration and automation. It has a large, rapidly growing ecosystem. Kubernetes services, support, and tools are widely available. The name Kubernetes originates from Greek, meaning helmsman or pilot. Google open-sourced the Kubernetes project in 2014. Kubernetes combines over 15 years of Google's experience running production workloads at scale with best-of-breed ideas and practices from the community.
Nomad	Nomad is a simple, flexible, and easy to use workload orchestrator to deploy and manage containers and non-containerized applications across on-prem and clouds at scale. Nomad runs as a single binary with a small resource footprint (35MB) and is supported on macOS, Windows, Linux. Developers use declarative infrastructure-as-code (IaC) for deploying their applications and defining how an application should be deployed. Nomad automatically recovers applications from failures.
OpenStack	OpenStack is an open source platform that uses pooled virtual resources to build and manage private and public clouds. The tools that comprise the OpenStack platform, called "projects," handle the core cloud-computing services of compute, networking, storage, identity, and image services. More than a dozen optional projects can also be bundled together to create unique, deployable clouds.
OpenShift	Redhat offers OpenShift Container Platform as a Service (PaaS). It helps in the automation of applications on secure and scalable resources in hybrid cloud environments. It provides enterprise-grade platforms for building, deployment, and managing containerized applications. Built on Redhat enterprise Linux and Kubernetes engine, Openshift has various functionalities to manage clusters via UI and CLI.
Rancher	Rancher is an open-source platform that uses container orchestration known as cattle. It allows you to leverage orchestration services like Kubernetes, Swarm, or Mesos. Rancher provides the software required to manage containers so that organizations don’t need to build container services platforms from scratch using a distinct set of open source technologies.

Orchestration Configuration

With such a wide variety of tools there is not an easy description of how to configure orchestration. Each tool has its own syntax for the config files, and the various attributes that need to be configured. The best approach would be to take some time to research a short list of potential tools, and investigate those specific tools. Once a platform is selected, spend some time gaining training and/or experience with that tool before proceeding with an implementation.

Adapted from:
"What's the difference between orchestration and automation?" by Seth Kenlon, OpenSource.com is licensed under CC BY-SA 4.0
"Infrastructure as code" by Multiple Contributors, Wikipedia is licensed under CC BY-SA 3.0