Container technology: The revolution in modern IT infrastructure
Container technologies such as Docker, DDev, Podman, Kubernetes and co are on everyone's lips and have become an integral part of IT. The reasons clearly speak for themselves. But some questions remain: What are containers? What is the motivation? What do I use containers for?
Admittedly, we are often asked such questions and that is absolutely understandable. The technology behind the concepts of containers is almost a science in itself. In the modern IT infrastructure, container technology plays a crucial role in the provision, management and scaling of applications. By using containers, software solutions can be executed in an isolated, lightweight and platform-independent manner, which offers both developers and companies numerous advantages. Compared to conventional virtualization approaches, containers enable more efficient use of resources and faster deployment cycles. Technologies such as Docker and Kubernetes have established themselves as the standard in this area and are driving automation and cloud-native development significantly forward.
Initial container technologies
As early as 1979, very simple process isolations were already possible using chroot on UNIX V7 systems. However, the process isolations of 1979 are not comparable with the isolations of today's container technologies such as Docker, Podman and co. In 2000, FreeBSD Jail appeared as the first technology for containers. With FreeBSD Jail, resources could already be virtualized and network interfaces could be equipped with separate IP addresses.
In the following years, various other pieces of software appeared that followed a similar approach. LXC was created in 2008 through an initiative by Google with the aim of developing a technology for limiting the resources of individual virtual instances, which was integrated into the Linux kernel. With the then innovative interfaces provided by LXC, developers with various programming languages such as Python, Ruby and Java were able to extend LXC to meet their needs, which ultimately led to the great popularity of LXC. In March 2013, Docker was released on the basis of LXC. With the founding of the Open Container Initiative (OCI) in 2015, which had set itself the goal of creating an open standard for handling and developing containers, the focus of Docker shifted away from LXC and the libcontainers library was developed. This is compatible with all common operating systems and container runtimes.
More about LXC: LXC
More about Docker: Docker (Software)
What is the Open Container Initiative (OCI)?
The Open Container Initiative (OCI) is an open, vendor-neutral organization that defines standards for container formats and runtimes. It was founded in 2015 by the Linux Foundation to promote standardized and interoperable container technology.
Why was the OCI founded?
Before the OCI, there were various container formats that were not always compatible with each other. Companies such as Docker, CoreOS and Red Hat developed their own standards, which led to problems with portability and interoperability. The OCI was created to establish uniform and open standards for containers and thus ensure compatibility between different platforms and runtimes.
What are Container?
Containers are basically isolated runtime environments for software, which have the advantage that they can be executed seamlessly on all common operating systems and are significantly smaller than conventional virtual machines. They bundle all the dependencies required for the software (libraries, configurations, environment variables) so that the software can be executed reliably regardless of the underlying infrastructure.
In contrast to virtual machines (VMs), containers do not require their own operating system, but share the kernel of the host system. As a result, they are more efficient in terms of memory and CPU usage and start much faster.
How does a container work?
The basis of a container is the technology for process insulation. Various technologies can be used for this purpose:
- Namespaces: Separate the container processes from each other so that they do not see any information about other containers or the host.
- cgroups (control groups): Limit and manage resource usage (CPU, RAM, network) for containers.
- Union Filesystems: Allow efficient storage of container images in multiple layers, saving disk space.
These mechanisms make it possible for each container to act like an independent small “virtual machine”, but without the overhead of a complete operating system.
Motivation
The basic motivation behind container technologies was already made possible in 1979 with simple mechanisms, the chroot. The idea or desire was to run processes in a separate file system environment without other processes being able to interact with them. The aim was clearly to create process isolations and improve security. This concept gave rise to Linux namespaces and control groups (cgroups) in the 2000s, which isolate processes more efficiently and improve resource utilization. These techniques form the basis of today's container technologies.
Over time, further motivations to rely on container technologies became established.
"It works on my machine"
Sayings like these represent a fundamental problem in the collaboration and release management of software. With virtual machines, developers have the possibility to configure their environments as they need them. It can easily happen that someone has installed a different version of an important library and the software no longer works properly.
In order to prevent such situations where different operating systems, library versions or other dependencies actively influence development, there was a desire for cross-system consistency. Today's containers make it easy to create standardized environments that contain all required dependencies, such as certain libraries or similar, in an encapsulated form, making them executable everywhere.
Efficiency compared to virtual machines (VMs)
It is often argued at this point that this could also be made possible with corresponding virtual machines, such as Virtualbox, VMWare, Vagrant and co. This is also correct as far as it goes. However, in contrast to a classic VM, containers significantly reduce the use of resources on the operating system. While virtual machines like to map the resources multiple times in order to isolate themselves accordingly, a container works much more “lightweight”. A virtual machine always requires an operating system with a functioning kernel in order to function in general. A container translates the corresponding sys calls via container runtime to the kernel of the host operating system and therefore does not require its own operating system. This means that the container and host share the same kernel.
This means that the container generates significantly less resource overhead (CPU, RAM) and works much more efficiently. Due to the lower resource overhead and the implementation of the containers, several containers can be operated easily and efficiently on the same physical hardware and scaled accordingly.
Thanks to the smaller size, the container can start in a few seconds, depending on the configuration and image. A VM usually takes minutes. Such fast iteration speeds are extremely important for operations with variable scaling.
Scalability and automation
With the emergence of cloud computing, the need arose to be able to scale applications dynamically. Thanks to the enormous iteration speed and efficient resource management of containers, containers have established themselves as the medium for scaled operation of applications. In 2015, Google launched the container orchestration tool Kubernetes. Thanks to its approaches, Kubernetes offers the perfect opportunity to operate and automatically manage applications on a large scale.
In the years that followed, there were various “tinkering” approaches to running VMs in a Kubernetes cluster. However, these did not catch on.
Thanks to the enormous potential for automation, various approaches were developed to optimize the release cycle and provide dedicated development statuses. Such concepts can be seamlessly integrated into a CI/CD pipeline.
What can containers be used for?
Container technology has spread to all areas of IT and is now an indispensable tool in modern software development. Thanks to their efficiency, portability and scalability, containers can be run on any hardware.
Software development & testing
In addition to the provision of standardized container images for developers, the same container images can also be started up with the equipped software as a test system. This means that the development environments and test environments are almost identical. Errors within the runtime or the same runtime can be identified and rectified more quickly and efficiently. Such standardized procedures reduce phrases such as “It works on my machine” to a minimum.
Scaling
Monolithic software architectures often have the problem that they are difficult to scale and maintain. It is often argued here that you can also create a “scaled” environment using 3 VMs with a load balancer in front of them. Well, this is partly true. Monolithic software within VMs cannot be scaled and operated as elegantly and cost-efficiently as microservice architectures in container environments. While in a monolithic architecture the entire software is scaled, including the software parts that may not be needed, in a microservice architecture only the software parts can be scaled that need it at that moment, e.g. due to high load. Manual deployment and scaling can often lead to errors. Container orchestration tools such as Kubernetes can reduce the error rate and automate deployment and scaling with the built-in automatisms.
Edge Computing & IoT
Physical hardware usually has limited computing power available to run compute-intensive software on the hardware, but so-called clusters can be provided using orchestration tools such as Kubernetes. This means that all types of software can be executed on any hardware. There are no limits here, see: www.golem.de/news/militaer-us-air-force-nutzt-kubernetes-fuer-neuen-b-21-stealth-bomber-2006-148891.html and here: The James Webb Space Telescope - Success through Redundancy.
Big Data & Artificial Intelligence (AI/ML)
Regardless of whether artificial intelligence or big data, both handle enormous amounts of data, sometimes in the terrabyte range, and must be efficient. This is where a flexible and scalable environment comes in handy. Container technologies and container orchestration tools such as Kubernetes offer precisely such environments. In addition, data analysis pipelines and AI models can be executed in an isolated and reproducible manner, which greatly improves the validation of the data.
Conclusion
IT infrastructure. Their development began with simple process isolations and has evolved into highly developed orchestration solutions such as Kubernetes. The advantages are obvious: containers offer an efficient, platform-independent and scalable way to deploy and manage applications.
By reducing resource overhead compared to virtual machines, containers enable optimized use of hardware and faster deployment of software. This not only improves development and testing, but also facilitates the operation and automation of large applications in the cloud or on edge devices.
Thanks to open standards, as defined by the Open Container Initiative (OCI), containers can be used across platforms and ensure consistency between development, test and production environments.
Whether for microservices, cloud computing, edge computing or AI applications - containers are the basis for a future-proof, flexible and automated IT landscape. Their influence will continue to grow as they help companies to work more efficiently, securely and agilely.