Skip to main content

Containers - Quick Low Level Guide

Containers Kernel, namespace, cgroups
Kernel space and user space

Before we actually get to explain containers let's define what is a kernel.  Because you know there is no such thing in reality as a kernel it's only how we name things, and different people name things differently.

cgroups, namespaces, UFS

We are going to discuss containers, cgroups, namespace, UFS, hypervisor, user space, kernetl space and more.
When we say "kernel" we mean this.  We have the hardware, this is not the kernel, now above the hardware we have a few layers of software, imagine now two boxes.

User mode is all the application you run while the kernel is the lower level is all the virtual memory management scheduling, connection to hardware devices, network drivers, it's basically the abstraction on top of the hardware + the basic services which allow this.

One box is closer to the hardware and contains a few layers, the second box sits on top of the kernel box and contains libraries we already use the lower box.

The lower box is the kernel it contains all the hardware abstraction layer, all the thread shielding the interrupts and many other lower level services such as memory management, power management etc.  The kernel is like the bare bones of the hardware plus all the operating system which manages this hardware.

The layer which connects the User space to the kernel space is system calls from user space to kernel space.

The Promise

Containers are built from cgroups, namespaces, tarballs, and they are just processes.  Let's elaborate on this.

The promise is that you can package your application in a container and then just run it in production, because all would be bundled inside the container, while we all know that is not really the truth containers do help us however if you would like to understand a little under the hoods or what containers really are let's start our discovery.  And while containers do contain processes it's best practice to use them for stateless applications due to their simple nature all we want to do is to contain processes not file systems, they are after all ephemeral.


If you look at a machine running containers and list the processes with PS you would not see any containers you would actually see real processes like java processes etc.  So what is going on, where are all those containers because those would appear to be super standard boring processes with the standard process output from the ps command.

So containers run as standard processes, what is the packaging that we all talk about? when you create a container you create a tarball file, so if you wanted to package a container manually you would zip your app in a tarball.

But tarball is that all there is to containers? I could tarball a process by myself, you could but you would need to take a few more steps for it to be equal to standard container for example you would need to update cgroups which we would soon discuss controlling the container better control how much CPU allocation it gets and how much memory.

When you run a container the tarball is being unzipped and run as any process.

As you want your container to have limited access for example to disk then there is a native way to do this on Linux, it's with the standard namespaces feature, it would allow containers to see certain processes and libraries.

The cgroups would limit the memory and CPU. 

VM Vs Containers

With vms you have multiple kernels but how do these multiple vm multiple kernels talk to the actual hardware and get all the basic thread scheduling services? That is with the hypervisor, so you have a kernel for each vm and each vm talks to the kernel of the hypervisor to get the basic OS services.

You can also load multiple OS like windows and linux, but in containers you have processes, each process runs inside a container.


Each container is a ting up together cgroup namespaces and UFS nifon capable file system this is a container, it looks like an OS the container

Namespace isolates and limits what you can use separate pid net mnt you have your own view of the system.
cgorups - how much you can use memory CPU

Processes from one namespace pid like if you have myapp1 namespace pid it cannot see any pid from myapp2 PID namespace

The access to resources is controlled by the cgroups.

UFS - combine multiple directories into one you get the illusion that you have a single directory.


Popular posts from this blog

Dev OnCall Patterns

Introduction Being On-Call is not easy. So does writing software. Being On-Call is not just a magic solution, anyone who has been On-Call can tell you that, it's a stressful, you could be woken up at the middle of the night, and be undress stress, there are way's to mitigate that. White having software developers as On-Calls has its benefits, in order to preserve the benefits you should take special measurements in order to mitigate the stress and lack of sleep missing work-life balance that comes along with it. Many software developers can tell you that even if they were not being contacted the thought of being available 24/7 had its toll on them. But on the contrary a software developer who is an On-Call's gains many insights into troubleshooting, responsibility and deeper understanding of the code that he and his peers wrote. Being an On-Call all has become a natural part of software development. Please note I do not call software development software engineering b

SQL Window functions (OVER, PARTITION_BY, ...)

Introduction When you run an SQL Query you select rows, but what if you want to have a summary per multiple rows, for example you want to get the top basketball for each country, in this case we don't only group by country, but we want also to get the top player for each of the country.  This means we want to group by country and then select the first player.  In standard SQL we do this with joining with same table, but we could also use partition by and windowing functions. For each row the window function is computed across the rows that fall into the same partition as the current row.  Window functions are permitted only in the  SELECT  list and the  ORDER BY  clause of the query They are forbidden elsewhere, such as in  GROUP BY ,  HAVING  and  WHERE  clauses. This is because they logically execute after the processing of those clauses Over, Partition By So in order to do a window we need this input: - How do we want to group the data which windows do we want to have? so  def c

Building Secure and Reliable Systems

A recent book was published this year by Google about site reliability and security engineering, I would like to provide you a brief overview of it and incorporate my own analysis and thoughts about this subject while saving you some time from reading, at least part of it. Take a few of your customers and ask them, what are the top 5 features on my product that you like.  The answer that you are likely to get is, I really like how polished the UI is, or the daily report I get by mail is just fantastic, or since I started using your product I was able to save one hour a day my productivity got up and the share /chat button on document that you added recently is doing a great job. Your customers are very unlikely to answer the question of what top 5 features of my product do you like with I really like its security or I really like that we lost no chat messages since I started using it.  No real customer will even think of it, moreover, assuming you did a very good job, they won&#