Back to Certified Kubernetes Administrator (CKA) Tutorial
note: Includes the following additional topics:
-
Networking: Understand the networking configuration on the cluster nodes
-
Troubleshoot: Troubleshoot Application Failure
-
Troubleshoot: Troubleshoot Networking
Addendum: Failed to show that the iptables-save command could be used to inspect the iptables that are managed by kube-proxy.
Search ipc to find Share Process Namespace between Containers in a Pod.
Everything a containerized application writes to stdout and stderr is handled and redirected somewhere by a container engine.
-Kubernetes-Logging Architecture
A simple example:
helm install dev debug
kubectl logs example-dev
note:: --container flag to specify container. --previous flag used to look at logs from previous container in case it crashes.
However, the native functionality provided by a container engine or runtime is usually not enough for a complete logging solution. For example, if a container crashes, a pod is evicted, or a node dies, you’ll usually still want to access your application’s logs. As such, logs should have a separate storage and lifecycle independent of nodes, pods, or containers. This concept is called cluster-level-logging. Cluster-level logging requires a separate backend to store, analyze, and query logs. Kubernetes provides no native storage solution for log data, but you can integrate many existing logging solutions into your Kubernetes cluster.
-Kubernetes-Logging Architecture
For EKS, see... Setting Up Container Insights on Amazon EKS and Kubernetes
Each Pod is assigned a unique IP address for each address family. Every container in a Pod shares the network namespace, including the IP address and network ports. Containers inside a Pod can communicate with one another using localhost. When containers in a Pod communicate with entities outside the Pod, they must coordinate how they use the shared network resources (such as ports).
-Kubernetes-Pod Overview
List of ports to expose from the container. Exposing a port here gives the system additional information about the network connections a container uses, but is primarily informational. Not specifying a port here DOES NOT prevent that port from being exposed. Any port which is listening on the default "0.0.0.0" address inside a container will be accessible from the network.
-Kubernetes-Reference
A simple example:
helm install dev networking
Follow logs:
kubectl logs example-dev --container httpd --follow
Login into httpd container:
kubectl exec example-dev -it --container httpd -- bash
View web page:
apt-get update
apt-get install curl -y
curl localhost
Observe source:
cat htdocs/index.html
Sidebar into logs: notice logs folder empty.
ls logs
Notice logs being sent to process' STDOUT:
grep CustomLog conf/httpd.conf
Can write directly to STDOUT (process id 1):
echo hello >> /proc/1/fd/1
Can also signal the process:
kill -HUP 1
Login into ubuntu container:
kubectl exec example-dev -it --container ubuntu -- bash
View web page:
apt-get update
apt-get install curl
curl localhost
Sidebar showing process on id 1:
ps -aux
Notice IP address.
kubectl describe pod example-dev
helm install another tester
kubectl exec example-another -it -- bash
apt-get update apt-get install curl -y curl XXX.XXX.XXX.XXX
How does does the traffic from this Pod (maybe on another Node) to this IP address make its way to the other Pod?
Login to Node and curl address of Pod.
How is traffic to this IP address getting to Pod?
First, notice that this IP address is allocated from the VPC subnet of the Nodes. This is because of the AWS Pod Networking add-on.
Next, look at AWS on Node EC2 and notice secondary IP address. This means that Node will listen on this IP address.
ifconfig
route
Let us change things up and look at another way to communicate between containers.
This page shows how to configure process namespace sharing for a pod. When process namespace sharing is enabled, processes in a container are visible to all other containers in that pod. You can use this feature to configure cooperating containers, such as a log handler sidecar container, or to troubleshoot container images that don’t include debugging utilities like a shell.
-Kubernetes-Share Process Namespace between Containers in a Pod
helm install dev ipc
Look at logs:
kubectl logs example-dev --container httpd --follow
kubectl exec example-dev --container ubuntu -it -- bash
ps -aux
kill -HUP XX