- complete this form https://docs.google.com/forms/d/1gaqJcHFy30RNTewF7v3WLZS_xuvK9t8WtPeiAAIFLVc/viewform
- join https://docker-in-depth.slack.com/ (I’ve sent you all invites)
- watch https://www.youtube.com/watch?v=LNAzzHS1Rho before Friday
- optionally watch https://www.youtube.com/watch?v=Q5POuMHxW-0
- linux kernel containerization mechanisms
- cgroups
- namespaces
- capabilities
- Docker’s image layering & the various subsystems implementing it
- container life-cyle
- what build-files are doing
- Docker’s networking / routing
create a team channel on https://docker-in-depth.slack.com
Example use cases:
- Util packing
- Dev tools in a box, host bind
- Demo shipping
- Production environments?
- Supervisor systems inside of related set of docker container
- focused on hands-on learning through exercises and exploration
- guided exploration rather than lectures and canned tutorials
curl -s https://get.docker.io/ubuntu/ | sh apt-get install linux-image-extra-`uname -r` cat /proc/filesystems | grep aufs
update-grub && reboot
vs explicit manual install from packages (lxc-docker, not docker.io)
- containerization is an old concept, Linux late to the game
- running a container in the foreground
- running it in the background and inspecting the logs
- running bash inside to inspect the env, network and file system
lxc-create -n test -t busybox
lxc-start -d -n test
lxc-info -n test
lscgroup | grep lxc
/var/lib/lxc/test/config
ls /var/lib/lxc/test/ # tree
# lxc-attach -n test
PID=$(lxc-info -p -n test | awk '{print $2}')
CGROUPS=$(lscgroup | awk -F: '/test/ {print $1}' | paste -sd,)
inside() {
cgexec -g "$CGROUPS:/lxc/test" -- \
nsenter --target $PID --mount --uts --ipc --net --pid -- "$@"
# hasn't dropped capabilities
# can be used to do things inside that are otherwise not allowed by lxc or docker
# such as additional mount points, routing changes etc.
}
inside cat /proc/self/cgroup
inside ls -al /proc/$PID/ns/
# other uses http://uwsgi-docs.readthedocs.org/en/latest/Namespaces.html
# exploring cgroups
tree -fd /sys/fs/cgroup/
tree -fd /sys/fs/cgroup/ | grep docker
tree -fd /sys/fs/cgroup/ | grep lxc
libcgroup / libcgroup-tools (ubuntu: cgroup-bin)
lssubsys
lscgroup
cgget
# to get list of cgroups docker is using
CGROUPS=$(lscgroup | awk -F: '/docker/ {print $1}' | uniq | paste -sd,)
cgexec -g "$CGROUPS:/docker/$CID"
# other tooling for working with cgroups:
#apt-get install python-dev
#pip install cgroup-utils
#https://pypi.python.org/pypi/cgroup-utils/0.5
# working with procs / cgroups
ps -O cgroup
ps -e -O cgroup
cat /proc/$PID/cgroup
alias psc='ps xawf -eo pid,user,cgroup,args'
apt-get install ruby ruby-dev libcap-dev
gem install cap2
ruby -e "require 'pp'; require 'cap2'; pp Cap2.process($PID).getcaps[:effective]"
… examine /usr/local/bin/dockhack and continue exploring with it
- docker -h
- exposing its API via -H
- running various storage backends on the same system
- explore -s drivers and fs layout
- run, stop/kill, start, restart, wait
- ps, inspect, top, logs, port
- attach
- diff, cp, commit
- rm
- images
- import
- tag
- inspect
- history
- save/load
- rmi
- build … covered later
- push
- pull
- search
some examples http://www.wouterdanes.net/2014/04/16/some-docker-tips-and-tricks.html
- shareable via mount binds
- some commands & arguments are subtly different
- http://docs.ansible.com/docker_module.html
- http://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.dockerio.html See http://thomason.io/automating-application-deployments-across-clouds-with-salt-and-docker/ for a great example of it in use.
- for your own internal use VS sharing with the world
- for your own use, bake all your common utils and config into the base
- minimize size & layering
- for external use, keep the base light or even better use a common 3rd party base image
- for your own use, bake all your common utils and config into the base
http://docs.docker.io/use/working_with_volumes/
https://github.com/discordianfish/docker-exporter/ https://github.com/discordianfish/docker-backup/ https://github.com/discordianfish/docker-lloyd/ backs up to s3 or use btrfs snapshots + send/receive
CID=short sha
TASKS=/sys/fs/cgroup/devices/docker/$CID*/tasks
PID=$(head -n 1 $TASKS) # use --format instead
mkdir -p /var/run/netns
ln -sf /proc/$PID/ns/net /var/run/netns/$CID
ip netns exec $CID ifconfig
ip netns exec $CID ip {link,addr,route} ...
ip netns exec $CID netstat -i
ip netns exec $CID tcpdump ...
ip netns exec $CID ss via -icc or custom bridges and lxc network options
same with links see diagrams in http://www.slideshare.net/hansode/hack-for-dockers-network
tcpdump -i docker0
- over gre tunnels
- or in ec2 with large mtu default on eth0 https://groups.google.com/forum/#!topic/docker-user/9klZLLPFG1I
http://goldmann.pl/blog/2014/01/21/connecting-docker-containers-on-multiple-hosts/ https://gist.github.com/noteed/8656989 https://communities.vmware.com/blogs/kevinbarrass for vmware examples that are also relevant http://networkstatic.net/open-vswitch-gre-tunnel-configuration/
http://tier.cs.berkeley.edu/drupal/howto/ip-tunnel-using-gre-on-linux http://tldp.org/HOWTO/Adv-Routing-HOWTO/lartc.tunnel.gre.html be aware of mtu size issue http://www.cisco.com/c/en/us/support/docs/ip/generic-routing-encapsulation-gre/13725-56.html
http://serverfault.com/questions/568736/expose-docker-containers-to-vpc-network
http://blog.docker.io/2013/09/docker-joyent-openvpn-bliss/ http://www.lartc.org/ https://wiki.archlinux.org/index.php/Advanced_Traffic_Control https://www.digitalocean.com/community/articles/how-to-use-iproute2-tools-to-manage-network-configuration-on-a-linux-vpshttps://groups.google.com/forum/m/#!topic/docker-dev/3paGTWD6xyw
wget https://raw.github.com/logentries/le/master/install/linux/logentries_install.sh && sudo bash logentries_install.sh https://logentries.com https://blog.logentries.com/2014/03/how-to-run-rsyslog-in-a-docker-container-for-logging/docker run –name logtest -d busybox sh -c “while true; do uptime; sleep 5; done” docker logs -f -t logtest | logger -t ‘docker:logtest’ docker run –name logtest2 busybox sh -c “while true; do uptime; sleep 5; done” | logger -t ‘docker:direct’
similar to lxc.kmsg
http://jasonwilder.com/blog/2014/03/17/docker-log-management-using-fluentd/- severely limited at the moment
- only container events are shown
- will not survive daemon restart
https://github.com/discordianfish/docker-spotter
graph_root=/lxc/docker
container_creations() {
docker events --since "1" | awk '/create/ {gsub(":",""); print $5; fflush()}'
}
log_creations() {
local config_file
while IFS= read -r cid; do
docker inspect "$cid" | jq -M -c '.[]'
done
}
watch_containers() {
container_creations | log_creations
}
http://blog.docker.io/2013/10/gathering-lxc-docker-containers-metrics/
counter args: http://phusion.github.io/baseimage-docker/
using runit http://tech.paulcz.net/2013/10/managing-docker-services-with-this-one-easy-trick.html http://docs.docker.io/examples/using_supervisord/ http://docs.docker.io/use/host_integration/ https://wiki.jenkins-ci.org/display/JENKINS/Docker+PluginFrom the authors of Fig https://github.com/orchardup/docker-jenkins/blob/master/Dockerfile
Blockade http://blockade.readthedocs.org for Testing Network Failures
If output is a single flat image:
- vs chroot
- vs lxc
- vs packer
If output is a diff or layered set of diffs against a base image:
- vs git, hg etc.
- vs btrfs or zfs snapshots + send/receive
- vs manually combining squashfs & unionfs http://tldp.org/HOWTO/SquashFS-HOWTO/creatingandusing.html
- mutable long-lived physical boxes or vms
- snow-flakes
- config managed
- mutable but short-lived vms
- immutable vms
- better packing and utilization
- remember it’s still not native performance.
- good usage of docker won’t add much extra overhead but you’re already virtualized
- good overview http://jasonwilder.com/blog/2014/02/04/service-discovery-in-the-cloud/
- DNS with skydock
- mention ZK
- etcd
- consul.io
- a la Airbnb’s SmartStack
- http://www.centurylinklabs.com/deploying-multi-server-docker-apps-with-ambassadors/
- http://www.centurylinklabs.com/linking-docker-containers-with-a-serf-ambassador/
- use host mount-bind volumes (see earlier in day)
saltstack in ha: same principles apply to docker+salt https://www.youtube.com/watch?v=R1bEEzwYeqk&app=desktop
- use what you know best
- salt
- ansible
- puppet
- chef
- etc.
- cpu -c –cpu-shares
- memory -m –memory
- disk IO
- network bandwith with tc
- reading material http://www.oracle.com/technetwork/articles/servers-storage-admin/resource-controllers-linux-1506602.html https://access.redhat.com/site/documentation/en-US/Red_Hat_Enterprise_Linux/6/html/Resource_Management_Guide/ch-Subsystems_and_Tunable_Parameters.html http://fritshoogland.wordpress.com/2012/12/15/throttling-io-with-linux/
- disk space quotas
- public cloud provider tools
- coreos.com
- https://github.com/boot2docker/boot2docker
- http://orchardup.github.io/fig/ (focused on dev envs)
- https://github.com/signalfuse/maestro-ng
ec2 elastic beanstalk http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_console.html
http://solum.io/ via openstack https://wiki.openstack.org/wiki/Docker
with mesos https://github.com/mesosphere/mesos-docker
- too early to really know about ‘best practices’:
- think critically, measure, test, and monitor
- what is your unit of deployment
- be wary of the less used backend drivers which haven’t had much battle testing yet
- e.g. lvm
- use device-mapper, btrfs or aufs
- use libcontainer instead of lxc execution engine
- pay attention to error reporting and handling. How does your infrastructure handle the unhappy path.
- keep an eye on the issues on github for bugs that may affect you
- try to develop your apps & systems so you can deploy to docker OR elsewhere. There is no need to lock yourself into docker-only approaches.
‘docker driver …’ command for driver backend commands such as resize-pool / resize