{"id":639,"date":"2018-09-16T21:31:37","date_gmt":"2018-09-16T19:31:37","guid":{"rendered":"http:\/\/dekarlab.de\/wp\/?p=639"},"modified":"2020-05-23T15:34:43","modified_gmt":"2020-05-23T13:34:43","slug":"book-notes-kubernetes-up-and-running-dive-into-the-future-of-infrastructure","status":"publish","type":"post","link":"https:\/\/dekarlab.de\/wp\/?p=639","title":{"rendered":"Book notes \u2013 Kubernetes: Up and Running: Dive into the Future of Infrastructure"},"content":{"rendered":"

You can buy this book from amazon.de<\/a>.
\n<\/p>\n

This book descirbes not only, how Kubernetes works, but also design principles and ideas behind it. Notes below are ideas from this book with my own understanding. Therefore it is recommended also to read this nice book.<\/p>\n

Containers should be immutable<\/strong>. That means, you should change them only during build process, not during runtime. Runtime changes only acceptable, if you would like quick recover running system, but these changes should be then integrated in build scripts.<\/p>\n

State handling is very important in Kubernetes. You define state of your systems, send this state to Kubernetes, and Kubernetes will apply this state to running system. For example, you need 2 parallel running containers instead of 1. You inform about this Kubernetes and it starts additional container for you. State is better to save in git, so anytime you are able to see in what state the system right now, and what state was before.<\/p>\n

Self-healing is one very attractive property. Kubernetes has possibility to perform automatic checks for health of the components. But it is clear, that development of components should be done with this idea in mind. In world of data applications, that means, components should be restartable, without the need of cleaning up the data storage.<\/p>\n

Define microservices independent from each other. They should share data only through API. This helps to reduce teams dependency and coordination. In world of data applications your microservice can have own domain model and should not share it with other applications, like it is done in data warehousing.<\/p>\n

Teams in Kubernetes world: Application, Cluster, Kernel, Hardware.<\/p>\n

Namespaces is very powerfull in Kubernetes. For example, if you have DEV\/TEST\/PROD environments, they can share same hardware, but can be separated on logical view using dedicated namespaces. For data intensive analytical applications this is a very nice feature, since for copying data from one to another environment and keeping it in sync is very time consuming task. And simply reusing data but in logically separated way will help to reduce development time.<\/p>\n

Layers in docker should be created from less likely changed layer to more often changed layers. Be careful, files, which are deleted on upper layers in any case occupy space.<\/p>\n

Secrets and images should never be mixed.<\/p>\n

Use registry for storing containers.<\/p>\n

Commands:
\nkubectl version
\nkubectl get componentstatuses
\nkubectl get nodes
\nkubectl describe nodes <\/strong><\/p>\n

Request vs limit for resources. Request is minimum needed, but can be more, limit is maximum allowed. CPU – easy to reduce, Memory, container will be killed.<\/p>\n

Proxy
\nfrom network to services
\nkubectl get daemonSets –namespace=kube-system kube-proxy<\/strong><\/p>\n

DNS
\nNames for services
\nkubectl get deployments –namespace=kube-system kube-dns<\/strong>
\nkubectl get services –namespace=kube-system kube-dns<\/strong>
\n\/etc\/resolv.conf<\/p>\n

UI
\nkubectl get deployments –namespace=kube-system kubernetes-dashboard<\/strong>
\nkubectl get services –namespace=kube-system kubernetes-dashboard<\/strong><\/p>\n

Cluster-IP vs Network IP<\/p>\n

Context
\nkubectl config set-context my-context –namespace=abc
\nkubectl config use-context use-context my-context<\/strong><\/p>\n

Objects
\nAs it was said before kubernetes check state of the system. System consists of objects, therefor to see current state or change state the following command can be used:
\nkubectl get [resource-name] [object-name] -o [json or yaml]<\/strong>
\nkubectl describe [resource-name] [object-name] -o [json or yaml]<\/strong>
\nkubectl apply -f obj.yaml<\/strong>
\nkubectl edit [resource-name] [object-name]<\/strong>
\nkubectl delete -f obj.yaml<\/strong><\/p>\n

Labels\/Annotations (add and remove)
\nkubectl label pods bar color=red
\nkubectl label pods bar -color<\/strong><\/p>\n

Logs
\nkubectl logs [pod-anme] -c [container]<\/strong><\/p>\n

Execution of commands in container:
\nkubectl exec -it [pod-name] — bash<\/strong><\/p>\n

Copy
\nkubectl cp [pod-name]:\/path\/to\/remote \/path\/to\/local<\/strong><\/p>\n

Pods
\nAll containers in pod always run on the same machine.
\n“Will these containers work correctly if they land on different machines?” no => use same pod. yes => use multiple pods.
\nPossible create pod as:
\nkubectl run [pod-name] –immage=[image path]<\/strong>
\nkubectl delete deployments\/[pod-name]<\/strong>
\nBetter is to change state and kubernetes will do this for you:
\nkubectl apply -f [pod-name].yaml<\/strong>
\nkubectl get pods<\/strong>
\nkubectl describe pods [pod-name]<\/strong>
\nkubectl delete pods\/[pod-name]<\/strong>
\nkubectl delete -f [pod-name].yaml<\/strong><\/p>\n

Access pod from outside:
\nkubectl port-forward [pod-name] [port-pod]:[ext:pod]<\/strong><\/p>\n

Health Checks
\nliveness – checks app specific logic
\nreadiness – ready to serve user requests
\ntcpSocket – checks socket
\nexec – any custom check<\/p>\n

Resource Management
\nRequests vs Limits<\/p>\n

Persisting Data with Volumes
\nContainer deleted => data is also deleted. Volumes are needed for persisted data.
\nPods => volumes => volumes may be accessed
\nContainers => volumeMounts => volume accessed
\nSpecial volume emptyDir => for caching, shared folder between containers. Will be deleted after pod is deleted.<\/p>\n

Labels\/Annotations
\nLabels => grouping objects in Kubernetes
\nAnnotations => description of objects for using by tools.
\nLabels can be used to separate between different environments: dev\/test\/prod. As labels we can mark different versions of application.
\nkubectl get pods –show-labels<\/strong>
\nPods can be selected by labels
\nkubectl get pods –selector=”[label=value]”<\/strong>
\nDifferent selectors are possible.
\nSelectors can be used also in yaml.<\/p>\n

Annotations can be used for
\n– track of changes
\n– what tool updates object
\n– build\/release\/image info
\n– track rollbacl for Deployment object<\/p>\n

Services and Service Discovery
\nService object in Kubernetes is for service discovery.
\nkubectl run [deploymnet-name]
\nkubectl expose deployment [deploymnet-name]
\nkubectl port-forward [pod] [pod port]:[external pod]
\n<\/strong>
\nCluster IP is a virtual IP.
\nDNS
\n[service-name].[namespace].svc.cluster.local
\nsvc means service.<\/p>\n

kubectl get endpoints [service name]<\/strong> –watch see changes of object overtime.
\nNodePort – access cluster from outside.
\nLoadBalancer – cloud integration
\nEndpoints – for accessing services without cluster IP<\/p>\n

Old way of service discovery using labels, but they should be in sync. New way using service object.<\/p>\n

Comunication
\nRequest => kube-proxy => Cluster IP (load balance + iptables rules to redirect) => Endpoints => Service<\/p>\n

ReplicaSets – pod manager replicates needed pods
\n– Redundancy
\n– Scale
\n– Sharding
\nReplicaSets are for self healing.<\/p>\n

Reconciliation Loops => check of desired state and current state. ReplicSet uses labels for this.<\/p>\n

Action for misbehaved pods => put them in quarantine using ReplicaSet.
\nkubectl describe rs [replica set name]<\/strong><\/p>\n

HPA horizontal (creates more pods) pod autoscaling. Special pod heapster communicates current load.<\/p>\n

Vertical (increases CPU) scaling is planned for implementation in Kubernetes.<\/p>\n

Always use ReplicaSet, also if only one pod is needed. This will help with scaling in the future.<\/p>\n

DaemonSets
\nDaemonSet is used to run one instance of specific pod on every node on cluster. This is like agents, for example logs collecting agents.<\/p>\n

kubectl describe daemonset [ds name]<\/strong>
\nIt is possible to label also nodes, not pods. And then use them in DaemonSet.
\nDaemonSets support rolling update.<\/p>\n

Jobs are useful for batch tasks.
\nPatterns: one shot, parallel fixed completions (parallelism and completions), work queue: parallel Jobs (producers and consumers).<\/p>\n

ConfigMaps and Secrets
\nConfigMap are supposed to be pod configuration for different environments. ConfigMap and Pod are combined together during start up.
\nConfigMap can be created from file, environment varible, command line arguments.
\nSecrets can be created in a cluster. The can be accessible throug secrets volume.<\/p>\n

Deployment
\nDeployment manages ReplicaSets.
\nStrategies: Recreate and RollingUpdate<\/p>\n

kubectl rollout status deployment [name]<\/strong>
\nkubectl rollout pause deployment [name]<\/strong>
\nkubectl rollout resume deployment [name]<\/strong>
\nkubectl rollout history deployment [name]<\/strong>
\nkubectl rollout undo deployment [name]<\/strong>
\nInstead of rollout undo it is better to revert YAML files from version control system.<\/p>\n

Dynamic Volume Provisioning
\nStorageClass
\nPersistentVolumeClaim<\/p>\n

StatefulSets
\n– each replica gets persistent hostname name-i
\n– each replica created in order of i
\n– each replica delted in reverse order of i<\/p>\n","protected":false},"excerpt":{"rendered":"

You can buy this book from amazon.de.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_bbp_topic_count":0,"_bbp_reply_count":0,"_bbp_total_topic_count":0,"_bbp_total_reply_count":0,"_bbp_voice_count":0,"_bbp_anonymous_reply_count":0,"_bbp_topic_count_hidden":0,"_bbp_reply_count_hidden":0,"_bbp_forum_subforum_count":0},"categories":[27,25],"tags":[57,35,49,50,31],"_links":{"self":[{"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=\/wp\/v2\/posts\/639"}],"collection":[{"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=639"}],"version-history":[{"count":41,"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=\/wp\/v2\/posts\/639\/revisions"}],"predecessor-version":[{"id":700,"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=\/wp\/v2\/posts\/639\/revisions\/700"}],"wp:attachment":[{"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=639"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=639"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/dekarlab.de\/wp\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=639"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}