Book notes – OpenShift in Action

BOOKS

You can buy this book in amazon.

OpenShift cuts operation costs and increases productivity. Also provides flexibility to use different technologies. Reusable code and infrastructure.

Part 1: Fundamentals

1 Getting to know OpenShift

Container platform = container run-time (create and manage containers) + orchestration engine (k8s).

Containers are discrete, portable, scalable units.

Application in container is isolated from other containers.

Isolation applies to: mounted filesystems, shared memory resources, host name and domain name, network resources (IP address, MAC address, memory buffers), and process counters.

OpenShift uses docker.

Master/node architecture in k8s.

OpenShift includes integrated image registry.

Nodes, pods, services. Service is proxy, which connects multiple pods and maps to an IP on one or modes in the cluster.

OpenShift provides routing layer to manage and share IP addresses.

Users access applications only through the routing layer (incl. DNS for application and consistent endpoint for all applications).

OpenShift uses software-defined networking (SDN).

Builder image is needed to deploy application.

OpenShift uses build pod for combining builder image with application code.

Application is deployed on nodes, pods are created, the service is created, and DNS name is added in routing layer.

Each VM has own virtualized linux kernel. Each container share same linux kernel.

Containers uses server resource more effectively and application density increases with containers.

Containers need separate storage solution. Containers are stateless. Stateless applications are easy to scale up and down with integrated load balancer.

2 Getting started

REST API, Web UI, CLI are ways to interact with OpenShift.

$ oc login -u dev -p dev https://ocp.nip.io:8443
$ oc new-project <project name>

Projects in OpenShift collects applications to logical groups, and helps with security.

  • Application components:
  • Custom container images
  • Image streams
  • Application pods
  • Build configs
  • Deployment configs
  • Deployments
  • Services

build config + source code + builder image => custom containers image for application

image streams monitors if something is changed to trigger build and deployments

deployment config (how to deploy) => deployments unique per app version => Pod (app in container) => Service (all pods as one service) => Load Balancer (Maps rout to deployed app) => Route (DNS entry in load balancer to access app) => User

Application replicas are defined in deployment. Replication controller is a pod, which controls replicas for application.

Pod Lifecycle => Pending, Running, Succeeded, Failed, Unknown

OpenShift supports also rolling upgrade

Image streams automate actions in OpenShift, like deployment and so on.

Command to deploy new application:

$ oc new-app \
> --image-stream=php\
> --code=https://github.com/app\
> --name=app

Information about application service:

$ oc describe svc/app
Name
Type    ClusterIP
Ip
Port
....

IP routable only inside of OpenShift cluster. This IP is only internal IP. You need to create Route (and DNS) to expose application to external world.

  • svc/app => service
  • bc/app => build config
  • dc/app => deployment config

HAProxy service is an open source load balancer.

Expose service with command: 

$ oc expose svc/app
$ oc describe route/app

Request routing for app: Route => Service => Pod

3 Containers are Linux

K8s inside of OpehShift consists of: Replication Controller, Service, Pods.

3 Kernel components for isolation: Linux namespaces (isolate applications in container), control groups (cgroups) (limit CPU and memory), SELinux contexts (limits access).

Userspace and kernelspace.

To see namespaces for linux process

$ lsns -p <PID>

Following namespaces are associated with each OpenShift container:

  • Mount (mnt)
  • Network (net)
  • Own process counters (pid)
  • Own host name and domain name (uts – unix time sharing)
  • Shared memory (ipc)

For node cgroups namespace is used. User namespace can be enabled, but performance issues.

OpenShit uses logical volume management (LVM).

To see all LVs created by docker:

$ lsblk

Command to get main docker process:

$ pgrep -f dockerd-current

Mount namespace in docker should be retrieved for docker.

To enter active namespace of application:

$ nsenter --target <PID>

System mount namespace => docker daemon mount namespace => app container mount namespace.

Scale application (OpenShift triggers recreation of deployment for app) and see list of pods, then see pod host name, after that show all processes in container

$ oc scale dc/app --replicas=2
$ oc get pods --show-all=false
$ oc exec app-<pod> hostname
$ oc exec app-<pod> ps

This command shows network in container

$ nsenter -t <PID> -n /sbin/ip a

Part 2 Cloud-native applications

4 Working with services

Replication controller to scale up and down of application.

Describe replicas for application (advanced info as yaml):

$ oc describe rc $(oc get rc -l app=app -o=jsonpath='{.items[].metadata.name}')
$ oc get rc app-cli-1 -o yaml

Objects in OpenShift

  • Image streams
  • Build configs
  • Deployment configs
  • Pods
  • Deployments
  • Container images
  • Services
  • Routes
  • Replication controllers

Pods tracked by labels and selectors. Labels and selectors can define bi directional relationship between pods. For example, replication controller use selectors to find labeled pods to mange.

Delete pod:

$ oc delete pod -l app=app-cli

Control through choreography:

  • Decoupled API components
  • Avoiding stateful information
  • Looping through control loops against various microservices

Application in OpenShift:

Routes => Services => Pods

  • Users access the application through the routing layer.
  • The routing layer gets pod IP addresses from the OpenShift API server so it can connect to pods directly.
  • The Routing layer connects directly to the presentation layer pod on user access, without the extra hop of connecting through the service.
  • Pods can connect directly with each other with or without the services layer.
  • Pods scale independently to handle their workloads.

Scaling pods:

oc scale dc app-cli-1 --replicas=1

Liveness probe:

  • HTTP Checks
  • Container execution check
  • TCP socket check

Readiness probe: check if container is ready to receive traffic.

5 Autoscaling with metrics

Determining expected workloads is difficult.

Autoscaling => automatically create new pods

To enable autoscaling metrics stack is needed (hawkular, heapster, cassandra), . In future it will be possible to use Prometheus.

Horizontal Pod Autoscaler (HPA) component responsible for autosclaing in OpenShift.

CPU is measured in OpenShift as millicore (1/1000 of core).

Resource request, what resources are needed. Resource limit => maximum of resources provided.

CPU request

oc set resources dc app-cli --requests=cpu=400m

Look in HPA

oc describe hpa app-cli

Apache Benchmark can be used for generating load to test autoscaling.

There are parameters to avoid thrashing by autoscaling.

6 Continuous integration and continuous deployment

Container images are the centerpiece of a CI/CD pipeline.

Containers reduces deployment risk on production, since same binary is tested and installed on production.

Container can be easily rolled back to previous version.

Image tagging is used to promote container to higher environments, for example, from development to acceptance.

OpenShift template is used for application installation.

Steps to deploy application: create project, import template into project, instantiate application from template.

Webhook from git can trigger OpenShift build.

You cannot patch application in container, since changes will not persist. You need to rebuild it.

Image stream is a concept in OpenShift to solve a problem of consistent application deployment, when application consists of more than one container container.

Every time the image stream is updated, a new deployment can be triggered in the test environment. You need to mark container with new tag, then test environment will pull all containers with this tag.

For every environment, like, development, test, and so on, you need separate project.

To combine project, you need to give pull permission, for example, test should have container pull permission for development, but not vice versa.

Service has static IP (ClusterIP), which is not visible for external world.

SkyDNS is used for internal discovery of services. Every pod knows, where SkyDNS is running (in container /etc/resolv.conf). Service has name: <service>.<pod_namespace>.svc.cluster.local.

Using environment variables for service discovery is a bad idea.

Image stream triggers are needed to move deployments between environments.

Manual tagging of image in dev is an art of approval process to promote to test.

Sensitive configuration can be protected with secrets.

oc create secret generic <secret name>

Config maps

oc create configmap <config name>

Jenkins is also part of OpenShift.

Deployment strategy: Rolling, Re-create, Blue/Green, Canary, Dark launches.

Part 3 Stateful applications

7 Creating and managing persistent storage

You need persistence volume (PV) for permanent storage of data.

OpenShift uses YAML as configuration format.

PVs are cluster wide resources.

PV main attributes: NFS path, recycle strategy (Retain or Recycle), NFS server, capacity, access mode.

Note: PV is better than direct mount.

PVs represent available storage, PVCs (claims) represent application’s need for that storage.

Volume is any filesystem, file, or data mounted into pod.

Container => (bind mount(Volume )) => Host => (physical mount (PV)) => remote storage

PVC defines link Volume <=> PV for given application and provides isolation.

  • During container restart bind mount can be created quickly (in comparison to physical mount)
  • Container storage is isolated from remote storage, which improves maintainability.

8 Stateful applications

Tools for stateful applications: headless services, sticky sessions, pod-discovery techniques, and stateful sets.

Headless services in OpenShift have ClusterIP as NA. Headless service groups pods, hence it is possible to retrieve of all pods from this service, but access is done directly to pod.

For headless services it is possible to use application-level clustering in Open-Shift using WildFly (JBoss).

Brokers in Kafka running on OpenShift uses headless services and OpenShift DNS entries to implement custom routing directly between pods.

Sticky sessions solve problem, with multiple pods in service, from which only one contains session data. Router OpenShift object uses cookies for automatic sticky session implementation. You need send request to OpenShift, grab cookie and reuse it for consequent requests.

StateFul set in OpenShift is used to control sequence of starting and stopping pods in a service and provide predictable network identity for pods.

Part 4 Operations and security

9 Authentication and resource access

OpenShift uses balanced approach between self-service and security.

OpenShift has Limit-range resource constraints for: pods, containers, images, image streams, persistent volume claims.

Limits are also possible on project level.

It is possible also to use cgroups to limit resources.

10 Networking

OpenShift creates Pod network.

oc get hostsubnet

OpenShift uses Open vSwitch (OVS): www.openvswitch.org.

Physical Interfaces in Node: lo, eth0

Virtual bridge: br0

Virtual interfaces, which are connected to br0: vethXXX , tun0, vxlan_sys.

VXLAN is overlay network between all nodes in cluster.

TUNnel mimics functionality of physical interface.

Pod outside cluster communication: Pod (eth0) => vethXXX => br0 => tun0 => Server Outside.

Pod inside cluster communication: Pod (eth0) => vethXXX => br0 => vxlan_sys => Node in Cluster.

HAProxy is used for traffice inside of the cluster. HAProxy pod is running in OpenShift project default.

Configuration of HAProxy is updated automatically by OpenShift, when pods are created.

OpenShift OVS configuartion: ovs-subnet (flat network for whole cluster), ovs-multitenant (isolated network per project), ovs-networkpolicy (network with complex rules).

11 Security

Each pod is assigned a security context that includes information about its capabilities
on the host.

SELinux

Scanning container images. OpenShift uses www.open-scap.org.

Annotating images with security information.

Post Views: 1,437