CyberAgent コンテナハンズオン

Dockerの復習

docker multi scale build

kubernetes

kubectl get ns , kubectl get namespace

context -> userのnamespaceや認証情報を束ねるもの

podsの起動

$ kubectl get pods
No resources found.
$
$ kubectl  create -f sample-pod.yaml 
pod "sample-pod" created

情報の確認

kubectl get pods -o wide
kubectl get pods -o json
kubectl get pods -o yaml
kubectl get pods --show-label

kubectl describe xxx

詳しいコマンドは4章を参照

podの作成や更新

kubectl apply -f sample-pod.yaml

podの作成のみ

kubectl create -f sample-pod.yaml

podsの削除

kubectl delete xxx

kubectl get pod ←保管が入る

Labelをodsにつけることで区別したり、ロードバランシングの切り替え

podsに入るとき

kubectl exec -it sample-pod bash

ENTRYPOINT/CMDとcommand/argsの対応づけ

DockerENTRYPOINTCOMMAND
Kubenetescommandargs

sample-2pod.yaml

apiVersion: v1
kind: Pod 
metadata:
  name: sample-pod2
spec:
  containers:
    - name: nginx-container1
      image: nginx:1.12
    - name: nginx-container2
      image: nginx:1.12

ReplicateSetについて

chapter05/sample-rs.yaml

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: sample-rs
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sample-app
  template:
    metadata:
      labels:
        app: sample-app
    spec:
      containers:
        - name: nginx-container
          image: nginx:1.12
          ports:
            - containerPort: 80

RSでpodsの起動

$ kubectl apply -f sample-rs.yaml
$ kubectl get pods
$ k get pods
NAME              READY     STATUS             RESTARTS   AGE
sample-pod        1/1       Running            0          59m
sample-pod2       1/2       CrashLoopBackOff   8          19m
sample-rs-lpstg   1/1       Running            0          37s
sample-rs-pjhm2   1/1       Running            0          37s
sample-rs-z9mwb   1/1       Running            0          37s

Kubernetesのインストール https://github.com/kubernetes/minikube/releases

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

sudo dnf install kubernetes

Minikubeのインストール https://kubernetes.io/docs/tasks/tools/install-minikube/

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.30.0/minikube-linux-amd64 && chmod +x minikube && sudo cp minikube /usr/local/bin/ && rm minikube

podsの状態を見ながら…

$ watch kubectl get pods
Every 2.0s: kubectl get pods                 gari: Sat Dec  1 15:05:19 2018

NAME              READY     STATUS             RESTARTS   AGE
sample-pod        1/1       Running            0          1h
sample-pod2   1/2       CrashLoopBackOff   9          25m
sample-rs-lpstg   1/1       Running            0          6m
sample-rs-z9mwb   1/1       Running            0          6m

RSでスケーリングする

$ kubectl scale --replicas 4 rs sample-rs
Every 2.0s: kubectl get pods                 gari: Sat Dec  1 15:06:48 2018

NAME              READY     STATUS             RESTARTS   AGE
sample-pod        1/1       Running            0          1h
sample-pod2   1/2       CrashLoopBackOff   9          26m
sample-rs-5v8bn   1/1       Running            0          28s
sample-rs-lpstg   1/1       Running            0          7m
sample-rs-wn6pc   1/1       Running            0          28s
sample-rs-z9mwb   1/1       Running            0          7m
## これは以下でも実現可能
$ kubectl get rs -w

$ kubectl delete pods sample-rs-z9mwb
Every 2.0s: kubectl get pods                 gari: Sat Dec  1 15:06:48 2018

NAME              READY     STATUS             RESTARTS   AGE
sample-pod        1/1       Running            0          1h
sample-pod2       1/2       CrashLoopBackOff   15         53m
sample-rs-f7flj   1/1       Running            0          5m
sample-rs-lpstg   1/1       Running            0          34m
sample-rs-n8vfm   1/1       Running            0          3m
sample-rs-wn6pc   1/1       Running            0          27m

レプリカセットはレプリカ数を指定してその数だけレプリカを起動しておくためのもの。

ロールバックと柔軟

$ kubectl set image deployment sample-deployment nginx-container=nginx:1.13
deployment.apps "sample-deployment" image updated

$ kubectl get deploy
NAME                DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
sample-deployment   3         4         2            3           2m
Deployment
     | ---- Replica
     | ---- Replica
     | ---- Replica

kubectl rollout undoで戻せる。--record をつけないと記録をしてくれない。ファイルをバージョン管理して apply で実行する。

基本的なポリシー: manifestを書いていく。

[gari:~/work_k8s/kubernetes-perfect-guide/samples/chapter06]$ k get pods
NAME                                 READY     STATUS    RESTARTS   AGE
sample-deployment-86d576464c-jjzw8   1/1       Running   0          2m
sample-deployment-86d576464c-jnv2x   1/1       Running   0          2m
sample-deployment-86d576464c-lnxbm   1/1       Running   0          2m

[gari:~/work_k8s/kubernetes-perfect-guide/samples/chapter06]$ k get pods -o wide
NAME                                 READY     STATUS    RESTARTS   AGE       IP            NODE
sample-deployment-86d576464c-jjzw8   1/1       Running   0          2m        10.116.2.7    gke-ca-k8s-22-default-pool-7b781e37-xcmr
sample-deployment-86d576464c-jnv2x   1/1       Running   0          2m        10.116.0.18   gke-ca-k8s-22-default-pool-7b781e37-dp4k
sample-deployment-86d576464c-lnxbm   1/1       Running   0          2m        10.116.1.17   gke-ca-k8s-22-default-pool-7b781e37-0j2h
[gari:~/work_k8s/kubernetes-perfect-guide/samples/chapter06]$ 
[gari:~/work_k8s/kubernetes-perfect-guide/samples/chapter06]$ 
[gari:~/work_k8s/kubernetes-perfect-guide/samples/chapter06]$ k get service  -w
NAME         TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
kubernetes   ClusterIP      10.119.240.1     <none>        443/TCP          8h
sample-lb    LoadBalancer   10.119.246.206   <pending>     8080:30082/TCP   47s
sample-lb   LoadBalancer   10.119.246.206   104.198.122.161   8080:30082/TCP   51s

HomeWork

Booted status

k get pods -L app,release
NAME                            READY     STATUS    RESTARTS   AGE       APP            RELEASE                                                                             
hw-nginx-96c686b56-4r8sh        1/1       Running   0          25s       homework-ngx   alpha                                                                               
hw-nginx-96c686b56-6vshh        1/1       Running   0          25s       homework-ngx   alpha                                                                               
hw-nginx-96c686b56-7vnw5        1/1       Running   0          25s       homework-ngx   alpha                                                                               
hw-nginx-host-569d6f448-hf4xd   1/1       Running   0          25s       homework-ngx   beta                                                                                
hw-nginx-host-569d6f448-qfxtl   1/1       Running   0          25s       homework-ngx   beta                                                                                
hw-nginx-host-569d6f448-ztkwb   1/1       Running   0          25s       homework-ngx   beta                                                

hw-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: hw-configmap
data:
  index.html: |
    hello world!

hw-nginx-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hw-nginx
spec:
  replicas: 3
  selector:
    matchLabels:
      app: homework-ngx
      release: alpha
  template:
    metadata:
      labels:
        app: homework-ngx
        release: alpha
    spec:
      containers:
        - name: nginx-container
          image: nginx:1.12
          ports:
            - containerPort: 80
          volumeMounts:
            - name: config-volume
              mountPath: /usr/share/nginx/html
      volumes:
        - name: config-volume
          configMap:
            name: hw-configmap

hw-nginx-host-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: hw-nginx-host
spec:
  replicas: 3
  selector:
    matchLabels:
      app: homework-ngx
      release: beta
  template:
    metadata:
      labels:
        app: homework-ngx
        release: beta
    spec:
      containers:
        - name: nginx-hostname-container
          image: makocchi/docker-nginx-hostname:latest
          ports:
            - containerPort: 80

hw-lb.yaml

apiVersion: v1
kind: Service
metadata:
  name: homework-lb
spec:
  type: LoadBalancer
  ports:
    - name: "http-port-normal"
      protocol: "TCP"
      port: 8080
      targetPort: 80
      nodePort: 30080
  selector:
    app: homework-ngx

メモリやCPUの制限(抜粋)

resources:
  requests:
    memory: 1024Mi
    cpu: 500m
  limits:
    memory: 2048Mi
    cpu: 1000m

nodeの状態を確認する

k get nodes
NAME                                       STATUS    ROLES     AGE       VERSION
gke-ca-k8s-22-default-pool-7b781e37-0j2h   Ready     <none>    1d        v1.11.2-gke.18
gke-ca-k8s-22-default-pool-7b781e37-dp4k   Ready     <none>    1d        v1.11.2-gke.18
gke-ca-k8s-22-default-pool-7b781e37-xcmr   Ready     <none>    1d        v1.11.2-gke.18
k describe node gke-ca-k8s-22-default-pool-7b781e37-0j2h
Name:               gke-ca-k8s-22-default-pool-7b781e37-0j2h
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/fluentd-ds-ready=true
                    beta.kubernetes.io/instance-type=n1-standard-2
                    beta.kubernetes.io/masq-agent-ds-ready=true
                    beta.kubernetes.io/os=linux
                    cloud.google.com/gke-nodepool=default-pool
                    cloud.google.com/gke-os-distribution=cos
                    failure-domain.beta.kubernetes.io/region=asia-northeast1
                    failure-domain.beta.kubernetes.io/zone=asia-northeast1-a
                    kubernetes.io/hostname=gke-ca-k8s-22-default-pool-7b781e37-0j2h
                    projectcalico.org/ds-ready=true
Annotations:        container.googleapis.com/instance_id=7900142075992893877
                    node.alpha.kubernetes.io/ttl=0
                    volumes.kubernetes.io/controller-managed-attach-detach=true
CreationTimestamp:  Sat, 01 Dec 2018 09:38:21 +0900
Taints:             <none>
Unschedulable:      false
Conditions:
  Type                          Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----                          ------  -----------------                 ------------------                ------                       -------
  FrequentUnregisterNetDevice   False   Sun, 02 Dec 2018 11:11:35 +0900   Sat, 01 Dec 2018 09:41:12 +0900   UnregisterNetDevice          node is functioning properly
  KernelDeadlock                False   Sun, 02 Dec 2018 11:11:35 +0900   Sat, 01 Dec 2018 09:36:10 +0900   KernelHasNoDeadlock          kernel has no deadlock
  NetworkUnavailable            False   Sat, 01 Dec 2018 09:38:40 +0900   Sat, 01 Dec 2018 09:38:40 +0900   RouteCreated                 RouteController created a route
  OutOfDisk                     False   Sun, 02 Dec 2018 11:11:53 +0900   Sat, 01 Dec 2018 09:38:21 +0900   KubeletHasSufficientDisk     kubelet has sufficient disk space available
  MemoryPressure                False   Sun, 02 Dec 2018 11:11:53 +0900   Sat, 01 Dec 2018 09:38:21 +0900   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure                  False   Sun, 02 Dec 2018 11:11:53 +0900   Sat, 01 Dec 2018 09:38:21 +0900   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure                   False   Sun, 02 Dec 2018 11:11:53 +0900   Sat, 01 Dec 2018 09:38:21 +0900   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready                         True    Sun, 02 Dec 2018 11:11:53 +0900   Sat, 01 Dec 2018 09:38:41 +0900   KubeletReady                 kubelet is posting ready status. AppArmor enabled
Addresses:
  InternalIP:  10.146.15.218
  ExternalIP:  35.189.154.83
  Hostname:    gke-ca-k8s-22-default-pool-7b781e37-0j2h
Capacity:
 cpu:                2
 ephemeral-storage:  98868448Ki
 hugepages-2Mi:      0
 memory:             7658156Ki
 pods:               110
Allocatable:
 cpu:                1930m
 ephemeral-storage:  47093746742
 hugepages-2Mi:      0
 memory:             5778092Ki
 pods:               110
System Info:
 Machine ID:                 6787c9c726acfaf745be0868a21e70cc
 System UUID:                6787C9C7-26AC-FAF7-45BE-0868A21E70CC
 Boot ID:                    c61fdcff-8e0e-4a5e-9594-27cbd7a64029
 Kernel Version:             4.14.65+
 OS Image:                   Container-Optimized OS from Google
 Operating System:           linux
 Architecture:               amd64
 Container Runtime Version:  docker://17.3.2
 Kubelet Version:            v1.11.2-gke.18
 Kube-Proxy Version:         v1.11.2-gke.18
PodCIDR:                     10.116.1.0/24
ExternalID:                  gke-ca-k8s-22-default-pool-7b781e37-0j2h
ProviderID:                  gce://cyberagent-239/asia-northeast1-a/gke-ca-k8s-22-default-pool-7b781e37-0j2h
Non-terminated Pods:         (9 in total)
  Namespace                  Name                                                   CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ---------                  ----                                                   ------------  ----------  ---------------  -------------
  default                    hw-nginx-96c686b56-4r8sh                               100m (5%)     0 (0%)      0 (0%)           0 (0%)
  default                    hw-nginx-host-569d6f448-hf4xd                          100m (5%)     0 (0%)      0 (0%)           0 (0%)
  default                    hw-nginx-host-569d6f448-ztkwb                          100m (5%)     0 (0%)      0 (0%)           0 (0%)
  kube-system                calico-node-6rdhb                                      100m (5%)     0 (0%)      0 (0%)           0 (0%)
  kube-system                fluentd-gcp-v3.1.0-sjjkm                               0 (0%)        0 (0%)      0 (0%)           0 (0%)
  kube-system                heapster-v1.6.0-beta.1-5fd4cdc95c-4w8ms                138m (7%)     138m (7%)   301856Ki (5%)    301856Ki (5%)
  kube-system                ip-masq-agent-6jqwq                                    10m (0%)      0 (0%)      16Mi (0%)        0 (0%)
  kube-system                kube-proxy-gke-ca-k8s-22-default-pool-7b781e37-0j2h    100m (5%)     0 (0%)      0 (0%)           0 (0%)
  kube-system                metrics-server-v0.2.1-fd596d746-jm8r2                  53m (2%)      148m (7%)   154Mi (2%)       404Mi (7%)
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  CPU Requests  CPU Limits  Memory Requests  Memory Limits
  ------------  ----------  ---------------  -------------
  701m (36%)    286m (14%)  475936Ki (8%)    715552Ki (12%)
Events:         <none>
openssl req -x509 -nodes -days 365 -newkey rsa:2048  -keyout ~/tls.key -out ~/tls.crt -subj "/CN=sample.example.com"

kubectl create secret tls --save-config tls-sample --key ~/tls.key --cert ~/tls.crt

kubectl apply -f sample-ingress-apps.yaml
kubectl apply -f sample-ingress.yaml

kubectl exec -it sample-ingress-apps-1 -- mkdir /usr/share/nginx/html/path1/
kubectl exec -it sample-ingress-apps-1 -- cp /etc/hostname /usr/share/nginx/html/path1/index.html
kubectl exec -it sample-ingress-apps-2 -- mkdir /usr/share/nginx/html/path2
kubectl exec -it sample-ingress-apps-2 -- cp /etc/hostname /usr/share/nginx/html/path2/index.html

kubectl get ing ADDRESSが表示されるまでちょっと待つ

INGRESS_IP=`kubectl get ingresses sample-ingress -o jsonpath='{.status.loadBalancer.ingress[0].ip}'`

(下記がうまく表示されない場合は、ちょっと待つ)

curl http://${INGRESS_IP}/path1/index.html -H "Host: sample.example.com"

青山真也/amsy810 [13:13]

ingressリソースの状態を確認(エラー起きてるかどうかなど)

kubectl describe ing
## kubectl get ingress 
NAME             HOSTS                ADDRESS          PORTS     AGE
sample-ingress   sample.example.com   35.244.196.127   80, 443   1h
k logs -f sample-ingress-apps-1