Kubernetes Worker Node Join

기존 Kubernetes Cluster 에 Worker Node를 추가로 Join 하는 경우에 대해 정리해 보았습니다.

 

1. Docker Install
2. Kubeadm Install
3. Join K8S Cluster

1. Docker Install

 

$ systemctl stop firewalld

$ systemctl disable firewalld

$ systemctl mask --now firewalld

 

/var/lib/docker 를 별도의 볼륨으로 마운트하는것을 권고합니다.

 

$ fdisk -l

Disk /dev/sdb: 75.2 GB, 75161927680 bytes, 146800640 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes

$ fdisk /dev/sdb

Command (m for help): n Partition type:    p   primary (0 primary, 0 extended, 4 free)    e   extended Select (default p): p Partition number (1-4, default 1): First sector (2048-146800639, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-146800639, default 146800639): Using default value 146800639 Partition 1 of type Linux and of size 70 GiB is set

Command (m for help): t  Selected partition 1  Hex code (type L to list all codes): L   0  Empty           24  NEC DOS         81  Minix / old Lin bf  Solaris   1  FAT12           27  Hidden NTFS Win 82  Linux swap / So c1  DRDOS/sec (FAT-   2  XENIX root      39  Plan 9          83  Linux           c4  DRDOS/sec (FAT-   3  XENIX usr       3c  PartitionMagic  84  OS/2 hidden C:  c6  DRDOS/sec (FAT-   4  FAT16 <32M      40  Venix 80286     85  Linux extended  c7  Syrinx   5  Extended        41  PPC PReP Boot   86  NTFS volume set da  Non-FS data   6  FAT16           42  SFS             87  NTFS volume set db  CP/M / CTOS / .   7  HPFS/NTFS/exFAT 4d  QNX4.x          88  Linux plaintext de  Dell Utility   8  AIX             4e  QNX4.x 2nd part 8e  Linux LVM       df  BootIt   9  AIX bootable    4f  QNX4.x 3rd part 93  Amoeba          e1  DOS access  Hex code (type L to list all codes): 8e  Changed type of partition 'Linux' to 'Linux LVM'  Command (m for help): w The partition table has been altered! Calling ioctl() to re-read partition table. Syncing disks.

Command (m for help): p Disk /dev/sdb: 75.2 GB, 75161927680 bytes, 146800640 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk label type: dos Disk identifier: 0x8a027e7e    Device Boot      Start         End      Blocks   Id  System /dev/sdb1            2048   146800639    73399296   8e  Linux LVM

$ mkfs.xfs /dev/sdb1 -n ftype=1 -f

 

$ vi /etc/fstab

: swap 영역은 주석처리

: /var/lib/docker 마운트 추가

#/dev/mapper/centos-swap swap                    swap    defaults        0 0 /dev/sdb1 /var/lib/docker                    xfs    defaults        0 0

$ swapoff -a

$ mkdir -p /var/lib/docker

$ mount -a

 

$ yum -y install yum-utils device-mapper-persistent-data lvm2

$ yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo 

$ yum update -y

$ yum install -y containerd.io-1.2.13 docker-ce-19.03.11 docker-ce-cli-19.03.11

$ mkdir /etc/docker

$ vi /etc/docker/daemon.json

{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m"
  },
  "storage-driver": "overlay2",
  "storage-opts": [
    "overlay2.override_kernel_check=true"
  ]
}

$ mkdir -p /etc/systemd/system/docker.service.d

$ systemctl daemon-reload
$ systemctl restart docker
$ systemctl enable docker

 

 

2. Kubeadm Install

 

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
> [kubernetes]
> name=Kubernetes
> baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
> enabled=1
> gpgcheck=1
> repo_gpgcheck=1
> gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
> EOF

$ setenforce 0

sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

$ yum install -y kubelet-1.18.10 kubeadm-1.18.10 kubectl-1.18.10 --disableexcludes=kubernetes

: 설치하고자 하는 버전 지정

$ systemctl enable --now kubelet

cat <<EOF > /etc/sysctl.d/k8s.conf
> net.bridge.bridge-nf-call-ip6tables = 1
> net.bridge.bridge-nf-call-iptables = 1
> EOF

$ sysctl --system

* Applying /usr/lib/sysctl.d/00-system.conf ...
net.bridge.bridge-nf-call-ip6tables = 0
net.bridge.bridge-nf-call-iptables = 0
net.bridge.bridge-nf-call-arptables = 0
* Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
kernel.yama.ptrace_scope = 0
* Applying /usr/lib/sysctl.d/50-default.conf ...
kernel.sysrq = 16
kernel.core_uses_pid = 1
kernel.kptr_restrict = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.all.accept_source_route = 0
net.ipv4.conf.default.promote_secondaries = 1
net.ipv4.conf.all.promote_secondaries = 1
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.d/k8s.conf ...
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

$ systemctl daemon-reload

$ systemctl restart kubelet

 

3. Join K8S Cluster

 

$ kubeadm join --token <token> <control-plane-host>:<control-plane-port> --discovery-token-ca-cert-hash sha256:<hash>

 kubeadm join --token 1sy6t8.cc2symwg3jzn117s 192.168.19.133:6443 --discovery-token-ca-cert-hash sha256:18b156c1587c387cdeef7e73a5cef43adbd6dc31adc5427eebad8bcb284e10b7

 

만약 token 값이 없는경우 master node에서 재생성합니다.

$ kubeadm token create

1sy6t8.cc2symwg3jzn117s

$ kubeadm token list

TOKEN                     TTL         EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
1sy6t8.cc2symwg3jzn117s   23h         2020-11-25T06:25:16+09:00   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token

 

discovery-token-ca-cert-hash 값을 구하려면,

$ openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
18b156c1587c387cdeef7e73a5cef43adbd6dc31adc5427eebad8bcb284e10b7

 

워커노드에서 kubeadm join명령 수행

$ kubeadm join --token 1sy6t8.cc2symwg3jzn117s 192.168.19.133:6443 --discovery-token-ca-cert-hash sha256:18b156c1587c387cdeef7e73a5cef43adbd6dc31adc5427eebad8bcb284e10b7

 

master노드에서 확인

$ kubectl get node

NAME        STATUS   ROLES    AGE     VERSION k8sworker   Ready    <none>   4m44s   v1.18.10 master01    Ready    master   12d     v1.18.10

 

만약, master 노드에도 스케줄되도록 설정하려면,

$ kubectl taint nodes --all node-role.kubernetes.io/master-
node/master01 untainted

 

다시 되돌리려면

$ kubectl taint nodes master01 node-role.kubernetes.io/master:NoSchedule
node/master01 tainted