- 使用 external etcd cluster w/o TLS cert
- 3 master node HA
- 3 worker node
- OpenRC
- containerd
prepare
ntpdate clock.stdtime.gov.tw
emerge --sync
etcd
emerge --ask dev-db/etcd
修改檔案
/etc/conf.d/etcd
PEER0="k8s-node1"
PEER0_IP="10.168.100.101"
PEER1="k8s-node2"
PEER1_IP="10.168.100.102"
PEER2="k8s-node3"
PEER2_IP="10.168.100.103"
MY_HOST="${PEER1}"
MY_IP="${PEER1_IP}"
ETCD_OPTS="--name ${MY_HOST} --initial-advertise-peer-urls http://${MY_IP}:2380 \
--listen-peer-urls http://${MY_IP}:2380 \
--listen-client-urls http://${MY_IP}:2379,http://127.0.0.1:2379 \
--advertise-client-urls http://${MY_IP}:2379 \
--initial-cluster-token etcd-cluster-1 \
--initial-cluster ${PEER0}=http://${PEER0_IP}:2380,${PEER1}=http://${PEER1_IP}:2380,${PEER2}=http://${PEER2_IP}:2380 \
--initial-cluster-state new"
start service
rc-update add etcd default rc-service etcd start
testing
etcd put foo bar
Load Balancer
emerge keepalived haproxy
rc-update add keepalived default
rc-update add haproxy default
其餘方式如下，省略
https://github.com/kubernetes/kubeadm/blob/master/docs/ha-considerations.md#options-for-software-load-balancing
k8s
安裝
emerge --unmerge docker
emerge --ask cri-tools containerd
emerge --ask ebtables ethtool
emerge --ask sys-cluster/kubeadm kubectl kubelet
emerge net-misc/socat conntrack-tools
rc-update add kubelet default
rc-update add containerd default
rc-service containerd start
系統設定
也可以寫進開機script
networking
modprobe br_netfilter
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
echo 1 > /proc/sys/net/ipv4/ip_forward
echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
swap
/etc/fstab 砍掉 swap 分區
sysctl -w vm.swappiness=0
swapoff -a
edit hostname
vim /etc/conf.d/hostname
add ip to dns server
我有內部DNS 懶惰也可以加 /etc/hosts
加入 kubelet service 的參數
/etc/conf.d/kubelet
command_args="--container-runtime=remote \
--container-runtime-endpoint=unix:///run/containerd/containerd.sock \
--image-service-endpoint=unix:///run/containerd/containerd.sock \
--runtime-request-timeout=3m \
--kubeconfig=/etc/kubernetes/kubelet.conf \
--config=/var/lib/kubelet/config.yaml \
--bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf"
using crictl
加進 .profile 檔
export CONTAINER_RUNTIME_ENDPOINT=unix:///run/containerd/containerd.sock
external ETCD cluster 設定法
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/high-availability/#set-up-the-first-control-plane-node
照著新增
kubeadm init –config=kubeadm-config.yaml …
my config
apiVersion: kubeadm.k8s.io/v1beta2
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 24h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: 10.168.100.101
bindPort: 6443
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
name: k8s-node1
taints:
- effect: NoSchedule
key: node-role.kubernetes.io/master
---
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
apiServer:
timeoutForControlPlane: 4m0s
extraArgs:
advertise-address: 10.168.100.100
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controlPlaneEndpoint: "10.168.100.100:6443"
controllerManager: {}
dns:
type: CoreDNS
etcd:
external:
endpoints:
- http://10.168.100.101:2379
- http://10.168.100.102:2379
- http://10.168.100.103:2379
imageRepository: k8s.gcr.io
kubernetesVersion: v1.20.0
networking:
dnsDomain: cluster.local
serviceSubnet: 10.197.0.0/16
scheduler: {}
---
加入其他 node
kubeadm join …
找回加入的token
kubeadm token create --print-join-command
kubeadm certs certificate-key
command example
kubeadm join 10.168.100.100:6443 --token dmjuyu.ym3ly49ntcokc8hj \
--discovery-token-ca-cert-hash sha256:70729369809845fc5f98b006b8f84e5e4cef7341c1899d74b9052b62b1757227
--control-plane \
--certificate-key b9d8bd1a612f9fafff08a5067da125d231612d1a7ab6fe098a62068a6996d2d3
新增 metrics server
kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
在 container args 加上
--kubelet-insecure-tls
