K8S 使用IPVS转发
一、介绍
kube-proxy有两种模式,ipvs和iptables两种模式，IPvs的性能更好，正是因为ipvs的高效性能，所以，将kube-proxy的模式切换为ipvs是很有必要的。
ipvs与iptables区别
相同点：ipvs和iptables都是通过netfitle内核进行转发。
异同点：iptables只是为防火墙设计的，iptables只是防火墙，而ipvs是专门用于高性能负载均衡的，并使用更高效的数据结构，如hash表并支持索引。
pvs与iptables相比较，其优势为：
（1）ipvs为大型集群提供了更好的可扩展性和性能
（2）ipvs支持比iptables更复杂的负载均衡算法，如rr、wrr、lc、wlc
（3）ipvs支持服务健康检查和链接重试等功能
（4）ipvs可以动态修改ipset集合
二、查看当前使用的模式
查看当前模式mode是为空的，默认为iptables
[root@k8s-master ~]# kubectl get configmap kube-proxy -n kube-system -o yaml | grep mode
mode: ""查看日志验证
[root@k8s-master ~]# kubectl -n kube-system get po
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-5bb5d4f7f4-knlgr 1/1 Running 0 49m
calico-node-2ds8v 1/1 Running 1 (56m ago) 18h
calico-node-6wj9b 1/1 Running 1 (56m ago) 18h
calico-node-pxrk7 1/1 Running 1 (56m ago) 18h
coredns-6d8c4cb4d-fcjr4 1/1 Running 1 (56m ago) 19h
coredns-6d8c4cb4d-l5rtm 1/1 Running 1 (56m ago) 19h
etcd-k8s-master 1/1 Running 1 (56m ago) 19h
kube-apiserver-k8s-master 1/1 Running 1 (56m ago) 19h
kube-controller-manager-k8s-master 1/1 Running 1 (56m ago) 19h
kube-proxy-dtq6r 1/1 Running 1 (56m ago) 19h
kube-proxy-gmg7w 1/1 Running 1 (56m ago) 19h
kube-proxy-hzfwc 1/1 Running 1 (56m ago) 19h
kube-scheduler-k8s-master 1/1 Running 1 (56m ago) 19h
metrics-server-86597c44b6-j9x82 1/1 Running 1 (56m ago) 17h
metrics-server-86597c44b6-z9qbw 1/1 Running 0 49m
[root@k8s-master ~]# kubectl -n kube-system logs kube-proxy-dtq6r
I0718 02:14:51.481375 1 node.go:163] Successfully retrieved node IP: 192.168.30.59
I0718 02:14:51.481455 1 server_others.go:138] "Detected node IP" address="192.168.30.59"
I0718 02:14:51.481549 1 server_others.go:561] "Unknown proxy mode, assuming iptables proxy" proxyMode=""
I0718 02:14:53.130494 1 server_others.go:206] "Using iptables Proxier"
I0718 02:14:53.130549 1 server_others.go:213] "kube-proxy running in dual-stack mode" ipFamily=IPv4
I0718 02:14:53.130557 1 server_others.go:214] "Creating dualStackProxier for iptables"
I0718 02:14:53.130572 1 server_others.go:491] "Detect-local-mode set to ClusterCIDR, but no IPv6 cluster CIDR defined, , defaulting to no-op detect-local for IPv6"
I0718 02:14:53.130968 1 server.go:656] "Version info" version="v1.23.0"
I0718 02:14:53.134344 1 conntrack.go:100] "Set sysctl" entry="net/netfilter/nf_conntrack_max" value=131072
I0718 02:14:53.134457 1 conntrack.go:52] "Setting nf_conntrack_max" nf_conntrack_max=131072
I0718 02:14:53.134857 1 conntrack.go:83] "Setting conntrack hashsize" conntrack hashsize=32768
I0718 02:14:53.139119 1 conntrack.go:100] "Set sysctl" entry="net/netfilter/nf_conntrack_tcp_timeout_close_wait" value=3600
I0718 02:14:53.139465 1 config.go:317] "Starting service config controller"
I0718 02:14:53.139495 1 shared_informer.go:240] Waiting for caches to sync for service config
I0718 02:14:53.139552 1 config.go:226] "Starting endpoint slice config controller"
I0718 02:14:53.139559 1 shared_informer.go:240] Waiting for caches to sync for endpoint slice config
I0718 02:14:53.241273 1 shared_informer.go:247] Caches are synced for service config
I0718 02:14:53.241285 1 shared_informer.go:247] Caches are synced for endpoint slice config
E0718 02:14:53.371157 1 proxier.go:1600] "can't open port, skipping it" err="listen tcp4 :31370: bind: address already in use" port={Description:nodePort for test/nginx-service IP: IPFamily:4 Port:31370 Protocol:TCP}根据日志可以看出，proxyMode为空时默认使用iptables
I0718 02:14:51.481549 1 server_others.go:561] "Unknown proxy mode, assuming iptables proxy" proxyMode=""
三、配置IPVS
1、所有节点安装ipvsadm
yum install -y ipvsadm ipset sysstat conntrack libseccomp2、添加配置
cat > /etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
modprobe -- ip_tables
modprobe -- ip_set
modprobe -- xt_set
modprobe -- ipt_set
modprobe -- ipt_rpfilter
modprobe -- ipt_REJECT
modprobe -- ipip
EOFchmod 755 /etc/sysconfig/modules/ipvs.modules
bash /etc/sysconfig/modules/ipvs.modules && lsmod |grep -e ip_vs -e nf_conntrack_ipv43、修改转发模式，将mode模式修改为ipvs
[root@k8s-master ~]# kubectl edit cm kube-proxy -n kube-systemmode: "ipvs"4、重启 kube-proxy
kubectl rollout restart daemonset kube-proxy -n kube-system5、查看kube-proxy状态，正在创建新的pod
[root@k8s-master ~]# kubectl -n kube-system get po
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-5bb5d4f7f4-knlgr 1/1 Running 0 66m
calico-node-2ds8v 1/1 Running 1 (73m ago) 18h
calico-node-6wj9b 1/1 Running 1 (73m ago) 18h
calico-node-pxrk7 1/1 Running 1 (73m ago) 18h
coredns-6d8c4cb4d-fcjr4 1/1 Running 1 (73m ago) 20h
coredns-6d8c4cb4d-l5rtm 1/1 Running 1 (73m ago) 20h
etcd-k8s-master 1/1 Running 1 (73m ago) 20h
kube-apiserver-k8s-master 1/1 Running 1 (73m ago) 20h
kube-controller-manager-k8s-master 1/1 Running 1 (73m ago) 20h
kube-proxy-7kpb9 0/1 ContainerCreating 0 1s
kube-proxy-dtq6r 1/1 Running 1 (73m ago) 20h
kube-proxy-gmg7w 1/1 Running 1 (73m ago) 20h
kube-scheduler-k8s-master 1/1 Running 1 (73m ago) 20h
metrics-server-86597c44b6-j9x82 1/1 Running 1 (73m ago) 17h
metrics-server-86597c44b6-z9qbw 1/1 Running 0 66m6、验证是否切换成功
等个几分钟查看ipvs转发信息
[root@k8s-master ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.17.0.1:31370 rr
-> 10.244.36.73:80 Masq 1 0 0
-> 10.244.36.74:80 Masq 1 0 0
TCP 192.168.30.57:31370 rr
-> 10.244.36.73:80 Masq 1 0 0
-> 10.244.36.74:80 Masq 1 0 0
TCP 10.96.0.1:443 rr
-> 192.168.30.57:6443 Masq 1 0 24
TCP 10.96.0.10:53 rr
-> 10.244.235.195:53 Masq 1 0 0
-> 10.244.235.196:53 Masq 1 0 0
TCP 10.96.0.10:9153 rr
-> 10.244.235.195:9153 Masq 1 0 0
-> 10.244.235.196:9153 Masq 1 0 0
TCP 10.99.203.227:8888 rr
-> 10.244.36.73:80 Masq 1 0 0
-> 10.244.36.74:80 Masq 1 0 0
TCP 10.111.61.136:443 rr
-> 10.244.36.70:443 Masq 1 0 0
-> 10.244.235.194:443 Masq 1 1 0
TCP 10.244.0.0:31370 rr
-> 10.244.36.73:80 Masq 1 0 0
-> 10.244.36.74:80 Masq 1 0 0
TCP 10.244.235.192:31370 rr
-> 10.244.36.73:80 Masq 1 0 0
-> 10.244.36.74:80 Masq 1 0 0
UDP 10.96.0.10:53 rr
-> 10.244.235.195:53 Masq 1 0 0
-> 10.244.235.196:53 Masq 1 0 0 正文到此结束
.jpg)
评论
登录后才能发表评论 登录/注册
0评论