Ingress Lab（Ingress-Nginx)

环境准备

GitHub - kubernetes/ingress-nginx: NGINX Ingress Controller for Kubernetes

本地环境使用NodePord，可以用Bare Metal cluster的yaml

https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.1.1/deploy/static/provider/baremetal/deploy.yaml

注意其中的images：k8s.gcr.io/ingress-nginx/controller:v1.1.1@sha256:0bc88eb15f9e7f84e8e56c14fa5735aaa488b840983f87bd79b1054190e660de

需要使用国内的源或自行下载：

root@worker:~# docker login --username=dewin.alsion@gmail.com registry.cn-hangzhou.aliyuncs.com
Password:
WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
root@worker:~# docker push registry.cn-hangzhou.aliyuncs.com/etaon/ingress-nginx-controller:v1.1.1
The push refers to repository [registry.cn-hangzhou.aliyuncs.com/etaon/ingress-nginx-controller]
An image does not exist locally with the tag: registry.cn-hangzhou.aliyuncs.com/etaon/ingress-nginx-controller
root@worker:~# docker pull registry.cn-hangzhou.aliyuncs.com/etaon/ingress-nginx-controller:v1.1.1
v1.1.1: Pulling from etaon/ingress-nginx-controller
a0d0a0d46f8b: Pull complete
3aae86482564: Pull complete
c0d03781abb3: Pull complete
0297e2ef8f7f: Pull complete
866a68ce3c13: Pull complete
95659ee0b3b1: Pull complete
004aaf0049fd: Pull complete
79acb83cc767: Pull complete
152f0f0a27e1: Pull complete
256313a1425c: Pull complete
81e081df9e03: Pull complete
1b6179f6007a: Pull complete
3e4300421da9: Pull complete
Digest: sha256:402333bae4fa45ff918939e77e6c836e0c39f53cafba734a91503fd38190c032
Status: Downloaded newer image for registry.cn-hangzhou.aliyuncs.com/etaon/ingress-nginx-controller:v1.1.1
registry.cn-hangzhou.aliyuncs.com/etaon/ingress-nginx-controller:v1.1.1
root@worker:~#

zyi@cp:~$ kubectl apply -f ingress-deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created

zyi@cp:~$ kubectl get po -n ingress-nginx
NAME                                        READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-gg9nk        0/1     Completed   0          94s
ingress-nginx-admission-patch-gcckz         0/1     Completed   0          94s

zyi@cp:~$ kubectl get svc -A
NAMESPACE              NAME                                 TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
default                front-app                            NodePort    10.101.172.40    <none>        8000:31583/TCP               19m
default                kubernetes                           ClusterIP   10.96.0.1        <none>        443/TCP                      30d
ingress-nginx          ingress-nginx-controller             NodePort    10.105.172.147   <none>        80:31283/TCP,443:31788/TCP   113s
ingress-nginx          ingress-nginx-controller-admission   ClusterIP   10.96.168.59     <none>        443/TCP                      113s

部署好以后，Ingress对外使用：80:31283/TCP,443:31788/TCP

实验拓扑

kubectl create deployment front-app --image=network-multitool:v1.1 --replicas=3
kubectl create deployment back-app --image=network-multitool:v1.1 --replicas=2

kubectl expose deployment front-app --port=80 --target-port=80 --type=NodePort
kubectl expose deployment back-app --port=8080 --target-port=80 --type=NodePort

zyi@cp:~$ kubectl get po -owide
NAME                         READY   STATUS    RESTARTS   AGE   IP                NODE     NOMINATED NODE   READINESS GATES
back-app-6b5f87d447-d4dgj    1/1     Running   0          21h   192.168.171.109   worker   <none>           <none>
back-app-6b5f87d447-v8tqz    1/1     Running   0          21h   192.168.171.108   worker   <none>           <none>
front-app-544b8cdb7f-7hnf7   1/1     Running   0          24h   192.168.171.103   worker   <none>           <none>
front-app-544b8cdb7f-dmdvz   1/1     Running   0          24h   192.168.171.104   worker   <none>           <none>
front-app-544b8cdb7f-kf9x9   1/1     Running   0          24h   192.168.171.102   worker   <none>           <none>
zyi@cp:~$ kubectl get svc -owide
NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE   SELECTOR
back-app     NodePort    10.100.37.208   <none>        8080:30439/TCP   21h   app=back-app
front-app    NodePort    10.101.172.40   <none>        80:31583/TCP     24h   app=front-app
kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP          31d   <none>

测试web服务：

/ # curl 10.101.172.40
Praqma Network MultiTool (with NGINX) - front-app-544b8cdb7f-kf9x9 - 192.168.171.102
Front-APP
/ # curl 10.100.37.208:8080
Praqma Network MultiTool (with NGINX) - back-app-6b5f87d447-v8tqz - 192.168.171.108
Back-app

实验步骤

按域名访问

设计目标是

front.etaon.top—→front-app(service)

back.etaon.top—→back-app(service)

官方配置参考

ingress-nginx/ingress.yaml at main · kubernetes/ingress-nginx

apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:
  name: ingress-host-bar
spec:
  ingressClassName: nginx
  rules:
  - host: "front.etaon.top"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: front-app
            port:
              number: 80
  - host: "back.etaon.top"
    http:
      paths:
      - pathType: Prefix
        path: "/"  
        backend:
          service:
            name: back-app
            port:
              number: 8080

效果如下：

Rewrite

重写可以参考官方文档：

Rewrite - NGINX Ingress Controller

metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
...
    http:
      paths:
      - path: /something(/|$)(.*)

捕获的组按时间顺序以 $1、$2 … $n 的形式保存在编号的占位符中。这些占位符可以用作 rewrite-target 注释中的参数
(.*) 捕获的任何字符都将分配给占位符 $2，然后将其用作 rewrite-target 注释中的参数

上面的Ingress定义将导致以下重写：

rewrite.bar.com/something rewrites to rewrite.bar.com/
rewrite.bar.com/something/ rewrites to rewrite.bar.com/
rewrite.bar.com/something/new rewrites to rewrite.bar.com/new

改写yaml，效果要求：back.etaon.top:31283/nginx—>back.etaon.top:31283/

apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
  name: ingress-host-bar
spec:
  ingressClassName: nginx
  rules:
  - host: "front.etaon.top"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: front-app
            port:
              number: 80
  - host: "back.etaon.top"
    http:
      paths:
      - pathType: Prefix
        path: "/nginx(/|$)(.*)"  
        backend:
          service:
            name: back-app  
            port:
              number: 8080

速率限制

官方参考

Annotations - NGINX Ingress Controller

可以定义很多规则：

nginx.ingress.kubernetes.io/limit-connections: number of concurrent connections allowed from a single IP address. A 503 error is returned when exceeding this limit.
nginx.ingress.kubernetes.io/limit-rps: number of requests accepted from a given IP each second. The burst limit is set to this limit multiplied by the burst multiplier, the default multiplier is 5. When clients exceed this limit, limit-req-status-code default: 503 is returned.
nginx.ingress.kubernetes.io/limit-rpm: number of requests accepted from a given IP each minute. The burst limit is set to this limit multiplied by the burst multiplier, the default multiplier is 5. When clients exceed this limit, limit-req-status-code default: 503 is returned.
nginx.ingress.kubernetes.io/limit-burst-multiplier: multiplier of the limit rate for burst size. The default burst multiplier is 5, this annotation override the default multiplier. When clients exceed this limit, limit-req-status-code default: 503 is returned.
nginx.ingress.kubernetes.io/limit-rate-after: initial number of kilobytes after which the further transmission of a response to a given connection will be rate limited. This feature must be used with proxy-buffering enabled.
nginx.ingress.kubernetes.io/limit-rate: number of kilobytes per second allowed to send to a given connection. The zero value disables rate limiting. This feature must be used with proxy-buffering enabled.
nginx.ingress.kubernetes.io/limit-whitelist: client IP source ranges to be excluded from rate-limiting. The value is a comma separated list of CIDRs.

选用limit-rps测试：每秒从给定 IP 接受的请求数。突发限制设置为此限制乘以突发倍数，默认倍数为 5。当客户端超过此限制时，返回 limit-req-status-code default: 503。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-limit-rate
  annotations:
    nginx.ingress.kubernetes.io/limit-rps: "1"
spec:
  ingressClassName: nginx
  rules:
  - host: "live.etaon.top"
    http:
      paths:
      - pathType: Exact
        path: "/"
        backend:
          service:
            name: back-app
            port:
              number: 8080

正常情况：

加快点击刷新：

出现503错误。