gloo基本知识

gloo基础知识入门

Architechture(架构)

Gloo通过Envoy XDS gRPC API来动态更新Envoy配置, 更方便的控制Envoy Proxy, 并保留扩展性..本质是一个Envoy xDS配置翻译引擎, 为Envoy提供高级配置(及定制的Envoy过滤器).它监控各种配置源的更新,并立即响应通过gRPC更新给Envoy.

Component Architechture

architechture

Discovery Architechture

discovery Architechture

Gloo支持k8s, consul的Upstream discovery, 还要以自己开发自定义的组件.

Deployment Architecture

Gloo可以在各种基础设施上以多种方式部署, 推荐是使用kubernets,它可以简化操作.但并不一定要部署在kubernets上.

sharded-gateway

点击查看更多的部署方式.

Concepts(核心概念)

通过下面这个简单的vritual services来理解gloo的核心概念:

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: default
  namespace: gloo-system
spec:
  virtualHost:
    domains:
    - '*'
    routes:
    - matchers:
      - prefix: /
      routeAction:
        single:
          upstream:
            name: my-upstream
            namespace: gloo-system

Vritual Services

Routes

Matchers

Matchers支持2种请求类型

Destinations

Upstreams

Upstreams定义了路由规则最终去向(Destinations).一般是通过服务发现(services discovery)自动加入,最基本的Upstream类型就是静态的Upstream: 它只需要告诉Gloo一个静态主机或dns名列表.复杂的Upstream有kubernets及AWS lambda upstream.

一个简单的Upsteams例子

apiVersion: gloo.solo.io/v1
kind: Upstream
metadata: 
  labels:
    discovered_by: kubernetesplugin
  name: default-redis-6379
  namespace: gloo-system
spec:
  discoveryMetadata: \{\}
  kube:
    selector:
      gloo: redis
    serviceName: redis
    serviceNamespace: gloo-system
    servicePort: 6379
status:
  reported_by: gloo
  state: 1 # Accepted

Functions

有些Upstream支持函数destinations, 比如: 我们可以在Upstream中添加一些HTTP函数.让Gloo根据这些函数把检验请求参数,然后将传入的请求格式化为Upstream服务所期望的参数.一个简单的示例:

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: default
  namespace: default
spec:
  virtualHost:
    domains:
    - '*'
    routes:
    - matchers:
       - prefix: /petstore/findWithId
      routeAction:
        single:
          destinationSpec:
            rest:
              functionName: findPetById
              parameters:
                headers:
                  :path: /petstore/findWithId/\{id\}
          upstream:
            name: petstore
            namespace: gloo-system
      options:
        prefixRewrite: /api/pets

调用curl http://url/petstore/findWithId/100会路由到函数findPetById(id)中,其中Id的是通过parameters中的规则赋值的.

Secrets

Traffic Management

Gloo核心是一个强大的路由引擎.可以处理API到API的简单路由.也可以处理HTTP到gRPC协议转换.

Request -> Router -> Destinations(Upstream)

得益于envoy proxy灵活的扩展性,gloo中在上面每一个环节中支持的类型都非常多样. 下面以HTTP REST API为例子,演示一下基础路由功能.

Gloo Configuration

Gloo配置布局分3层: Gateway listeners, Virtual Services, Upstreams.大多数情况,我们只与VirtualServices进行交互.可以通过它配置暴露给Gateway的API细节,还可以配置具体的路由规则. overview

Upstream代表后端服务, Gateway控制监听端口,请求的入口.

PetStore精确匹配

部署一个完整的PetStore应用.路由规则matcher使用Path精确匹配.

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: petstore
  name: petstore
  namespace: default
spec:
  selector:
    matchLabels:
      app: petstore
  replicas: 1
  template:
    metadata:
      labels:
        app: petstore
    spec:
      containers:
      - image: soloio/petstore-example:latest
        name: petstore
        ports:
        - containerPort: 8080
          name: http
---
apiVersion: v1
kind: Service
metadata:
  name: petstore
  namespace: default
  labels:
    service: petstore
spec:
  ports:
  - port: 8080
    protocol: TCP
  selector:
    app: petstore

YAML中定义了使用soloio/petstore-example:latest镜像创建一个app,并以8080端口对集群内服务.使用kubectl执行.

$ kubect apply -f ./petstore.ymal
deployment.extensions/petstore created
service/petstore created

​ 检查服务是否正常启动:

$ kubectl -n default get pods 
NAME                        READY   STATUS    RESTARTS   AGE
petstore-6f67bbbb74-tg872   1/1     Running   0          20h
$ kubectl -n default get svc petstore
NAME       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
petstore   ClusterIP   10.105.234.177   <none>        8080/TCP   20h

因为是k8s服务,所以会通过服务发现自动注册到gloo中.使用gloo查看Upsteam.

$ glooctl get upstreams

这个可以看到所有运行中的upstreams.有一些是系统,比如gloo-system-gateway-443,你也可以在里面找到

default-petstore-8080 Kubernetes | Accepted | svc name:      petstore         |

查看upstream的详细情况:

$ glooctl get upstream default-petstore-8080 --output yaml

默认情况下,Upstream非常简单。它代表了一个特定的kubernetes服务, 但petstore应用是一个swagger服务。Gloo可以发现这个swagger规范,但默认情况下,为了提高性能,Gloo的函数发现功能被关闭了。为了在我们的petstore上启用函数发现服务(fds),我们需要给命名空间打上function_discovery标签。

$ kubectl label namespace default  discovery.solo.io/function_discovery=enabled
apiVersion: gloo.solo.io/v1
kind: Upstream
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      \{"apiVersion":"v1","kind":"Service","metadata":\{"annotations":\{\},"labels":\{"service":"petstore"\},"name":"petstore","namespace":"default"\},"spec":\{"ports":[\{"port":8080,"protocol":"TCP"\}],"selector":\{"app":"petstore"\}\}\}
  creationTimestamp: null
  generation: 4
  labels:
    discovered_by: kubernetesplugin
    service: petstore
  name: default-petstore-8080
  namespace: gloo-system
  resourceVersion: "5488"
spec:
  discoveryMetadata: \{\}
  kube:
    selector:
      app: petstore
    serviceName: petstore
    serviceNamespace: default
    servicePort: 8080
    serviceSpec:
      rest:
        swaggerInfo:
          url: http://petstore.default.svc.cluster.local:8080/swagger.json
        transformations:
          addPet:
            body:
              text: '\{"id": \{\{ default(id, "") \}\},"name": "\{\{ default(name, "")\}\}","tag":
                "\{\{ default(tag, "")\}\}"\}'
            headers:
              :method:
                text: POST
              :path:
                text: /api/pets
              content-type:
                text: application/json
          deletePet:
            headers:
              :method:
                text: DELETE
              :path:
                text: /api/pets/\{\{ default(id, "") \}\}
              content-type:
                text: application/json
          findPetById:
            body: \{\}
            headers:
              :method:
                text: GET
              :path:
                text: /api/pets/\{\{ default(id, "") \}\}
              content-length:
                text: "0"
              content-type: \{\}
              transfer-encoding: \{\}
          findPets:
            body: \{\}
            headers:
              :method:
                text: GET
              :path:
                text: /api/pets?tags=\{\{default(tags, "")\}\}&limit=\{\{default(limit,
                  "")\}\}
              content-length:
                text: "0"
              content-type: \{\}
              transfer-encoding: \{\}
status:
  reported_by: gloo
  state: 1

Endpoints是由Gloo的Function Discovery(fds)服务发现的。之所以能够做到这一点,是因为petstore实现了OpenAPI(在petstore-svc/swagger.json处发现了一个Swagger JSON文档).

Prefix前置匹配

新增路由/find-pet/\{id\} -> default-petstore-8080/api/pets/\{id\}, 把Id传到对应HTTP rest API中函数入参.

glooctl add route \
  --path-prefix /find-pet \
  --dest-name default-petstore-8080 \
  --prefix-rewrite /api/pets

这就是把/find-pet/\{id\} -> default-petstore-8080/api/pets/\{id\}

使用glooctl 查看virtual service的对应的配置

    - matchers:
      - prefix: /find-pet
      options:
        prefixRewrite: /api/pets
      routeAction:
        single:
          upstream:
            name: default-petstore-8080
            namespace: gloo-system

同时因为这个是提供的是OPENAPI方式,上面的destination也可以指定函数来确定(达到一样的路由效果):

glooctl add route \
  --path-prefix /pets \
  --dest-name default-petstore-8080 \
  --rest-function-name findPetById \
  --rest-parameters :path='/pets/\{id\}'

这就是把/pets/\{id\} -> default-petstore-8080中的findPetByIdrest函数中.函数的入参id通过--rest-parameters中取.

使用glooctl 查看virtual service的具体配置.

$ glooctl get vs default --output yaml
...
  - matchers:
    - prefix: /pets
    routeAction:
      single:
        destinationSpec:
          rest:
            functionName: findPetById
            parameters:
              headers:
                :path: /pets/\{id\}
        upstream:
          name: default-petstore-8080
          namespace: gloo-system
...
$ glooctl get upstream --name default-petstore-8080 --output yaml
---
....
  serviceSpec:
    rest:
      swaggerInfo:
        url: http://petstore.default.svc.cluster.local:8080/swagger.json
      transformations:
        addPet:
          body:
            text: '\{"id": \{\{ default(id, "") \}\},"name": "\{\{ default(name, "")\}\}","tag":
              "\{\{ default(tag, "")\}\}"\}'
          headers:
            :method:
              text: POST
            :path:
              text: /api/pets
            content-type:
              text: application/json
        deletePet:
          headers:
            :method:
              text: DELETE
            :path:
              text: /api/pets/\{\{ default(id, "") \}\}
            content-type:
              text: application/json
        findPetById:
          body: \{\}
          headers:
            :method:
              text: GET
            :path:
              text: /api/pets/\{\{ default(id, "") \}\}
            content-length:
              text: "0"
            content-type: \{\}
            transfer-encoding: \{\}
        findPets:
          body: \{\}
          headers:
            :method:
              text: GET
            :path:
              text: /api/pets?tags=\{\{default(tags, "")\}\}&limit=\{\{default(limit, "")\}\}
            content-length:
              text: "0"
            content-type: \{\}
            transfer-encoding: \{\}
....

可以看到Virtual Service中的destinationSpec 与Upstream中serviceSpec对应上了.都是findPetById(Id),所以路由才能通.

$ curl "$(glooctl proxy url)/pets/1"
\{"id":1,"name":"Dog","status":"available"\}

注意: paramters中的:path是精确匹配的.如果你把url最后多写一个/, 变成/pets/1/,那就会

curl "$(glooctl proxy url)/pets/1/"
[\{"id":1,"name":"Dog","status":"available"\},\{"id":2,"name":"Cat","status":"pending"\}]

这里返回了所有pets,因为多了/后rest-parameters里面的:path是/pets/\{id\},多了/后变得无法匹配,所以相当于没有传Id,导致请求的是findPetById(""),此函数返回的是所有pets.

regex正则匹配

由于find-pet路由没有增加对查询Id的范围限制,所以我们可以把它使用regex作限制.

glooctl add route \
  --path-regex '/find-pet-1/[1-9]' \
  --dest-name default-petstore-8080 \
  --rest-function-name findPetById \
  --rest-parameters :path='/find-pet-1/\{id\}'
$ curl http://localhost:80/find-pet-1/1
\{"id":1,"name":"Dog","status":"available"\}
$ curl http://localhost:80/find-pet-1/11
\{"code":404,"message":"path /api/pets-1/11 was not found"\}\%

可以看到参数已经被限制在1-10之间了.

前面增加Router都是通过glooctl add route 命令行来完成的..下面我们再通过YAML配置文本来做管理. 误区: 由于PetStore是k8s中的一个services, 他可以直接通过命令

kubectl edit service -n default petstore

打开编辑器直接编辑, 但是这个打开的内容是没有Gloo附加在上面的路由信息的.路由信息存在vritual service里面,所以也不能在这里编辑.你通过 kubectl get service -n default petstore --output yaml

apiVersion: v1
kind: Service
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      \{"apiVersion":"v1","kind":"Service","metadata":\{"annotations":\{\},"labels":\{"service":"petstore"\},"name":"petstore","namespace":"default"\},"spec":\{"ports":[\{"port":8080,"protocol":"TCP"\}],"selector":\{"app":"petstore"\}\}\}
  creationTimestamp: "2020-04-22T14:32:09Z"
  labels:
    service: petstore
  name: petstore
  namespace: default
  resourceVersion: "729"
  selfLink: /api/v1/namespaces/default/services/petstore
  uid: f55adfc3-7181-414f-809d-b29cf5e163b7
spec:
  clusterIP: 10.105.234.177
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: petstore
  sessionAffinity: None
  type: ClusterIP
status:
  loadBalancer: \{\}

可以看到在这个k8s的service中根本没有我们刚加入的gloo中的routers.

所以我们只能在gloo中的virtual service中找到routers编辑.

得到virtual service配置:glooctl get vs default --output kube-yaml

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      \{"apiVersion":"gateway.solo.io/v1","kind":"VirtualService","metadata":\{"annotations":\{\},"creationTimestamp":null,"generation":48,"name":"default","namespace":"gloo-system","resourceVersion":"93283"\},"spec":\{"virtualHost":\{"domains":["*"],"routes":[\{"matchers":[\{"prefix":"/pets"\}],"routeAction":\{"single":\{"destinationSpec":\{"rest":\{"functionName":"findPetById","parameters":\{"headers":\{":path":"/pets/\{id\}"\}\}\}\},"upstream":\{"name":"default-petstore-8080","namespace":"gloo-system"\}\}\}\},\{"matchers":[\{"exact":"/all-pets"\}],"options":\{"prefixRewrite":"/api/pets"\},"routeAction":\{"single":\{"upstream":\{"name":"default-petstore-8080","namespace":"gloo-system"\}\}\}\},\{"matchers":[\{"regex":"/add-pet/[1-9]/[a-z]\{2,10\}/(pending|available)"\},\{"methods":["GET"]\}],"routeAction":\{"single":\{"destinationSpec":\{"rest":\{"functionName":"addPet","parameters":\{"headers":\{":path":"/add-pet/\{id\}/\{name\}/\{tag\}"\}\}\}\},"upstream":\{"name":"default-petstore-8080","namespace":"gloo-system"\}\}\}\}]\}\},"status":\{"reported_by":"gateway","state":1,"subresource_statuses":\{"*v1.Proxy.gloo-system.gateway-proxy":\{"reported_by":"gloo","state":1\}\}\}\}
  creationTimestamp: null
  generation: 73
  name: default
  namespace: gloo-system
  resourceVersion: "99511"
spec:
  virtualHost:
    domains:
    - '*'
    routes:
    - matchers:
      - regex: /find-pet-1/[1-9]
      routeAction:
        single:
          destinationSpec:
            rest:
              functionName: findPetById
              parameters:
                headers:
                  :path: /find-pet-1/\{id\}
          upstream:
            name: default-petstore-8080
            namespace: gloo-system
    - matchers:
      - prefix: /pets
      routeAction:
        single:
          destinationSpec:
            rest:
              functionName: findPetById
              parameters:
                headers:
                  :path: /pets/\{id\}
          upstream:
            name: default-petstore-8080
            namespace: gloo-system
    - matchers:
      - prefix: /find-pet
      options:
        prefixRewrite: /api/pets
      routeAction:
        single:
          upstream:
            name: default-petstore-8080
            namespace: gloo-system
    - matchers:
      - exact: /all-pets
      options:
        prefixRewrite: /api/pets
      routeAction:
        single:
          upstream:
            name: default-petstore-8080
            namespace: gloo-system
status:
  reported_by: gateway
  state: 1
  subresource_statuses:
    '*v1.Proxy.gloo-system.gateway-proxy':
      reported_by: gloo
      state: 1

复制后保存为yaml文件,并在router结尾中增加路由规则.

- matchers:
      - regex: /add-pet/[1-9]/[a-z]\{2,10\}/(pending|available)      
      routeAction:
        single:
          destinationSpec:
            rest:
              functionName: addPet
              parameters:
                headers:
                  :path: /add-pet/\{id\}/\{name\}/\{tag\}
          upstream:
            name: default-petstore-8080
            namespace: gloo-system

这个命令会直接通过编辑器打开它的YAML配置文件.我们直接加入新路由配置后保存.

这个命令把path上的参数匹配后传到了destination中的addPet的body中,完成了路由regx及body transformation.

Tips:为了做好版管理,所以用get得到的YAML格式中有一个字段resourceVersion.如果你apply同一个文件2次,第二次会出错.你必须重新get最新的YAML文件以获取新的resourceVersion.

删除route

你可以使用glooctl删除不需要的路由规则.

glooctl rm route -i

-i----interactive模式,一步步通过提示删除路由

Matcher陈了上面说过的对Path进行匹配外,还可以对Header, Query Parameter, Method也作同样的匹配.

Header路由示例

- matchers:
         - headers:
            - name: version
              value: "v1"
            - name: os_type
            - name: type
              regex: true
              value: "[a-z]\{1\}"
            - name: Istest
              invertMatch: true
            - name: Istrace
              value: 0
              invertMatch: true
           prefix: /        

各个条件之间是与(and)的关系.上面就是: version=v1 and 必须有os_type字段 and type在小写的a-z之间and 没有Istest字段andIstrace必须有且不等于0

Query Parameter路由示例

- matchers:
   - queryParameters:
      - name: os
        value: ios
      - name: location
      - name: userno
        regex: true
        value: "a[a-z]\{9\}"
      prefix: /

os是ios and 必须有location字段 and userno 是以a开头,全小写,共10位的用户.

Method路由示例

- matchers:
   - methods:
      - GET
     prefix: /

限制HTTP Method,可以指定一个列表.

Transformations

Gloo可以在请求到达到指定的Service前把请求进行任意修改(requestTransformation),也可以在应答返回给Client之前把应答进行任意修改(responseTransformation).

Transformations属性定义在Virtual Services, 你可以在它的VritualHosts, Routes, WeightedDestionations的属性下定义Transformations, 它们的格式都是一样的.唯一的区别是作用的范围大小不一样.所有的子属性都会受到对应的transformations影响.如果你要同时在VritualHostsRoutes都定义了2个transformations,那Routers不会合并VritualHosts,两者各不影响.

transformations:
  clearRouteCache: bool
  requestTransformation: \{\}
  responseTransformation: \{\}

transformationTemplate

transformationTemplate:
  parseBodyBehavior: \{\}
  ignoreErrorOnParse: bool
  extractors:  \{\}
  headers: \{\}
  # Only one of body, passthrough, and mergeExtractorsToBody can be specified
  body: \{\} 
  passthrough: \{\}
  mergeExtractorsToBody: \{\}
  dynamicMetadataValues: []
  advancedTemplates: bool

Templates是Transformation的核心,本质就是利用上面这几个关键字对Request/Response的所有内容进行任意转换,写出一个你想要的转换函数API.

Update Response Code

很多Rest API的设计会把Response请求都返回200 ok, 业务出错的情况则在body里面规定一个ret返回码,和err_msg字段.比如:腾讯公开的API都是这样设计的:https://wiki.open.qq.com/wiki/v3/user/get_info 如果我们不希望把具体的业务错返回用户,则可以写一个transformations只有body里面有ret不为0,则返回400.

options:
  transformations:
    responseTransformation:
      transformationTemplate:
        headers:              
          ":status":
           text: '\{\% if default(ret, 0) != 0 \%\}400\{\% else \%\}\{\{ header(":status") \}\}\{\% endif \%\}'

这里可以直接使用ret变量,是因为前面默认是以json解析body,然后inja template支持这样的语法取json body.

Extrac Query Parameters

把QueryString变成header里面的kv.

options:
  transformations:
    requestTransformation:
      transformationTemplate:
        extractors:              
          foo: #extractors的名字,相当于变量名
            # The :path pseudo-header contains the URI
            header: ':path'
            # Use a nested capturing group to extract the query param
            regex: '(.*foo=([^&]*).*)'
            subgroup: 2
          bar: #extractors的名字,相当于变量名                
            header: ':path'                
            regex: '(.*bar=([^&]*).*)'
            subgroup: 2            
        headers:
          foo:
            text: '\{\{ foo \}\}'
          bar:
            text: '\{\{ bar \}\}'

header中使用:path是因为envoy使用的是http2的协议来做transformat,所以如果你使用的是http1.1的话,就需要使用 :path. http2的path就是header中的:path字段.

curl "http:xxxxx/get?foo=foo-value&bar=bar=bar-value"
#转换后效果相当于
curl -H foo=foo-value -H bar=bar-value "http:xxxxx/get"

Update Request Path

options:
  transformations:
    requestTransformation:
      transformationTemplate:
        headers:
          # By updating the :path pseudo-header, we update the request URI
          ":path":
            text: '\{\% if header("foo") == "bar" \%\}/post\{\% else \%\}\{\{ header(":path") \}\}\{\% endif \%\}'          
          ":method":
            text: '\{\% if header("foo") == "bar" \%\}POST\{\% else \%\}\{\{ header(":method") \}\}\{\% endif \%\}'

这个比较简单,都没有用到extractor.效果相当于: 如果header有字段foo=bar则无把path改成/post.并把http方法也改成POST.