Auto Program OpenAPI (Swagger) Spec on Enroute Universal API Gateway
EnRoute HowTos
Simplifying operations at DevOps speed
As organizations are moving towards a DevOps driven application delivery, the need for infrastructure to keep up at this pace of change is vital. enroutectl provides that flexibility and automation to quickly program Enroute Universal API Gateway for most common use-cases.
In this article, we describe how enroutectl radically simplifies programming the openapi spec on Enroute Universal Gateway
enroutectl is packaged in the enroute-gw docker image under the path /bin/enroutectl
Up and running with OpenAPI spec in 30 seconds
Here we demonstrate how easy it is to get up and running. This example demonstrates running the standalone gateway. Note that similar process can also be followed for kubernetes ingress gateway.
Let us start by running the standalone gateway in host mode. This makes the host network stack accessible to the running container. This does not change anything from functionality perspective. The standalone API gateway container can also be run in non-host mode with similar configuration.
sudo docker run --net=host saarasio/enroute-gw:latest
The above command starts the API Gateway. When run in host mode, the accessible ports are -
Interesting Ports
Envoy Ports
8002 - Envoy stats on 8002/stats
9001 - Envoy admin interface on 9001/admin
Enroute Interface Ports
1323 - REST interface to interact with standalone API gateway
8001 - xDS server listen port for Envoy
Enroute Process Ports
6060 - Debug info for enroute pprof tool
8000 - prometheus metrics for enroute
8888 - GraphQL server used by control plane to configure Enroute
5432 - postgres server when running in stateful mode
Ports used by Rate-Limit service
8003 - RateLimit server listen port
6379 - Runs redis for rate-limit and other Enroute state.
Enroute ports for traffic visibility
9090 - prometheus server
3000 - port to access grafana-server
Enroute ports for traffic
8080 - http traffic
8443 - https traffic
Note that when running in non-host mode, port redirection can be achieved as needed while exposing selected ports. For instance, when running https traffic, we can expose the following ports for operation -
sudo docker run -p 443:8443 -p 3000:3000 -p 80:8080 saarasio/enroute-gw:latest
The above command will run the gateway that will forward traffic ports and expose grafana port to look at the metrics. The rest of the ports are isolated in the container network stack. If direct access to Envoy admin interface is desired, the following command can be used -
sudo docker run -p 443:8443 -p 3000:3000 -p 80:8080 -p 9001:9001 saarasio/enroute-gw:latest
For the current example, we’ll run with –net=host mode and make the host network stack accessible to the API gateway container.
Accessing enroutectl
enroutectl is packaged along with docker container of standalone gateway and can be accessed by opening an interactive shell on the container. Here are the steps to do it.
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b8072f7b4b8c saarasio/enroute-gw:latest "/usr/bin/supervisor…" 24 minutes ago Up 24 minutes silly_bohr
$ docker exec -it silly_bohr /bin/bash
silly_bohr:/bin#
# ls -l enroutectl
-rwxrwxr-x 1 root root 51513918 Jun 29 18:16 enroutectl
silly_bohr:/bin# ./enroutectl
enroutectl can be used to generate declarative configuration for
- Enroute Standalone Gateway
- Kubernetes Ingress Gateway
enroutectl can also read openapi spec to generate config for gateway
Usage:
enroutectl [flags]
enroutectl [command]
Available Commands:
help Help about any command
init Initialize enroutectl
openapi Automatically program Enroute Universal API Gateway using OpenAPI Spec
Flags:
--enroutectl-config string config for enroutectl if no enroutectl.yaml (or enroutectl.json) found in current OR home directory
-h, --help help for enroutectl
--verbose be verbose when running
Use "enroutectl [command] --help" for more information about a command.
The petstore example OpenAPI (swagger) spec is under the examples directory in this container -
silly_bohr:/examples# ls -l /examples
total 16
-rw-rw-r-- 1 postgres postgres 13843 Jun 29 18:16 petstore.json
To program the API gateway, simply run enroutectl with petstore.json as an argument
bashsilly_bohr:/bin#./enroutectl openapi --openapi-spec /examples/petstore.json --to-standalone-url http://localhost:1323/
Note that enroutectl uses the REST interface on port 1323 to program the OpenAPI spec on the gateway. We can query the gateway state using dump verb. Note that we use the gateway name gw to dump state -
bash$curl -s localhost:1323/proxy/dump/gw | jq
{
"data": {
"saaras_db_proxy": [
{
"proxy_globalconfigs": [],
"proxy_name": "gw",
"proxy_services": [
{
"service": {
"fqdn": "*",
"routes": [
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "POST"
}
],
"prefix": "/v2/user/createWithArray$"
},
"route_filters": [],
"route_name": "openapi-createUsersWithArrayInput-3ae533b2b5-d0fee9541d",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "POST"
}
],
"prefix": "/v2/user/createWithList$"
},
"route_filters": [],
"route_name": "openapi-createUsersWithListInput-7011820c21-69bee49818",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "DELETE"
}
],
"prefix": "/v2/store/order/(?P<orderId>.*)$"
},
"route_filters": [],
"route_name": "openapi-deleteOrder-a0e2a93d66-6e736e9382",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "DELETE"
}
],
"prefix": "/v2/pet/(?P<petId>.*)$"
},
"route_filters": [],
"route_name": "openapi-deletePet-35222ce8e8-9fe883ae9e",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "DELETE"
}
],
"prefix": "/v2/user/(?P<username>.*)$"
},
"route_filters": [],
"route_name": "openapi-deleteUser-ea777d883f-d17f08eca4",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "GET"
}
],
"prefix": "/v2/pet/findByStatus"
},
"route_filters": [],
"route_name": "openapi-findPetsByStatus-090b3721b1-d1a5b91889",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "GET"
}
],
"prefix": "/v2/pet/findByTags"
},
"route_filters": [],
"route_name": "openapi-findPetsByTags-224f0e4d2d-b45f3e6a76",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "GET"
}
],
"prefix": "/v2/store/inventory"
},
"route_filters": [],
"route_name": "openapi-getInventory-85650e00a2-b0f5441374",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "GET"
}
],
"prefix": "/v2/store/order/(?P<orderId>.*)$"
},
"route_filters": [],
"route_name": "openapi-getOrderById-a0e2a93d66-3afe640f9d",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "GET"
}
],
"prefix": "/v2/pet/(?P<petId>.*)$"
},
"route_filters": [],
"route_name": "openapi-getPetById-35222ce8e8-2e85250528",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "GET"
}
],
"prefix": "/v2/user/(?P<username>.*)$"
},
"route_filters": [],
"route_name": "openapi-getUserByName-ea777d883f-1963d3289a",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "GET"
}
],
"prefix": "/v2/user/login"
},
"route_filters": [],
"route_name": "openapi-loginUser-c69fdf5bba-4aeb13de96",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "GET"
}
],
"prefix": "/v2/user/logout"
},
"route_filters": [],
"route_name": "openapi-logoutUser-6f8b3a245c-3713915009",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "POST"
}
],
"prefix": "/v2/store/order$"
},
"route_filters": [],
"route_name": "openapi-placeOrder-f930c1447d-13d63d6189",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "PUT"
}
],
"prefix": "/v2/pet$"
},
"route_filters": [],
"route_name": "openapi-updatePet-v2pet$-147d5289ee",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "POST"
}
],
"prefix": "/v2/pet/(?P<petId>.*)$"
},
"route_filters": [],
"route_name": "openapi-updatePetWithForm-35222ce8e8-26bb2fc896",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "PUT"
}
],
"prefix": "/v2/user/(?P<username>.*)$"
},
"route_filters": [],
"route_name": "openapi-updateUser-ea777d883f-97c79328fd",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "POST"
}
],
"prefix": "/v2/pet/(?P<petId>.*)/uploadImage$"
},
"route_filters": [],
"route_name": "openapi-uploadFile-67b0e00cbb-0b5b39c320",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "POST"
}
],
"prefix": "/v2/pet$"
},
"route_filters": [],
"route_name": "openapi-addPet-v2pet$-b021280030",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
},
{
"config_json": {
"header": [
{
"header_name": ":method",
"header_value": "POST"
}
],
"prefix": "/v2/user$"
},
"route_filters": [],
"route_name": "openapi-createUser-v2user$-8185d9fbf4",
"route_prefix": "",
"route_upstreams": [
{
"upstream": {
"upstream_ip": "petstore.swagger.io",
"upstream_name": "openapi-upstream-petstore.swagger.io",
"upstream_port": 80
}
}
]
}
],
"service_name": "openapi-petstore.swagger.io",
"service_secrets": [],
"service_filters": []
}
}
]
}
]
}
}
Attaching advanced rate-limiting to securing APIs.
Enroute uses a filter based architecture to attach additional functionality at the global level and route level. Note the route_filters: [] is an empty list above. Filters can be attached here. Eg: attaching a rate-limit filter here will apply rate-limits to this route. Similarly the service_filters:[] is an empty list above. Filters can be attached here for service level traffic. An example of such filter is the lua filter.
For a more detailed example on rate-limit, refer to the article on - getting started with advanced rate-limiting.
enroutectl also provides the ability to follow a similar workflow to program OpenAPI spec on kubernetes ingress gateway which also follows a similar filter architecture for configuration. Refer to the article on getting started with kubernetes ingress gateway for more details
Using Prometheus and Grafana to monitor Gateway state
Enroute Gateway is built using Envoy proxy and supports programming the OpenAPI Swagger spec on the gateway automatically.
Enroute Standalone comes packaged with Prometheus and Grafana to monitor the gateway. It provides information with request-level stats, latency and overall health of the system. The dashboard can also be tweaked as per the requirement of the user and notifications can be programmed to be notified on different conditions. As mentioned above, the dashboard can be accessed on port 3000 with default login password admin/admin
Conclusion
enroutectl can be used to program OpenAPI spec on Enroute Universal Gateway. This works for both Standalone Gateway and Kubernetes Ingress Gateway/ Once programmed, uniform policy can be enforced either at the global level or per-route level by attaching filters.
enroutectl simplifies operations and helps automate configuring Enroute Universal Gateway. Both the kubernetes ingress gateway and standalone gateway can be configured.
EnRoute Gateway works in any cloud environment that runs a Kubernetes Cluster. EnRoute gateway works with public cloud provider Kubernetes or any other distribution.
This guide provides details on running EnRoute gateway on AWS, most common deployment configuration and different options available in AWS environment.
In AWS environment, EnRoute service annotations can program the AWS load balancer. AWS has several different loadbalancers that differ in the functionality provided. We explore different configurations to work with AWS LoadBalancers, the details of configuration and how good architectural practices.