Iman Tumorang
Iman Tumorang
Software Engineer - Blogger - Open Source Enthusiast - Startup Enthusiast
Jul 31, 2019 10 min read

Utilize Open API 3 for the Faster Software Development Process

How I can make a Mock Server, APIClient SDK, and a Live API Documentations with ease in Kubernetes Just using open-source tools that you can get free

Photo by toine G on Unsplash.com
Photo by toine G on Unsplash.com

I’m enthusiastic to open source world. There so many great things and free to use in the open-source world. Let’s say Linux, Kubernetes, Docker, NodeJS, Golang, anything. And this ecosystem always brings new technology and solutions for the better of the world (Software Engineering world).

Today, in my free time. I’m trying to explore a few open-source projects, that might be useful to use. If you ever read my story about API Driven Development practice that we use in Kurio, specifically in my article here: API Driven Development in Kurio: Speeding Up the Development Process with Open API 3. 

You will found terms like Open API 3, Mock Server, SDK Generator. But, in that article, I don’t explain anything like how do we utilize Open API 3 in practice or how do we create a Mock Server, or how we generate the SDKs.

So, what I will tell here is, how to create a Mock server and generate API SDK with free tools (a.k.a Open Source projects). And host it in Kubernetes(K8s) cluster, for my case I will use my own personal Kubernetes cluster.

So all the steps below will use the K8s cluster, and with that, I assume that you already have a live and production-ready of K8s cluster (or at least staging-ready). 

And, for my exploration, I already prepared an example of Open API 3 specification, and I will use this specification for this article. Looks the specification in my Github repo here: https://github.com/bxcodec/tweetor/blob/master/docs/openapi.yaml

So to summarize, there are 3 kinds of things I will do here with using Open API Specification

Creating a Live API Documentations from Open API 3 Specs

Tools to be prepared:

The easiest one is to create a live API Documentations, and with a live API Documentations, we hope every developer can read it and understand what is our API, how it works, the request, response, etc. Actually, there are so many tools for this one. But in this article, I will use the default one provided by Swagger as the creator of Swagger, it’s Swagger UI

They already provide a public docker-image that we can use, so if we have a K8s cluster, we just need to add a deployment for it into our K8s cluster, easy-peasy.

Step-1: Create a Docker Image of Swagger UI

The first step is, we need to create a Docker image that will be used and deployed to our K8s cluster. And don’t forget to use the Swagger UI as the base docker-image.  Here is an example of my Dockerfile.

FROM swaggerapi/swagger-ui:v3.23.1
ADD tweetor.yaml /usr/share/nginx/html/tweetor.yaml
# Add another spec here

  • tweetor.yaml is my Open API specification. And I just need to add it to the base image folder, the complete file can be found here https://github.com/bxcodec/tweetor/blob/master/docs/openapi.yaml

  • Since the base image swaggerapi/swagger-ui:v3.23.1 use the NGinx, what I need to do is just adding my Spec into the NGinx folder, which is in /usr/share/nginx/html/.

And then, I just need to build and push it to my private container registry (I use GCR for my personal registry). 

$ docker build -t asia.gcr.io/kube-xmas/tweetor-docs:latest .
//docker build process will happen here...
$ docker push asia.gcr.io/kube-xmas/tweetor-docs:latest
//docker push process will happen here...
Step-2: Create a Kubernetes Deployment Configuration for Swagger UI

The next step is, create the K8s Deployment component. I made my version here. If you want to copy this, just copy it and change anything that required to change.

Execute and run the deployment on Kubernetes.

$ kubectl apply -f swagger_ui_deployment.yaml
$ kubectl get pods --namespace=tweetor-docs
NAME                             READY   STATUS    RESTARTS   AGE
tweetor-docs-786d889d67-65h45   1/1     Running   0          14m

So, from the deployment above, I only use 3 component of K8s, Namespace, Deployment and Service (Node Port). And the last step is, to add the Ingress component to the service, so every engineer will able to see the documentation.

$ kubectl apply -f ingress_swagger_docs.yaml

Live Swagger UI Documentations

And now the docs will available and accessible by any engineer. And everyone can see the docs.

*Notes: If you encountered a Petstore swagger when visiting your API-Docs, enter the swagger yaml name that you created in the docker file. In my case, I use tweetor.yaml, as you can see in the search-bar/explore. So all the docs you have can be hosted in one docker-container. If you had time, try to change the default swagger file in the index page.

That’s all, it’s simple.

Creating A Lightweight and Blazing Fast Mock Server from Open API 3

Tools to be prepared:

Another trick, that may be useful for Software Development is creating a mock-server. Mock-server means, a dummy server that represents a real server, usually doesn’t have any specific logic. Just as is, can accept any request but the response usually static. 

Why we need mock-server? The more details are already written in my past article here, but just to remind, I’ll tell again.

The simple example I could think is, imagine that we’re working between 2 teams, backend, and frontend. Start on the same sprint. This is when the mock-server really useful. Because, normally, the frontend team will need the API to be ready first, so they can work. But when working on the same sprint, it will be blocked, because the backend still not implemented the API.

So how to create a mock-server with ease?

Step-1: Creating the Dockerfile

I already have a live Kubernetes cluster. And a ready-to-used of Open API 3 Specs https://github.com/bxcodec/tweetor/blob/master/docs/openapi.yaml

And, there is a good tool/library, here: https://github.com/danielgtaylor/apisprout. This is a simple Mock-server generator based on Open API 3 Specification. The code is really simple, written in Golang. If only I got the idea first, that’s would be me who created this tool. But, it’s okay, instead re-invent the wheel, I’ll just use it for my own. 

What I need to do is only creating a Docker image from it, and add my own Open API specification into the docker image. 

This is my Dockerfile.

FROM danielgtaylor/apisprout
ADD tweetor.yaml /data/tweetor.yaml

Yes, just it. Create the docker image, and push it to my docker registry.

$ docker build -t asia.gcr.io/kube-xmas/tweetor-mock:latest .
//docker build process will happen here...
$ docker push asia.gcr.io/kube-xmas/tweetor-mock:latest
//docker push process will happen here...

Just it, the rest is to add a deployment file to Kubernetes.

Steps-2: Creating the Kubernetes Deployment for Mock Server

And the next is adding the K8s deployment. Here is my deployment configuration.

$ kubectl apply -f mock_tweetor_deployment.yaml

And add the ingress to DNS management and after that, I can now access the mock-server.

$ curl mock.tweetor.xyz/tweets
[
  {
    "createdTime": "2018-12-24T09:21:41.827Z",
    "id": "abc-f45def-5sdaf-5636f",
    "text": "Merry Christmast Everyone!!!"
  },
  {
    "createdTime": "2018-12-23T09:21:41.827Z",
    "id": "abc-f45def-5sdaf-5636f",
    "text": "I believe santa will give me a great present"
  },
  {
    "createdTime": "2018-12-22T09:21:41.827Z",
    "id": "abc-f45def-5sdaf-5636f",
    "text": "Hello my secret santa. Thank you!!!"
  }
]

And now, our frontend team will able to develop the frontend using the Mock API.

Generating the HTTP SDK Clients from Open API 3 Specs

Tools to be prepared:

And the last things I wrote here is, how do we utilize Open API Specs to help us to generate HTTP Client SDK. SDK stands for Software Development Kit, means a bunch of library or things that can help us to do something useful and helpful, to ease us to integrate/using some services or tools.

In microservice worlds, where there are so many services live and fight and running to do their jobs. Every service has its endpoint that may differ with each other, but within its differences, they still have the same pattern (I’m talking a RESTfull Microservices). They use HTTP Verbs (GET, POST, PUT, DELETE), they use Status Code (200,201,202,400,401,403,404,500,etc)

And each service may depend more than services. And to connecting each service to another, usually, programmer build their own function to do HTTP request.

service A and B dependent on C

service A and B dependent on C

For an example from the picture above.

Let say that Service A and Service B is connected to Service C. And imagine a programmer build the connector (REST HTTP Client) manually from service A. And that programmer also will build the connector from service B. And imagine there are also so many services that will be connected to service C. And all the connector was built manually.

From the example above, we see here a redundant job. What if the connector we just separate and import them to our project as a library. And what if, instead of building the connector manually, we can generate it automatically?

So, that’s how Open API will be used here. Previously, in Swagger 2 (previous version of Open API 3), this is already common about generating an HTTP Client SDK from swagger. But, in Open API 3, it’s still new. So that’s why I want to write this into this article. 

Well, luckily, there is a great tool has been built by the communities here: https://github.com/OpenAPITools/openapi-generator. So I just want to use it, and again, it’s open-source, and it’s free. And it’s very easy to use. Especially, they already support for CLI, and Docker Image.

Generate SDK With Docker (I just copy it from the Github repository, and it works) 😋

I prefer to use with docker because it can be used in CI/CD, especially if the CI/CD support containerization like Buddy.Works. And also with using Docker, I don’t need to install Java SDK, because if we want to install using CLI, we need to install Java SDK. Since I don’t want to add any extras job, I just use the method using docker. 

  • Change the ./tweetor.yaml with your Open API Specification. 
  • -g: specify the programming language of the generated SDK. The supported programming-language can be seen here. In the example above, I want to generate an HTTP Client SDK for Golang.
  • -o: specify the target folder of the generated SDK. 

This is a sneak peek, the generated HTTP Client SDK. This an example folder structure after the client is generated in Go. 

.
├── README.md
├── api
│   └── openapi.yaml
├── api_tweet.go
├── client.go
├── configuration.go
├── docs
│   ├── Tweet.md
│   └── TweetApi.md
├── git_push.sh
├── go.mod
├── go.sum
├── model_tweet.go
└── response.go

I can’t tell exactly how the code generated, but you can try it on your own, and see the generated SDK. It’s really helpful and helps us develop project faster if we were working on many microservices. 

But the obstacles and future issue that may arise from this generated SDK is:

  • Maintaining of the versioning of target API. Because it’s a generated HTTP Client SDK, so it’s maybe hard to handle versioning. So the optimal solution is after we generate the SDK, we must push to a Git repository, and add git tag to for each generated SDK (actually, the shell script already prepared in the generated folder, so it’s solved :D).

What’s Next?

Actually, there are so many tools that might be useful to us, related to Open API or Swagger. You can find every tool listed here: https://openapi.tools. 

All the process above written is still manually prepared. And the next steps are how to automate all the process with (your chosen) CI/CD, so when an engineer updates the Open API Specification, it will automatically update all the API Documentations, Mock Server and the generated SDK.

I’ve automated all the process I wrote above with CI/CD, I use Buddy.Works, but I still looking the right time to write it later.🤧🤕


comments powered by Disqus