Testing custom Github Actions relying on KinD

It has been a while since I’ve written an article here. Last few months have been busy.

I’m still working on my “Elixir and Kubernetes : A love story” series, but recently, I have been prioritizing automating the deployment of Kubirds on our infrastructure.

🚀 Deploying, the GitOps way

We have a few repositories to do this:

  • cloud-infra : containing our Terraform manifests to create the Kubernetes cluster and DNS records
  • k8s-cluster-state : containing our klifter project to deploy our Helm charts to the cluster
  • charts : containing our Helm repository

Everything is automated with Github Actions.

klifter and klander are homemade tools to provide Continuous Deployment and Reconciliation of our Kubernetes cluster. But, using them within a Github workflow was quite tricky.

This is why we developed custom Github Actions to simplify this:

- uses: datapio/actions/klifter@main
with:
path: path/to/klifter-project
kubeconfig: path/to/kubeconfig.yml
- uses: datapio/actions/klander@main
with:
kubeconfig: path/to/kubeconfig.yml

🧪 Testing our Github Actions

The next tricky part was testing those actions.

We started by creating a Kubernetes cluster with the engineerd/setup-kind Github Action, which creates the cluster the KinD (short for “Kubernetes in Docker”). It edits the current user’s kubeconfig automatically.

The problem:

The kubeconfig points to 127.0.0.1:<random port>, which in turns route trafic to the API Server within the Docker container deployed by KinD.

Because our custom Github Action is based on a Docker container as well, we can’t just use the kubeconfig as-is. Inside the Docker container, there is no API Server on 127.0.0.1.

The solution:

We need to edit the kubeconfig in order to point to the Github’s Docker host instead. The following script allows us to get the IP address of this host:

Then, in our workflow, before running the klifter or klander actions, we modify the kubeconfig :

Now, our actions can use the new kubeconfig to connect to the KinD cluster.

The next problem:

We are now getting a Connection Refused error when running kubectl commands.

This is because the KinD cluster’s API Server is configured to listen on 127.0.0.1, not the Docker host IP address.

The solution:

Fortunately, KinD can take a configuration file as input before creating the cluster, allowing us to configure the IP address of the API Server:

We save this configuration to .github/config/kind.yml.in and then generate the final configuration using our get-docker-host.sh script:

Conclusion

Using this method, we can now create a new empty Kubernetes cluster for our test suite, and run Docker images (either with docker run, or as a custom Github Action) interacting with this cluster.

To see a complete example, you can see the workflow we use here.

--

--

--

CEO & Co-Founder at Link Society

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

AWS EKS cluster integration to JENKINS

Understanding the different Number Systems

Git Stash

Graph Databases: The Story-tellers of the Database World

A Serverless Approach to Data Warehousing

How to use the Google Cloud Vision API and ClickSend to keep tabs on your pets

An open source boilerplate for you next SaaS product, written in django

Bi-Weekly Update #2

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
David Delassus

David Delassus

CEO & Co-Founder at Link Society

More from Medium

Orchestrating Microservices in Nodejs using Kubernetes

Setup Container Redis, run commands, and publish/subscribe messages

Setup Elasticsearch Cluster and Kibana in Local

Kubernetes Dapr on OKE-Distributed Application Runtime on OCI Kubernetes Engine