Integrating Tracetest with GitHub Actions in a CI pipeline

June 24, 2022
Est. reading time:
6 min.
See the challenges faced & approach taken in designing a CI pipeline using Github actions to run integration tests against an actual Tracetest deployment using Cypress and Tracetest.
Sebastian Choren
Engineer
Kubeshop
Table of Contents
Integrating Tracetest with GitHub Actions in a CI pipeline

# Trace-testing and Continuous Integration: setting up Tracetest with a CI Pipeline

The software development team here at [Tracetest](http://tracetest.io) cares a lot about software quality, which involves continuous delivery and continuous integration (CI/CD or CI CD) and Tracetest itself is no exception.

We were working with a pretty basic CI pipeline, just running some unit tests before merging PRs. But that wasn’t nearly enough. We want to be able to deliver the best possible quality software, and that requires more.

We need a way to deploy and run integration tests over new code and understand code changes. Now, we have the perfect tool to run those tests: Tracetest.

*Our goal is to test Tracetest using Tracetest!*

# Designing the CI Pipeline & Getting Ready to Build

Probably the hardest part of any design process is defining the goals. Setting a CI pipeline is no exception.

We wanted to set up a GitHub Actions pipeline so that we can run Tracetest tests in that CI environment. So, we created a tentative step-by-step recipe for that:

1. When creating a Pull Request, build the code and deploy it using our [Helm chart](https://github.com/kubeshop/helm-charts/tree/main/charts/tracetest).

2. Run Tracetest tests, and other end-to-end tests.

3. Cleanup created resources.

While it sounds simple to do that, in practice it has its nuances.

1. Which Tracetest instance will run the tests ("main instance")?

2. How are we going to communicate from the GHAction to the "main instance"?

3. How will the "main instance" communicate with the newly deployed version (called "target instance")? Same question goes for running cypress and other end-to-end tests.

4. Deploying and running integration or end-to-end tests is time-consuming. How can we minimize that?

We already have developed most of the tools needed to solve the problems listed above:

1. We start with deployment of a real, working Tracetest instance; our CD pipeline automatically deploys to our [demo environment](https://demo.tracetest.io) using our [Helm chart](https://github.com/kubeshop/helm-charts/tree/main/charts/tracetest), so it would be easy to build Docker images and deploy them for each Pull Request on demand. We can also keep a separate "main instance" running the latest stable version.

2. We have a CLI that we can use to talk with the "main instance." We just need to use kubectl to port-forward to the "main instance," and point the CLI to the correct address.

3. Making two services that are not exposed to the internet talk to each other can be tricky. Luckily, we use Kubernetes to run the "main instance" as well as the "target instance," so we can make this work with little effort by deploying both instances to the same cluster, and relying on the Kubernetes internal DNS. If needed, we can pass environment variables to the Tracetest CLI so we can configure tests as needed.

4. We can organize the pipeline so the fastest processes are run at the beginning and the slowest at the end, so we can fail fast instead of waiting several minutes just to see a unit test failing.

# The Implementation: Continuous Testing for Developers

This is what our final pipeline looks like:

You can see it in action here: [https://github.com/kubeshop/tracetest/actions/workflows/pull-request.yaml](https://github.com/kubeshop/tracetest/actions/workflows/pull-request.yaml)

# The 3 Stages of CI Pipeline

## 1. Run the fastest failing tests in parallel: unit tests for CLI, server, and web.

This also ensures that we are ready to build the Docker image and CLI tool used later.

## 2. Build and deploy a docker image.

Tag it specifically for this Pull Request and deploy it to its own Kubernetes namespace.

We use our Helm chart for this since it is easy to configure the instance. Also, we can wait until the instance is ready to run.

## 3. Run long tests to track code changes.

###  Cypress end-to-end tests

For this, we set a port-forward using kubectl, so Cypress can work as if it was a locally started instance listening at <http://localhost:8080/>.

### Tracetest tests

We use Tracetest’s CLI, so we will build it at this stage.

We set a port-forward to the "main instance" so the CLI can talk to it. Since the "target instance" runs in a dynamically created namespace, we need to use environment variables to make the CLI inject the correct URL when running test definitions. The good thing about this is that we are implicitly testing the CLI too!

After those steps are completed, the output of the tests is persisted with the GitHub Action, and we can safely remove the created Helm release and namespace.

We are currently working on actually testing Tracetest with trace test. The pipeline right now is only "ready" to support this, but we will move our tests in the following days.

You can see the actual pipeline definition here:

[tracetest/pull-request.yaml at main · kubeshop/tracetest · GitHub](https://github.com/kubeshop/tracetest/blob/main/.github/workflows/pull-request.yaml)

# Trace based testing with Tracetest: The New CI/CD Frontier

While [Tracetest](http://tracetest.io) is still in its early stages, it has all the tools needed to easily integrate into a CI/CD pipeline.

Creating end-to-end and integration tests is usually not an easy task, but we found that doing it with Tracetest and Kubernetes was pretty straightforward.

Tags:
Trace-based

Recent Articles

Subscribe to stay up to date by getting the blog direct to your inbox
Or join the Kubeshop Community in one of these channels