Boosting Your Development Feedback Loop with Preview Environments

The process of developing a new feature for a project usually involves the same steps:

• Create a new branch in the repository for the new feature (feature branch)
• Commit the code and create a new Pull Request
• Review the code changes
• Merge the Pull Request into the base branch
• Deploy the base branch to the Staging environment
• Test the feature (QA) in Staging
• Approve and deploy to Production or request changes opening the PR again.

If you follow git-flow (or an alternative flow based on feature branches) this will sound familiar to you. The main point here is that until you don't deploy your new feature to the Staging environment it can't be tested in a realistic environment (local environments don't count ? ).

Also, this workflow leads to the following questions:

• What happens when the team is developing multiple features at the same time?
• Is the code in the base branch always deployable (in terms of business logic)?
• What if two new features are merged and tested but the team only wants to release one of them and delay the other one?
• Which feature is responsible for a bug found in Staging if two or more feature branches were merged into the base branch?

As you imagine this way of working with branches and environments produces situations which would be better to avoid. The solution is to implement Preview Environments (sometimes also known as Preview Deployments).

A Preview Environment allows us to have a new complete and isolated environment for each new feature. The advantages of having Preview Environments are:

• Test features independently: preview features
• Improves feedback loop
• Avoid merging into the base branch until the feature is approved
• Avoid mixing new features in the Staging environment
• Preview changes as they would look in Production

The following diagram reflects different environments: Production, Staging, and one Preview Environment for every active Pull Request

Lifecycle

Preview Environments are disposable environments, which means that they are created, updated, and destroyed. They have the purpose of testing a feature, and once the task is done it doesn't make sense to keep them alive. This process can take from minutes to months, it depends on the size of the feature, the time it takes to test it, if it makes sense to merge the Pull Request at the moment due to business requirements, etc.

When a new Pull Request is created a new Preview Environment is created automatically with the name of the Pull Request.

If the Pull Request is updated with new commits then it is deployed again to show the latest changes.

When a Pull Request is merged or closed the environment is destroyed automatically.

To be able to test a Preview Environment, every environment must have a unique preview URL. This URL can be automatically generated from the branch name. e.g.: for a branch named feature/user-register in a project named Manhattan the preview URL could be https://feature-user-register.manhattan.com.

Environment Data

Another advantage of Preview Environments is that data can be seeded and updated without affecting other environments. It means that every environment must create a new exclusive database when the environment is created, and delete the database once the environment is destroyed.

The initial data of this database can be cloned from the Staging database or seeded with default values. This makes the data in this environment independent from other environments, so QAs can create, update and delete anything they need without worrying about affecting other QAs testing other features. Also, the data in the environment can be reset to the initial values as needed.

Service Dependency

When we think of a new feature in a project we tend to think about a new branch in a repository, this can be usual in projects with a single repository and a single service, but this is not the common case in modern projects where multiple services are involved.

A new feature could concern a backend service and a frontend service at the same time. For the new feature, the frontend could depend on a new endpoint developed in the backend. Therefore we need a way to coordinate both repositories if we want to test the new feature. We can't just create a new Preview Environment for the frontend and then use the API available in the Staging environment because the new endpoints in the backend won't exist.

A good practice is to use the same branch name in both repositories, e.g.: feature/new-feature. Then we could deploy both backend and frontend to the same Preview Environment. By doing this we could be able to test the new feature completely and the frontend will use the backend endpoints of the new feature.

Setting up Environments

Creating a Preview Environment can be done in many different ways. Creating the environment depends on the design of the project infrastructure and the way of deploying the application to Production. In this article, I am going to cover two options: deploying the app code directly to a server and using Docker containers.

Deploy code to servers

This process deploys the code to a server using methods like uploading the code using SFTP via SSH. As the environment doesn't exist it requires some steps before being able to deploy the code. Supposing we already have an existing server dedicated to Preview Environments, the step to create a new Environment would be:

• Install and set up everything needed to run the code to be deployed: Web server, Database, Language Runtime, Virtual hosts, etc.
• Create DNS records for the preview URL(s)
• Deploy the code using SFTP

This way of creating Preview Environments has the following problems:

• Slow: Installing and setting up services takes a lot of time. Also, if the code of the project has hundreds of files the deployment process will be slow too. We want to have these Preview Environments ready as soon as possible.
• Complex: There are a lot of steps to maintain. We need an orchestration tool (like Puppet, Chef, or Ansible) to execute all the steps required and maintain that code in the future.
• Error-prone: Sometimes setting up a virtual host can fail due to a misconfiguration of the webserver or a repository with an external dependency can be down. If the process fails the environment won't be created.

Containers

A much better alternative is to use Docker containers. Supposing we already have an existing server dedicated to Preview Environments with Docker installed, the steps to create a new Environment would be:

• Create DNS records for the preview URL(s)
• Build the application container in the CI/CD service
• Upload the container to a container image registry
• Connect to the server and run the Docker container

This process seems to be more agile than before, but there are still some questions to solve:

• How are the Docker services exposed as external services? We would need something like a container to act as a reverse proxy with forwarding rules.
• How do we request the issue and manage the renovation of SSL certificates using Let's Encrypt?
• How many servers do we need for Preview Environments in our organization?
• How do projects map to Preview Environment servers in the organization?

It would be great if we could have a cluster of servers and we could forget about the specific server where the code must be deployed. If we need more resources we can just add more servers to the cluster. The best alternative to achieve this is to use a container scheduler like Kubernetes. We just need:

• A Kubernetes cluster in a provider (like DigitalOcean Kubernetes, Civo, Google GKE, Amazon EKS, etc)
• Write Dockerfiles for every repository
• Write the Kubernetes manifests to deploy every service

Writing Dockerfiles and Kubernetes manifest is out of the scope of this article.

Instead of using a provider to launch a maintain your own Kubernetes cluster you can use existing services to achieve the same result. One of my favorite options is Okteto Kubernetes service and their feature to create Preview Environments from our code. This allows us to forget about creating and maintaining the Kubernetes cluster, dealing with SSL certificates, and creating DNS records so that we can focus on what provides value for the project.

Conclusion

• Using only environments like Staging and Production is not a good idea
• Having an independent environment for every new feature improves the feedback loop of the team
• Getting feedback fast means you decrease the time needed to deploy a new feature to Production, and also the costs associated
• Using containers and schedulers like Kubernetes simplifies the creation of Preview Environments
• Working with external services like Okteto makes the whole process easier to create and maintain