Elastic Synthetics Projects: A Git-friendly way to manage your synthetics monitors in Elastic Observability

Elastic has an entirely new Heartbeat/Synthetics workflow superior to the current workflow. If you’re a current user of the Elastic Uptime app, read on to learn about the improved workflow you can use today and should eventually migrate toward.

We’ve recently released a beta feature that provides a Git-friendly IaaC oriented workflow. You can now push Heartbeat monitors with the same ease with which you push code changes in Git or config changes in Terraform. The features discussed in this blog are all currently in beta, and we urge users trying these features out to upgrade to the latest stack version first. When these features become GA, this new workflow will be the preferred way of configuring monitors in the Elastic Stack. If you’re starting a new project, you may want to consider setting it up this way instead of via our more classic configuration.

Today, using Heartbeat is simple. You just need to write a little YAML and monitoring data shows up in Elasticsearch, visible in the Uptime UI. While the UI is indeed simple, there’s some hidden complexity there that we’ve improved with a new UI (the Synthetics app) and augmented with an even more automation friendly CLI workflow via our new Projects feature, which will be discussed below.

How do you manage your configs written in YAML? Many of our users will manage YAML in Git and use tooling such as Ansible, Helm, or similar to manage their infrastructure as code (IaaC). As with any other organization, Elastic also heavily utilizes IaaC in all parts of our operations. Hence it’s only natural we developed a capability to provide you with similar support for the current Heartbeat capability and the upcoming synthetics monitoring capabilities.

Projects: A new way to organize and distribute configs

Let’s dive right into what we’re calling “Synthetics Projects” and how they differ from traditional Heartbeat config files. To use this feature, you would start by creating a project in a Git repo containing your configs. At a high level, setting up a project requires performing the following tasks:

1. Run npx @elastic/synthetics init to create a project skeleton in a directory. See more details on the npmjs.com site.
2. Run git init and git push on the generated directory to version it as a Git repository.
3. Add your lightweight YAML files and browser javascript/typescript files to the journeys folder.
4. Test that it works by running npx @elastic/synthetics push command to sync your project to your Elastic Stack.
5. Configure a CI/CD pipeline to test pull requests to your Git repo and to execute npx @elastic/synthetics push on merges to the main branch. 

So, once configured, adding, removing, and editing monitors involves:

1. Editing a monitor’s config, either YAML for lightweight monitors, or Javascript/Typescript for browser based ones locally
2. Testing your local configs with npx @elastic/synthetics journeys
3. Creating a new PR to your main branch via a Git push
4. Waiting for your CI server to perform the same validation and waiting for someone else on your team to review your PR
5. Merging your result to the main branch
6. Waiting for your CI server to push the changes to your Elastic stack

We’ve depicted the flow of data in the diagram below: