How to Build a Website Monitoring Pipeline in CI/CD

A website monitoring pipeline is a CI/CD job that runs a diagnostic audit against the deployed URL after the deploy step and fails the job when a check fails. The pipeline has 5 stages: the API key secret, the audit step, the gate level, the check list, and the verdict file. The UpMonitor CLI runs the http, ssl, dns, performance and securityHeaders probes in 1 command and exits 0, 1 or 2. The job exits 2 before any probe runs, if the runner has no API key.

What Is a Website Monitoring Pipeline?

A website monitoring pipeline is a CI/CD job that probes the live URL after every deploy and turns the probe result into a pass or a fail for the pipeline. It sits after the deploy job and ahead of the traffic switch, so a broken release is caught by the runner and not by a visitor. The audit catches the 4 deploy regressions that leave the server running and the site broken:

• Missing security headers after a new proxy or CDN config, so every response ships without HSTS and CSP.
• A broken certificate chain after a renewal, so browsers refuse the TLS handshake.
• A TTFB regression from a new script or query, so the first byte takes 2 times longer to arrive.
• A redirect loop from a router rule, so the URL never reaches a final status.

Each of the 4 maps to 1 of the default checkers: securityHeaders, ssl, performance and http.

How to Build the Pipeline in 5 Stages

The pipeline has 5 stages, built in this order: the secret, the audit step, the gate level, the check list, and the verdict file. The first 2 stages produce a working job; stages 3 to 5 tune what the job blocks on and what it keeps.

Stage 1: Store the API Key as a CI Secret

The CLI reads the API key from the UPMONITOR_API_KEY environment variable first and from ~/.config/upmonitor/credentials.json second. The credentials file is written by upmonitor login on your own machine, and a CI runner has no such file, so the job needs the variable. Create a repository secret named UPMONITOR_API_KEY in GitHub under Settings, Secrets and variables, Actions, and export it into the job environment as Stage 2 shows. GitHub masks the value in the job log. A job without the variable exits 2 before any probe runs, and prints this line to stderr:

ERROR: No API key found. Run `upmonitor login` or set the UPMONITOR_API_KEY environment variable.

Stage 2: Run the Audit After the Deploy Step

The audit job declares needs: deploy, so it starts after the deploy job finished and probes the URL that deploy produced. The CLI documentation for the check command lists the 5 options the check command takes. The job below uses 2 of them: --ci for plain log output and --format json for the verdict file. The env: value below spans 2 lines because YAML folds the line break into 1 space, so copy it as shown or join it onto 1 line in your file. The gate step reads the verdict file and exits 1 when the overall status is down or any result reports failure. The upload step keeps the file even when the gate fails:

audit:
  needs: deploy
  runs-on: ubuntu-latest
  env:
    UPMONITOR_API_KEY: ${{
      secrets.UPMONITOR_API_KEY }}
  steps:
    - name: Run the audit and keep the verdict
      run: npx @upmonitor/cli check https://upmonitor.io --ci --format json > audit.json
    - name: Fail the job on any failed check
      run: |
        node -e "
          const run = require('./audit.json');
          const failed = run.results.filter((r) => r.status === 'failure').map((r) => r.name);
          if (run.status === 'down' || failed.length > 0) {
            console.error('failed checks:', failed.join(', '));
            process.exit(1);
          }
        "
    - name: Upload the verdict
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: audit
        path: audit.json

Stage 3: Pick the Gate Level With --fail-on

--fail-on failure makes the CLI exit 1 when the run's overall status is down, and the API grades a run down when the http probe reports anything but success. That makes the flag the right 1-line gate for a routing-only job, where http is the only probe in the run:

npx @upmonitor/cli check https://upmonitor.io --checks http --ci --fail-on failure

In CLI 1.1.6 and earlier, the per-check statuses from ssl, dns, performance and securityHeaders do not move the exit code. The CLI reads each result's status from a field the API response does not carry, and --fail-on warning never trips for the same reason. The verdict file in Stage 5 is the gate that covers all 5 checks. The 3 exit codes stay the same in both paths: 0 when the gate passed, 1 when it tripped, 2 on a configuration or network error.

Stage 4: Narrow the Scope With --checks

--checks takes a comma-separated list of checker ids and replaces the 5 defaults, which are http, ssl, dns, performance and securityHeaders. The published 1.1.5 package names the first default httpStatus, and the API runs http and httpStatus through the same checker, so both ids work. The registry holds 5 more ids that a job adds by name: dnssec, seoMeta, compression, content and lighthouse. The lighthouse checker needs a PRO or Agency plan, so a Free key gets a 402 for it. An unknown id is skipped without an error, which is why --checks ssl,headers runs only ssl and exits 0. Check the spelling of every id against the list above before you trust a green run.

Stage 5: Keep the Verdict With --format json

--format json prints 1 object with 4 fields, url, status, latency and results, and each entry in results carries name, status, message and howToFix. The gate step in Stage 2 reads that file: status is up or down for the run, and each result's status is success, warning or failure. Change the filter to r.status !== 'success' and the same step blocks on warnings too. The howToFix field holds the first fix step for each failed result, so the artifact answers the question a red job raises without a second run. The pretty and csv formats exist for a terminal and a spreadsheet, and neither is machine-readable enough to gate on.

The Same 5 Stages in GitLab CI

GitLab CI carries the same 5 stages with 3 changes. The secret is a CI/CD variable marked masked and protected. The job takes stage: post-deploy with needs: [deploy]. image: node:20 supplies npx, which a GitHub ubuntu-latest runner ships by default. A masked variable is hidden in the job log, and a protected one is exposed only on protected branches, which is the right pair for a production key. The variable name stays UPMONITOR_API_KEY, so no env: block is needed. artifacts: when: always keeps the verdict file after a failed gate:

stages:
  - deploy
  - post-deploy

audit:
  stage: post-deploy
  image: node:20
  needs: [deploy]
  script:
    - npx @upmonitor/cli check https://upmonitor.io --ci --format json > audit.json
    - |
      node -e "
        const run = require('./audit.json');
        const failed = run.results.filter((r) => r.status === 'failure').map((r) => r.name);
        if (run.status === 'down' || failed.length > 0) {
          console.error('failed checks:', failed.join(', '));
          process.exit(1);
        }
      "
  artifacts:
    when: always
    paths:
      - audit.json

The 5 Checks the Pipeline Runs by Default

The default run probes 5 checkers, and each grades its result success, warning or failure by its own threshold:

The 7 headers the securityHeaders checker reads are Strict-Transport-Security, Content-Security-Policy, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, Permissions-Policy and Cross-Origin-Opener-Policy. An HSTS max-age under 6 months (15552000 seconds) adds a message to the result and leaves the status at success. The http grade is the one that sets the run's overall status, so a 502 after deploy is the fastest red a pipeline produces. Run the free HTTP status checker for the URL you deploy once by hand to see the same 4 http verdicts before you automate them. The TTFB performance test with the same 500 ms threshold shows where your baseline sits against the 501 ms warning edge.

Should the Gate Block the Deploy or Only Warn?

Block the pipeline on failure from the first run, and widen the gate to warning once a clean run reports 0 warnings. A failure on any of the 5 checkers means a visitor already sees the defect, so a red job is the correct answer. A warning names a threshold the site was already past before the deploy, such as a 30-day certificate window or a 700 ms TTFB. A gate that blocks on it from day 1 turns every release red until the backlog is cleared. Blocking is safe under 3 conditions:

• The URL is the production hostname, so the verdict describes what visitors get.
• The job runs post-deploy, so the probe sees the new release and not the old one.
• A rollback step exists, so a red gate has an action attached and not only a log line.

Warn-only is the right mode for a staging URL, where the point is to read the artifact and not to stop anything.

Common Pipeline Failures and How to Fix Them

The 5 failures below cover every exit code the audit job produces, and each one names the line the runner prints, the cause, the fix and the prevention:

1. ERROR: No API key found with exit code 2. The secret exists in the repository, and the job never exported it into its environment. Add the env: block at the job level as Stage 2 shows, and confirm the secret is named UPMONITOR_API_KEY and not a variant. Prevention: use the same secret name in every repository, so the job YAML is copied unchanged.
2. ERROR: Check request failed: <reason> with exit code 2. The API rejected the request or did not answer, and the reason names which. Unauthorized means a wrong or revoked key. payment_required means the 100 requests a month the Free plan allows are used up. rate_limit_exceeded means more than 20 requests in a minute. forbidden_target means a private or loopback URL. Request timed out means no answer within 30 seconds, and API unreachable means the runner has no route to the API. Fix the named cause, then re-run the job. Prevention: 1 audit job per deploy, so a busy release day stays inside the monthly quota.
3. failed checks: securityHeaders with exit code 1 after a proxy change. The new edge config dropped every response header the checker reads, so all 7 are absent and the result is failure. Restore the header block in the proxy or CDN config, redeploy, and confirm the 7 names with the security headers checker for the 7 audited headers. Prevention: keep the header config in the repository next to the app, so a proxy rewrite is a reviewed diff.
4. failed checks: http with exit code 1. The message field in the verdict file names the cause. It reads Server Error: HTTP 502 for a 5xx, Routing check failed: Redirect loop detected for a loop, or Routing check failed: Exceeded maximum redirect limit when 10 redirects are hit. Read the code against the guide to HTTP status codes explained for redirect and error responses, then fix the origin or the router rule that produced it. Prevention: the routing-only gate from Stage 3 on every deploy, because http is the check that goes red first.
5. failed checks: ssl with exit code 1. The certificate has expired, so daysRemaining is 0 or lower and the result is failure; the 30-day window before that reports warning, which the default gate ignores. Renew the certificate, redeploy, and re-run the job. Prevention: add a scheduled monitor for the hostname, so the 30-day warning lands in the monitor results before the expiry lands in a red job.

How to Keep Monitoring After the Pipeline Passes

The pipeline probes the URL once per deploy, and a scheduled monitor probes it on a fixed interval between deploys. A Free plan monitor runs every 300 seconds from 1 region. A PRO monitor runs every 60 seconds from up to 6 regions, and an Agency monitor every 60 seconds from up to 12 regions. A monitor runs its checkers on every interval, deploy or no deploy. A certificate that enters its 30-day window on a Tuesday shows the warning in that day's results, not on the next deploy. The CLI creates one from the same runner with upmonitor monitor add https://upmonitor.io --regions europe-west2. The guide to free uptime monitoring on a fixed interval covers the setup of a monitor on the Free plan.

Frequently Asked Questions

Does the CLI need an account to run in CI?

Yes. Every check run is 1 authenticated request to the API, and the CLI exits 2 with No API key found before any probe runs when the key is absent. Create the key once with upmonitor login or upmonitor register, store it as the UPMONITOR_API_KEY secret, and the job authenticates on every run.

Can the pipeline run before the deploy instead of after?

Only against a URL that is already serving. The audit probes a live hostname, so a pre-deploy run points at the staging URL of the release candidate, and a post-deploy run points at production. Running both is the common layout: staging blocks the merge, production blocks the traffic switch.

What is the monthly limit for check runs on the Free plan?

100 API requests a month and 20 a minute on Free, 10,000 a month on PRO, and no monthly cap on Agency. Each check run is 1 request whatever --checks lists, so a Free key covers 3 deploys a day with 10 requests to spare. The upmonitor usage command prints the current count.

Can I run the check from a specific region in CI?

Not with check. The CLI accepts --region <id>, and the API ignores it: the /v1/run-check route reads only url and checkers, so the probe runs from the API's own location. To probe from 1 of the 12 regions, create a monitor with --regions <id> and let the schedule run it.

Does the check run on every commit or only on deploy?

Only on deploy. The job declares needs: deploy, so a push that does not deploy produces no audit run. A run against an unchanged site spends 1 of the monthly requests on a verdict that already exists. Pull requests get their own audit against a preview URL, if the deploy job publishes one.