Prometheus Query Workshop

This workshop is designed to help you become proficient with Prometheus Query Language (PromQL). You will learn how to write effective queries to extract meaningful insights from your metrics data, from basic selectors to advanced aggregation patterns.

Shared CLI tools image (OpenShift 102 Workshop)

If you prefer not to install the command-line tools on your laptop, run the workshop’s tools image on the cluster.

It has oc, kubectl, helm, istioctl, jq, yq, git, gpg, curl, wget, skopeo and a docker shim, and an interactive bash session lists them when it starts. You don’t need a local oc client to start it: apply the manifest below from the web console.

Image (change the tag to match the release you deployed):

ghcr.io/shpwrck/openshift-102-workshop-tools:v1.0.14

To start it from the web console:

  1. In the OpenShift web console, select your Project.

  2. Click + in the top bar, then Import YAML.

  3. Paste the manifest below, change its image: line if you mirror images, then click Create.

  4. Go to Workloads → Deployments → openshift-102-workshop-tools, open the Pods tab, select the running pod, then Terminal. Start bash if the terminal opens sh.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: openshift-102-workshop-tools
  labels:
    app.kubernetes.io/name: openshift-102-workshop-tools
    app.kubernetes.io/part-of: openshift-102-workshop
spec:
  replicas: 1
  revisionHistoryLimit: 2
  selector:
    matchLabels:
      app.kubernetes.io/name: openshift-102-workshop-tools
  template:
    metadata:
      labels:
        app.kubernetes.io/name: openshift-102-workshop-tools
    spec:
      containers:
        - name: tools
          image: ghcr.io/shpwrck/openshift-102-workshop-tools:v1.0.14
          imagePullPolicy: IfNotPresent
          resources:
            requests:
              cpu: 50m
              memory: 128Mi
            limits:
              cpu: "1"
              memory: 512Mi

The release’s offline bundle has the same manifest as tools-deployment.yaml, with TOOLS.md, which describes the image, and images-mirror.txt, which lists the images to mirror for a disconnected cluster. The pod keeps running until you delete the Deployment.

Commands in the pod act as the pod’s default ServiceAccount, which has no permissions in your project. Log in as yourself with oc login inside the pod first (in the web console, click your user name, then Copy login command), or ask a project admin to grant the ServiceAccount edit. If the login reports certificate signed by unknown authority, add --certificate-authority=/var/run/secrets/kubernetes.io/serviceaccount/ca.crt to it.

The workshop’s Helm chart, openshift-102-workshop, is in ~/chart in the pod. To install the site from the pod, into your existing project and without a second tools pod:

helm upgrade --install workshop ~/chart \
  --namespace <your-project> \
  --set tools.enabled=false

Leave out --create-namespace, which fails for users who can’t create projects.

The deployed site also serves the chart as a Helm chart repository at /helm/, so you can add it from a workstation without cloning the repository:

helm repo add openshift-102-workshop \
  https://<your-site-route>/helm/

There’s no container engine in the image. docker maps a few commands to skopeo: manifest inspect, pull (to a local directory), image inspect and history.

Workshop Structure

This workshop consists of six modules:

  • Module 1: Prometheus Query Language Fundamentals - Learn the basics of PromQL, including metric names, labels, data types, and basic query syntax

  • Module 2: Selectors and Filtering - Master label selectors, regular expressions, and filtering techniques (includes exercises)

  • Module 3: Aggregation Functions - Understand how to use aggregation functions like sum, average, quantiles, and top-k (includes exercises)

  • Module 4: Advanced Query Patterns - Explore rate functions, subqueries, logical operators, and optimization techniques (includes exercises)

  • Module 5: Alerting Rules - Collect the metrics of your own application with user workload monitoring, write recording rules and alerting rules, and view and silence alerts (includes exercises)

  • Module 6: Grafana Dashboards - Run Grafana in your project with the Grafana Operator, connect it to the metrics of your project, and build a dashboard from your queries (includes exercises)

Prerequisites

This workshop assumes the following:

  • Basic understanding of Prometheus concepts (metrics, labels, time series)

  • Access to an OpenShift 4.16 or later cluster with Prometheus installed

  • Ability to access the Prometheus web UI or use the Prometheus API

  • Familiarity with basic command-line operations

A cluster administrator prepares the cluster before the workshop. All cluster components, roles, tools and images are listed in Setup Requirements; in short:

  • Modules 1 to 4: each workshop user has the read-only cluster-monitoring-view cluster role

  • Module 5: user workload monitoring is enabled, and each workshop user has the monitoring-edit role in their project

  • Module 6: the community Grafana Operator (version 5) is installed for all namespaces

If you are new to Prometheus, we recommend reviewing the Prometheus documentation before proceeding.

Expectations

After completing this workshop, you will be able to:

  • Write basic PromQL queries using selectors and filters

  • Apply aggregation functions to summarize metrics across multiple time series

  • Construct advanced queries using rate functions, subqueries, and logical operators

  • Debug and optimize Prometheus queries for better performance

  • Use PromQL effectively in Grafana dashboards and alerting rules