Skip to content

Portal | Level: L2: Operations | Topics: Loki | Domain: Observability

LogQL Drills

20 drills for Loki query language muscle memory. Each should take 1-5 minutes.

Difficulty: [E] Easy = stream selector + filter | [I] Intermediate = parsers or metric queries | [H] Hard = complex aggregations

Remember: LogQL has two query types: log queries (return log lines) and metric queries (return numeric values). Log queries use stream selectors + filters. Metric queries wrap log queries in functions like count_over_time(), rate(), bytes_rate(). Mnemonic: "Streams for reading, Metrics for graphing."

Gotcha: LogQL filter operators: |= (contains), != (not contains), |~ (regex match), !~ (regex not match). The |= operator is case-sensitive by default. For case-insensitive matching, use the regex form: |~ "(?i)error".

Debug clue: If a Loki query returns no results, check: (1) Is the stream selector correct? {namespace="x"} must match an actual label. (2) Is the time range long enough? Loki defaults to the last hour. (3) Is Promtail/the agent actually shipping logs from that source?

Drill 1: All logs from a namespace [E]

Question: Show all logs from the grokdevops namespace. 

# Your query here

Answer `{namespace="grokdevops"}`

Drill 2: Filter for errors [E]

Question: Show only log lines containing "error". 

# Your query here

Answer `{namespace="grokdevops"} |= "error"` > **One-liner:** `|=` is a line filter — it scans each log line for the substring. It is much faster than regex (`|~`) because Loki can use bloom filters for exact substring matches.

Drill 3: Exclude health checks [E]

Question: Show logs but exclude lines mentioning "/health". 

# Your query here

Answer `{namespace="grokdevops"} != "/health"`

Drill 4: Regex match [E]

Question: Find log lines containing any 5xx status code. 

# Your query here

Answer `{namespace="grokdevops"} |~ "status[=: ]+5[0-9]{2}"`

Drill 5: Multiple filters [I]

Question: Find error logs that are NOT related to health checks. 

# Your query here

Answer `{namespace="grokdevops"} |= "error" != "/health" != "healthcheck"`

Drill 6: Parse JSON logs [I]

Question: Parse JSON-formatted logs and filter for status >= 500. 

# Your query here

Answer `{namespace="grokdevops"} | json | status >= 500` > **Under the hood:** The `| json` stage parses each log line as JSON and extracts all fields as labels. After this, you can filter on any JSON key. If your logs are not JSON, use `| logfmt` for key=value format or `| pattern` for custom formats. > > **Default trap:** `| json` fails silently on non-JSON lines — they are dropped from results. If you see fewer results than expected, some log lines may not be valid JSON.

Drill 7: Parse and filter by level [I]

Question: Show only ERROR level logs from JSON output. 

# Your query here

Answer `{namespace="grokdevops"} | json | level = "error"`

Drill 8: Count errors per minute [I]

Question: How many error logs per minute in the grokdevops namespace? 

# Your query here

Answer `sum(count_over_time({namespace="grokdevops"} |= "error" [1m]))` > **Remember:** `count_over_time()` counts log lines, `rate()` gives per-second rate, `bytes_over_time()` gives total bytes, `bytes_rate()` gives bytes/s. Mnemonic: "**Count for totals, Rate for speed**" — same pattern as PromQL's `increase()` vs `rate()`.

Drill 9: Error rate from logs [I]

Question: What is the rate of error log lines per second? 

# Your query here

Answer `sum(rate({namespace="grokdevops"} |= "error" [5m]))`

Drill 10: Log volume in bytes [I]

Question: How many bytes of logs per second is grokdevops producing? 

# Your query here

Answer `sum(bytes_rate({namespace="grokdevops"}[5m]))`

Drill 11: Log volume by container [I]

Question: Break down log volume by container. 

# Your query here

Answer `sum by (container) (bytes_rate({namespace="grokdevops"}[5m]))`

Drill 12: Top 5 noisiest pods [I]

Question: Which pods produce the most log lines per second? 

# Your query here

Answer `topk(5, sum by (pod) (rate({namespace="grokdevops"}[5m])))`

Drill 13: Extract request path from JSON [I]

Question: Show only the request path and status from JSON logs with status >= 400. 

# Your query here

Answer `{namespace="grokdevops"} | json | status >= 400 | line_format "{{.path}} {{.status}}"`

Drill 14: Unique error messages [H]

Question: Count unique error messages in the last hour. 

# Your query here

Answer `count(sum by (msg) (count_over_time({namespace="grokdevops"} | json | level="error" [1h])))`

Drill 15: Duration percentile from logs [H]

Question: Calculate p99 request duration from JSON logs that include a "duration" field. 

# Your query here

Answer `quantile_over_time(0.99, {namespace="grokdevops"} | json | unwrap duration [5m]) by (path)` > **Gotcha:** `unwrap` converts a label value to a numeric sample for aggregation. The field must be a number. If the field is a string like `"0.5s"`, you need to strip the unit first with `| label_format duration=...` or ensure your app logs numeric-only values. Non-numeric values are silently dropped.

Drill 16: Pattern parser [I]

Question: Parse Apache-style access logs using pattern parser and filter for 5xx. 

# Your query here

Answer `{namespace="grokdevops"} | pattern " - - [<_>] \" <_>\" <_>" | status >= 500`

Drill 17: Count by label value [I]

Question: Count log lines per HTTP method in the last 5 minutes. 

# Your query here

Answer `sum by (method) (count_over_time({namespace="grokdevops"} | json [5m]))`

Drill 18: Detect panics [E]

Question: Find any log lines containing "panic" or "PANIC". 

# Your query here

Answer `{namespace="grokdevops"} |~ "(?i)panic"`

Drill 19: Compare log volume over time [H]

Question: Compare current log volume to 1 hour ago. 

# Your query here

Answer `sum(rate({namespace="grokdevops"}[5m])) / sum(rate({namespace="grokdevops"}[5m] offset 1h))`

Drill 20: Logs from a specific pod [E]

Question: Show logs from pod "grokdevops-abc123" in the grokdevops namespace. 

# Your query here

Answer `{namespace="grokdevops", pod="grokdevops-abc123"}`

Wiki Navigation

Prerequisites

Related Content