Cut the Azure log bill per pattern

On Azure Monitor the Analytics plan charges by the gigabyte ingested, so repeated messages pay the full rate every time they print. This post takes one of them and sends it to the Basic or Auxiliary table plan, or to an S3 or S3-compatible bucket the organization owns. Every line stays readable after the move.

On an Azure Monitor bill the expensive line is ingestion into the Analytics plan, the table plan that keeps logs fully queryable. Because that rate already covers the first 31 days of retention, a high-volume log is billed almost entirely on the way in. Per-GB billing then applies to every repeat, so a health check or a retry printed in a loop pays the full rate each time round, whether or not anyone reads it.

A large system's volume is dominated by a few messages that repeat: a health check, a retry, a request-complete line, printed millions of times each. 10x moves those to cheaper storage and holds back the ones an alert or a dashboard depends on. Nothing is deleted, so the target is a smaller Azure bill with the investigation experience unchanged. The difficulty is in the first step, choosing which messages are safe to move at all.

Reading happens ahead of Azure Monitor. 10x runs as a sidecar to the log forwarder (Fluent Bit, the OpenTelemetry Collector, Fluentd, and more), so every line passes the engine before the workspace sees it. Each line picks up a stable pattern ID there, the same on every node running the same engine, library and configuration, and each pattern carries its own volume and cost. That ID is what the two levers act on, sending a pattern either to a cheaper table plan or to an S3 or S3-compatible bucket the organization owns.

Both levers reach only as far as the forwarder does, though, and the standard bill still covers whatever Azure diagnostic settings, Application Insights, or the Azure Monitor Agent send straight to the workspace.

Where the pattern ID comes from

Two lines from an API gateway show what the engine reads.

2026-07-12 14:03:11 INFO api-gateway request complete route=/v1/charge status=200 bytes=1420 duration_ms=183
2026-07-12 14:03:12 INFO api-gateway request complete route=/v1/refund status=200 bytes=644 duration_ms=97

Both came from a single logging statement; the difference between them sits entirely in the values. INFO, request complete, and the field names route, status and bytes appear in every line that statement prints. The route, the status code, the byte count and the timestamp change on each pass, so the engine drops that group, keeps the fixed one as the structure, then hashes what remains into the pattern ID.

Which tokens count as fixed comes from a vocabulary of the strings the code can print, compiled ahead of time from application and framework code. Frequency plays no part, which is why the ID holds: the same statement yields the same ID next week and on every host. The exception is a line with no code behind it, a raw access log, where a changing value can pass for a fixed token and a single statement yields a few IDs.

Azure's KQL `reduce` operator clusters similar lines into shared shapes too, returning a pattern, a count and a representative line. The clustering happens per query, over the table and time range that query names, with nothing durable behind the clusters and no cost history. A cluster can also change when the lines it was built from change, which a separate post measures against Drain, a common clustering algorithm.

The pattern ID does not change that way, because it comes from the vocabulary rather than the traffic. A deploy that adds a field or writes a value into the text will stop a log alert matching on that text, though the pattern ID and its cost history carry on. Only rewriting the message itself gives a new ID.

Every pattern carries its own numbers. One of them was measured over a week in a demo environment running the OpenTelemetry demo shop and priced at Azure list rates for East US, where Analytics ingestion is $2.30 per GB with the first 31 days of retention included: a high-volume INFO line from the cart service, GetCartAsync_called_with_userId, ID YZplfQ4a1Xw, about 1.5 KB per event.

Across that window ingestion is the whole bill for this pattern, since the Analytics plan carries 31 days of retention in the rate and does not charge Analytics queries by the byte. Those are list rates over synthetic traffic, so the table shows the accounting rather than a saving.

Tier down and offload

Tier down sends a pattern to the cheaper Basic or Auxiliary table plan, where ingestion drops from the Analytics rate of $2.30 per GB to $0.50 or $0.05. The engine marks the events to move and the forwarder writes them to a table on the cheaper plan. A workspace sets the plan per table, so the rest of the workspace stays on Analytics.

Those lines remain queryable in the same Logs interface with KQL at workspace scope, billed by the gigabyte scanned, and an asynchronous search job reaches data past the interactive window. Queries run fast on Basic, though slower on Auxiliary, which makes tier down a fit for the patterns an incident query might still want to reach.

Offload keeps a pattern's events out of the workspace altogether, so no ingestion is charged on them. The forwarder is what writes them out, into an S3 or S3-compatible bucket the organization owns: plain JSON lines, readable by any client, with a pattern-ID index alongside. Retrieval names a pattern ID, reads only the objects that match it and hands back the events, re-ingesting nothing. Where an operator has a raw line and no ID, the same tool resolves the line to its pattern first.

How a pattern is chosen

Azure sells the cheaper table plans and any forwarder can write to an S3 bucket, so neither lever is the hard part. What stays open is which pattern is safe to move, and whether that choice still holds as the logs change.

Nothing moves by default: an operator sets a monthly budget, and the per-container limits from it are proposed by the log10x MCP in a pull request that a person merges. A container with no budget is left alone. Inside a budgeted container, only a pattern that dominates recent volume, above a fifth of it over a few-minute window, has its excess moved.

Two things are held back whatever the share: the first events of each window, and at least half of a pattern's Error events, which severity floors guarantee. Anything below the share stays on the Analytics plan too. Getting the choice wrong does not lose data, since the moved lines answer from the cheaper table, or come back from the bucket under their ID.

Because Basic carries only a simple rule and Auxiliary none, a pattern read by a log alert is exempt and stays on the Analytics plan. Pre-move volume also leaves the engine as a metric series, Azure Monitor among the supported backends, which keeps the number on a dashboard after the events have moved.

Each budget and each exemption is committed to the repository under its pattern ID, with a reason and an expiry recorded next to it; the per-pattern history completes the account of why a move happened. The log10x MCP server installs into an AI client and runs against a live demo with no account, which is where the table above came from. Azure enforces whatever plan a table carries, so the whole intervention reduces to writing a pattern's events to a different table, chosen on its cost history.