Cut the Elasticsearch log bill per pattern

An Elasticsearch cluster is sized by what sits in its hot tier, and messages logged on every request keep adding to it. This post follows one of them into the frozen tier on a live Elastic Cloud Hosted deployment. A search on the pattern ID brought all 400 documents back from the snapshot repository.

The size of an Elasticsearch cluster follows how much data sits in its hot tier, because hot nodes carry the local SSD and the memory that keep that data searchable. Adding data there means adding nodes or a bigger deployment, so a message printed in a loop, a health check or a retry, grows what the cluster has to be sized for every time it repeats, whether or not anyone searches it.

Volume in a large system concentrates in a handful of repeating messages: a health check, a retry, a request-complete line, each printed millions of times. 10x shrinks the hot tier by moving those out of it, holding back the ones an alert or dashboard depends on. Nothing is deleted. The hard part is choosing which messages are safe to move, and a choice like that needs a unit.

That unit is a stable pattern ID, the same on every node running the same engine, library and configuration, with one volume figure against each ID. The engine sits beside the log forwarder (Fluent Bit, the OpenTelemetry Collector, Fluentd, and more) as a sidecar, taking each line while it is still upstream of the cluster.

From there a chosen pattern can go to a cheaper tier inside the cluster or out to a bucket the account owns, and the line survives either route. The levers cover the sources a forwarder fronts. Logs an Elastic Agent integration ships straight into the cluster stay on whatever tier they land on.

One ID per log statement

An API gateway prints these two lines.

2026-07-12 14:03:11 INFO api-gateway request complete route=/v1/charge status=200 bytes=1420 duration_ms=183
2026-07-12 14:03:12 INFO api-gateway request complete route=/v1/refund status=200 bytes=644 duration_ms=97

Both lines trace back to one statement, differing only where the statement interpolates a value. INFO, request complete and the field names route, status and bytes recur in every line it prints, and the route, the status code, the byte count and the timestamp do not. What recurs is the structure, so the engine retains it, discards the rest, and the hash of what is left is the pattern ID.

Fixed tokens are identified against a vocabulary compiled ahead of time from application and framework code, the set of strings that code is able to print. Token frequency is never consulted, and the stability follows from that: an ID computed on one host this week matches the one computed on another host next week. That holds except where no code stands behind the line. Raw access logs fall outside it, since a changing value there can pass for a fixed token, splitting one statement across a few IDs.

Elasticsearch groups logs into shared shapes too, through the `categorize_text` aggregation that Kibana's log pattern analysis is built on, and through machine learning categorization jobs. Both run over documents already indexed, which is too late to route by, because the index a line is written to is the tier that holds it and the grouping therefore has to exist before the write. A vocabulary gives that, having been compiled from code rather than read off the traffic. An ID derived from clustering instead moves with the history that produced it, measured against Drain, a common clustering algorithm, in a separate post, whereas the pattern ID has no history to move with.

Over a week in a demo environment running the OpenTelemetry demo shop, one pattern carried its own numbers: a high-volume INFO line from the cart service, GetCartAsync_called_with_userId, ID YZplfQ4a1Xw, about 1.5 KB per event, printing 16.2M lines and 24.2 GB a day, which is 486M lines and 726 GB over thirty days.

Every one of those gigabytes has to be held somewhere, and the index a line is written to is what decides which tier holds it. Synthetic traffic produced those counts, so they describe the accounting and not a saving.

Two levers on a cluster

Tier down moves a pattern to the frozen tier, where its data sits in a snapshot repository instead of on hot-tier SSD. The lines stay searchable from the same cluster with the same query, slower than hot because the shards read from the repository, though nothing leaves Elasticsearch. Self-managed, that is disk the cluster stops needing. On Elastic Cloud Hosted, priced by the resources provisioned for it, the hot tier can shrink, but the deployment has to be resized for that to land on the bill.

Offload sends a pattern's events out of the cluster, so they are never indexed. The forwarder writes them to an S3 bucket the account owns, as plain JSON lines any client can read, with a pattern-ID index beside them. A retrieval by pattern ID reads only the matching objects and returns the events, with nothing re-indexed. An operator who has a raw line instead of an ID pastes it into the same tool, which finds its pattern first.

A third lever applies where the cluster is the account's own to change. 10x can write the events in compact form, keeping each log statement's message text once as a template and shipping each line as that template's hash plus the values that change, with a plugin in the cluster reading that form back at query time. What it leaves on disk is measured in its own post, although the plugin it depends on rules it out on a managed deployment. Tier down and offload need nothing inside the cluster, which is what makes them the two levers that reach Elastic Cloud Hosted.

The index and the ILM policy

On Datadog the down-tiered slice goes to a Flex index, on CloudWatch to an Infrequent Access log group, on Azure Monitor to a Basic or Auxiliary table, and on Coralogix to the same endpoint as everything else with a policy moving its class. Elasticsearch tiers by index, so the marked slice goes into an index of its own and an ILM policy on that index does the rest. There is no second endpoint to stand up and no plugin to install, which is what makes this the one lever that reaches a managed deployment.

The engine has to make the call, because it cannot be recovered from the event afterwards. Whether a pattern has passed its byte budget for the current window is a fact about a stream, counted across events in the sidecar, and a rule reading one document has nothing to compute it from. So the engine writes its decision onto the event and the forwarder reads it to pick the index.

{"@timestamp": "2026-07-12T14:03:11Z",
 "container": "cart",
 "message": "GetCartAsync called with userId=7ec21970-9368-11f1-b95d-b6b0e8e896b2",
 "tenx_hash": "YZplfQ4a1Xw",
 "routeState": "tier_down"}

routeState carries that decision and tenx_hash carries the pattern ID. Events marked tier_down go to one index and everything else to another, with a policy on the first index alone sending it to the frozen tier. Both fields are mapped as keyword, which is what keeps them searchable after the move.

That chain was run end to end on 2026-07-31 against a live Elastic Cloud Hosted deployment, version 9.4.4, with a second index kept hot as a control. The marked index reported 400 documents and zero bytes of local store on a frozen node, the control 200 documents and 19.2 kb on a hot node. Zero bytes is the saving: the documents are in the snapshot repository, while the control kept its own in hot, so only the marked slice moved.

Two things to know before reproducing it: searchable snapshots are licensed, needing Enterprise self-managed or Gold and above on Cloud Hosted, and a registered snapshot repository, though Elastic Cloud provides that by default. ILM polls every ten minutes, so a transition is not visible the moment an index becomes eligible for one.

Searching the frozen slice

Whether the moved slice still answered was checked before the lever shipped. Both hashes were recorded before the transition and searched for after it. A term query on tenx_hash against the frozen index returned 400 of 400 documents, routeState was still aggregatable at 400 in the tier_down bucket, with the document bodies coming back whole. The control answered 200 of 200 for its own hash from hot.

The identity the sidecar stamps is what makes the moved slice findable later: an operator holding a pattern ID gets its events back with an ordinary search, from the same cluster, with no step in front of them. Had that query come back empty, the lever would not have shipped.

The choice and the record

Elastic sells the frozen tier, and writing to a bucket is something any forwarder does. Deciding which pattern is safe to move is what remains, together with holding that decision steady while the logs change.

An operator's monthly budget is the precondition; nothing moves without one. The log10x MCP derives per-container limits from that budget and opens a pull request for a person to merge. Containers the budget does not name are left as they are. Inside one that it does name, a pattern has to hold above a fifth of recent volume over a few-minute window before its excess moves.

Data under that share stays hot. So do the opening events of each window, and severity floors keep at least half of a pattern's Error events out of the move as well.

The two levers differ in what they risk: a frozen index is still inside the cluster, so a search or a dashboard pointed at a down-tiered pattern still finds it, which makes tier down the first choice on Elasticsearch. Offload is the fallback, because events outside the cluster stop being visible to whatever was reading them, and any pattern behind an alert is exempted from it. Each pattern's volume before the move goes out as a metric series to the account's own metrics backend, which is where a dashboard keeps reading it from.

Committed files hold the budgets and the exemptions, keyed by pattern ID and carrying a reason and an expiry, which together with the per-pattern history make up the record behind a move. Reading it needs no account: the log10x MCP server installs into an AI client and runs against a live demo, the source of the numbers above. ILM runs the transition on its own ten-minute schedule, and what arrives upstream of it is a mark on the event that decides which index the line is written to.