Cut the Coralogix log bill per pattern

Coralogix prices a gigabyte by the class it lands in, and data matching no policy lands in High, the class priced for fast search. A message nobody searches pays the searching rate on every repeat. 10x picks one such message and a policy moves it from High to Medium, where queries and alerts keep working. The move was run end to end against a live Coralogix tenant.

Coralogix bills a gigabyte at the rate of the priority class it lands in, so a message printed in a loop, a health check or a retry, pays that rate every time it repeats. Data matching no TCO policy lands in High (Frequent Search), the class that carries fast queries and real-time alerting, which puts a message nobody searches in the class priced for searching.

10x lowers that bill by moving the messages that cost the most to a cheaper class, holding back the ones an alert or dashboard depends on. Nothing is deleted: every line stays retrievable, and the alerts and dashboards keep working.

10x runs as a sidecar to a log forwarder (Fluent Bit, the OpenTelemetry Collector, Fluentd, and more) and reads each line before Coralogix receives it. The engine gives each line a stable pattern ID, the same on every node running the same engine, library and configuration, and each pattern carries its own volume. Two levers then move a chosen pattern, to a cheaper priority class or to a bucket the account owns, without either of them losing the line.

The levers only touch what passes the forwarder, which leaves a cloud integration shipping straight into Coralogix on the standard bill.

Pattern IDs and templates

The pair below came off an API gateway.

2026-07-12 14:03:11 INFO api-gateway request complete route=/v1/charge status=200 bytes=1420 duration_ms=183
2026-07-12 14:03:12 INFO api-gateway request complete route=/v1/refund status=200 bytes=644 duration_ms=97

One logging statement printed both, and the values are all that separate them. In every line that statement prints, INFO, request complete and the field names route, status and bytes are the same, while the route, the status code, the byte count and the timestamp are not. Those fixed tokens are the structure, so the engine holds on to them, discards the values, then hashes the remainder into the pattern ID.

The engine identifies the fixed tokens from a vocabulary, the strings the code can print, compiled from application and framework code ahead of time. Nothing in that decision looks at how often a token repeats, which is what anchors the ID across hosts and across weeks. Lines with no code behind them, raw access logs, do not behave that way: a changing value can pass for a fixed token there, so one statement can end up with a few IDs.

Coralogix groups logs into shared shapes too. Templates, the feature formerly called Loggregation, cluster incoming logs by their constant structure, replace the values with placeholders, then expose a template ID to filter on. Those clusters come from the traffic, though: a template is created once a shape crosses an occurrence threshold, and an ID derived from clustering moves with the history that produced it, which a separate post measures against Drain, a common clustering algorithm.

The pattern ID has a different source, the vocabulary rather than the traffic, so it does not move with the history. The ID also exists before Coralogix has seen the line, which is what lets it drive the routing decision below, whereas a template is formed inside Coralogix from events that have already arrived.

The numbers attach to the pattern: a high-volume INFO line from the cart service, GetCartAsync_called_with_userId, ID YZplfQ4a1Xw, ran at 16.2M lines and 24.2 GB a day over a week in a demo environment running the OpenTelemetry demo shop, about 1.5 KB per event.

At the Coralogix list rate for High (Frequent Search), $1.15 per GB, that one message is $835 over 30 days. High is where anything no policy matches lands, so that is what the pattern pays until a policy moves it. Both the traffic and the rate are stand-ins, one synthetic and one from the list, which makes the figure an accounting exercise rather than a saving.

The two destinations

Tier down moves a pattern from High (Frequent Search) to Medium (Monitoring), where the raw data is held in storage the account owns and stays queryable with DataPrime, alerting and dashboarding. A query reaches those lines where they sit, with no rehydration step in front of them, which is what makes the move a down-tier. Nothing forks to get there, since the events ship to the same ingest endpoint as everything else and a policy assigns the class.

Under offload a pattern's events never reach Coralogix. What the forwarder writes to the account's own S3 bucket is plain JSON, one line per event, with a pattern-ID index sitting next to it, and any client can read the result. Retrieval works off that index, so only the objects carrying the requested pattern ID are opened, nothing is re-ingested, and a raw line serves as well as an ID because the same tool resolves the line to its pattern first.

The marker on the event

Elsewhere the down-tiered slice has a separate place to go: a Flex index on Datadog, an Infrequent Access log group on CloudWatch, a Basic or Auxiliary table on Azure Monitor. Reaching any of those means the forwarder splitting the stream. Coralogix offers no second destination for a down-tier, though, which narrows the forwarder's job here to leaving the engine's routing decision inside the event, where a policy can match it.

A rule inside Coralogix cannot work that decision out for itself. Passing a byte budget for the current window is a property of a stream, counted across events in the sidecar, and a policy sees one event at a time. The outcome is therefore written onto the event, and the forwarder ships it without stripping it.

The engine's default output hands the forwarder that marker as its own field, which the forwarder's filters branch on, and here is what arrives.

{"applicationName": "checkout",
 "subsystemName": "tier_down",
 "text": {"message": "request complete route=/v1/charge status=200",
          "routeState": "tier_down"}}

routeState is the engine's decision for this event, held inside text, the event body. Coralogix policies match on any field in the event body, nested fields included, so the marker alone is enough to key a policy on, with no label mapping. The forwarder also copies the decision to subsystemName, which an operator can see in the Coralogix UI and a policy can key on instead.

The chain was run end to end against a live Coralogix tenant, the engine feeding a Fluent Bit running the shipped recipe, with the same 160 events going through twice. With no policy in place all 160 arrived, 114 of them marked tier_down. With one policy keyed on the marker and set to Medium, 46 were left in Frequent Search. Those 46 are exactly the events the engine never marked: 6 from the capped container before its cap was crossed, plus all 40 from a second container that had no cap and so was never routed.

A second test rules out the label: two events went up in one request, same application and same subsystem, differing only in the marker. The one marked pass stayed visible and the one marked tier_down did not, which leaves the body field as the only thing the policy could have matched.

One note for anyone reproducing this: a policy is not in force the moment it is applied. A check a minute after the apply showed nothing moved, though the same query at six minutes showed the slice gone from Frequent Search.

Budgets and exemptions

Neither lever is new: Coralogix sells the cheaper classes, and any forwarder can write to a bucket. Which pattern is safe to move is the part left over, along with keeping that answer current while the logs change.

The budget comes first and it comes from a person: a monthly figure, turned into per-container limits by the log10x MCP, proposed in a pull request that someone merges. Containers the budget leaves out are left alone. Where a budget does apply, the trigger is dominance of recent volume, a fifth of it or more across a few-minute window, and only the excess above that moves.

Along with whatever falls under that share, High keeps the opening events of each window, and severity floors hold back no less than half of a pattern's Error events. A pattern moved in error is still fully available, in Medium to a query and in the bucket to a retrieval by ID.

Medium keeps alerting and dashboarding, so a rule that reads a down-tiered pattern keeps working, which is why tier down is the first lever on Coralogix and offload the fallback. Offload takes the events out of Coralogix entirely, however, so anything reading them stops seeing them, and a pattern behind an alert is exempted from that lever. Volume is reported separately as a metric series to the account's own metrics backend, taken before the move, so a dashboard reading it is unaffected by which lever ran.

A committed file records each budget and each exemption under its pattern ID, with a reason and an expiry attached, so a move can be traced through those files and through the pattern's own history. The record is visible through the log10x MCP server, which installs into an AI client and runs against a live demo with no account. The numbers above came from there. On Coralogix the enforcement is one policy matching one field, which puts the weight on the marker the engine wrote and on the budget behind it.