Reading ClickHouse logs from the table and the bucket
Most teams put logs in ClickHouse because storage is cheap. The bill grows with compute on the way in, and most of that comes from a handful of log statements. 10x gives each one a budget.
Most teams put logs in ClickHouse because storage is cheap. The bill grows with compute on the way in, and most of that comes from a handful of log statements. 10x gives each one a budget.
An Elasticsearch cluster is sized by what sits in its hot tier, and messages logged on every request keep adding to it. This post moves one of them to the frozen tier on a live Elastic Cloud Hosted deployment. A search on the pattern ID brought all 400 documents back
Coralogix prices a gigabyte by its priority class, and data matching no policy is billed at High, the class priced for fast search. A message nobody searches pays the searching rate on every repeat. 10x picks one such message and a policy moves it from High to Medium, where queries
A log platform groups repeated lines and shows a count beside each group. A saved search or an alert points at one of those groups by its stored text. Drain3, the open-source implementation of the Drain algorithm, rewrites that text as new lines arrive. This post shows how far
A log platform meters by the byte, and most bytes in a log line are text that the same statement prints every run. 10x stores that repeated text once and compacts the line before the platform meters it, and expand returns the original. Compact lines stay queryable in place rather
An agent cutting a log bill works in steps days apart: rank the expensive messages, cap one, check a week later. A rule written against the text of a log line stops matching once a deploy rewords the line. This post walks the four MCP calls behind that work, each
A CloudWatch Logs bill is mostly ingestion, charged per gigabyte, so a health check that repeats all day pays that rate all day. Over a week of demo traffic, ingestion was 94% of what one cart-service message cost. This post covers the two cheaper places that message can go:
Indexing, the charge that keeps log lines searchable, is the expensive part of a Datadog bill. The usual fix is an exclusion rule written against the line's text, which stops matching when a deploy reformats the line. 10x gives each line a pattern ID compiled from code and
On Azure Monitor the Analytics plan charges by the gigabyte ingested, so repeated messages pay the full rate every time they print. This post takes one of them and sends it to the Basic or Auxiliary table plan, or to an S3 or S3-compatible bucket the organization owns. Every
How logs kept in S3 are searched without running an index server.
Four compile stages shaped like a C build, with a checksum cache that skips unchanged files.
What breaks when each query re-clusters the logs, traced through a spike that was seven events.