Why Logs Are Not a Record

Most teams already have the raw material for a record of what their equipment did. It sits in a shared drive as thousands of exported log files, one per run, each named after a timestamp or a lot ID that made sense to whoever pulled it that day. Ask a simple question — "how many runs used this recipe last month, and which ones drifted outside spec on this column?" — and the answer requires opening files one at a time, because a folder of logs is not a queryable record. It is just files.

Not A Data Problem

The gap is not the data. The data is usually fine: complete, high-resolution, column after column of exactly what a run key column recorded. The gap is that nothing has turned "a run happened, and this file describes it" into something you can search across every run at once. A folder answers "what is in this one file." An index answers "which runs looked like this," "when did this column start drifting," and "which runs came from this run key column value in that date range" — without you opening a single file by hand.

Building The Index

Building an index means walking every log once, pulling out the run boundaries, the columns each run wrote, and enough metadata to filter runs later — recipe, run key column value, start and end time — and writing that into a structure that supports lookups instead of linear scans. It is unglamorous work. It also changes what questions are askable. Before an index exists, "show me every run this quarter where this column exceeded its usual range" is a multi-hour manual exercise. After, it is a filter.

Run Windows As Join Keys

The other thing indexing changes is what a run window means. A run window is just the start and end bounds of one contiguous execution — trivial to state, easy to get wrong when it is buried in a filename convention that changes every few months. Once run windows are extracted into an index rather than inferred from file naming, they become a stable join key: you can line a run window up against a step boundary, a saved run from the reusable library, or another run's window for comparison, without re-deriving it from scratch each time.

The Logs Remain The Truth

None of this requires throwing away the logs. The files remain the source of truth; the index is a derived, queryable layer on top of them. What changes is that the question shifts from "which file do I open" to "which runs match," and that shift is the entire point of treating exported logs as a record instead of an archive.