A 2GB .etl is not a file you open. It is a file you stream. PerfView will load it, but the UI is a liability when the incident clock is running: symbol resolution, tree building, and the WPF heap all compete with the pod that is about to recycle. The TraceEvent library is the same engine PerfView uses, and it is scriptable. This is the procedure for turning a multi-gigabyte ETW or EventPipe trace into a top-allocators report keyed by type and time interval, from a console process, with no UI in the loop.
What the trace actually contains
Allocation data in a .NET trace comes from the CLR runtime provider Microsoft-Windows-DotNETRuntime. The relevant keyword is gcsampledobjectallocationhigh (hex 0x200000) for the high-detail sampled allocation events, and allocationsampling (hex 0x80000000000) for the newer allocation sampling keyword. The gc-verbose profile in dotnet-trace is documented as tracking GC collections and sampling object allocations; the gc-collect profile tracks collections only at very low overhead. If your trace was collected with gc-collect, there are no allocation events to aggregate, and no script will conjure them. Confirm the profile before you write a line of code.
Two event shapes matter:
- GCAllocationTick — emitted by the runtime on the allocation sampling path. It carries an allocation amount and an allocation kind (Small, Large, or Pinned), plus a type name when the type-name keyword is enabled.
- Type-name resolution — the type name on the tick is a TypeID. Resolving it to a string requires the type-name events that the runtime emits when
gcheapandtypenames(hex0x1000000) is enabled. Without that keyword, you get a TypeID and a number, not a class name.
The dotnet-trace documentation lists both keywords explicitly in the CLR provider keyword table, so you can verify what was requested at collection time by reading the collection command or the profile name recorded in the trace metadata.
Choose the right entry point for the format
TraceEvent exposes different front doors for the two container formats. Consult the TraceEvent documentation for the current reader API for your version of the library.
- .etl — Windows ETW. Use the TraceEvent reader for .etl files and drive the callback loop as shown in the library’s samples.
- .nettrace — EventPipe. Use the EventPipe reader over a
Stream, or the EventPipe dispatcher for live sessions.dotnet-traceis built on EventPipe, and its default output format is NetTrace.
If you have a .nettrace and you want the .etl path, dotnet-trace convert exists, but it is a conversion, not a free lunch: it rewrites the file and can change event ordering or drop events the converter does not understand. Prefer the native reader for the format you actually collected. If the trace came from dotnet-trace collect-linux, note the documentation’s warning that the .nettrace format there is updated and that convert and report may not work with it yet. Read it with the EventPipe reader and do not round-trip it.
The streaming pattern
Do not materialize events. The TraceEvent callback model is a push loop: you register handlers, then call Process(), and the library reads the file in blocks and invokes your callbacks. Your job is to keep the callback cheap and the state small.
using Microsoft.Diagnostics.Tracing;
using Microsoft.Diagnostics.Tracing.EventPipe;
using Microsoft.Diagnostics.Tracing.Parsers;
using Microsoft.Diagnostics.Tracing.Parsers.Clr;
// .etl path
using var log = TraceLog.CreateFromEventTraceLogFile(etlPath);
// .nettrace path
// using var src = new EventPipeEventSource(nettracePath);
var clr = new ClrTraceEventParser(log);
// Bucket state: (intervalIndex, typeName) -> bytes
var buckets = new Dictionary<(int, string), long>();
var typeNames = new Dictionary<long, string>();
clr.TypeBulkType += data =>
{
for (int i = 0; i < data.Count; i++)
{
var t = data.ClrInstanceID; // placeholder; see note below
}
};
clr.GCAllocationTick += data =>
{
var typeName = data.TypeName ?? $"TypeID:{data.TypeID}";
var interval = (int)((data.TimeStampRelativeMSec - startMs) / intervalMs);
var key = (interval, typeName);
buckets.TryGetValue(key, out var cur);
buckets[key] = cur + data.AllocationAmount64;
};
log.Process();Three things in that sketch are load-bearing and worth stating plainly:
- Use
TimeStampRelativeMSec, not wall clock. ETW timestamps are QPC-based and the library normalizes them to milliseconds relative to the trace start. Wall-clock conversion is a separate step and is not needed for interval bucketing. - Resolve TypeID lazily. The type-name events arrive interleaved with allocation ticks. If you resolve eagerly you will miss names that arrive later; if you resolve at the end you need to keep the TypeID-to-name map alive. The map is small compared to the event stream, so keep it and resolve at report time.
- Do not call
data.TypeNameif the keyword was not enabled. It will be null or empty, and you will silently bucket everything under one key. Check once at startup whether the trace contains type-name events; if not, fail fast with a message that names the missing keyword.
Memory: the 2GB problem is not the file, it is your dictionary
A 2GB .etl with allocation sampling at the default rate can contain tens of millions of ticks. If you key your dictionary on (interval, typeName) and the type name is a fresh string per event, you will allocate a string per tick and the process will die before the report prints. Two mitigations:
- Intern type names. Keep a
Dictionary<long, string>from TypeID to a single canonical string, and key buckets on the TypeID, not the string. Resolve to strings only when you emit the report. - Bound the interval count. A 10-minute trace at 1-second intervals is 600 buckets per type. That is fine. A 10-minute trace at 1-millisecond intervals is 600,000 buckets per type, and that is not fine. Pick the interval from the question you are answering, not from the resolution of the trace.
The dotnet-trace documentation notes that the in-memory buffer defaults to 256 MB and that if the target emits events faster than they can be written to disk, the buffer may overflow and events will be dropped. That warning applies to collection, but the same arithmetic applies to your reader: if your callback is slow, the library’s internal queues grow. Keep the callback to a dictionary increment and nothing else.
Intervals and session boundaries
An .etl can contain multiple trace sessions if it was merged. The TraceLog exposes session boundaries; if you bucket purely on relative milliseconds you will splice two sessions into one timeline and produce a report that looks like an allocation spike at the seam. Check the session count before you process, and if it is greater than one, either split the file or offset the second session’s timestamps. The same applies to .nettrace files that were concatenated.
For a single-session trace, the interval math is trivial: interval = floor((TimeStampRelativeMSec - t0) / intervalMs). Emit the report as interval start time in relative milliseconds, plus the type name and the summed allocation bytes. If you need wall-clock labels, convert once at the end using the trace’s start time, not per event.
What the report should say
A top-allocators report that is useful during an incident has three columns and one sort:
- Interval — relative start, in milliseconds or seconds.
- Type — the resolved class name, or
TypeID:<n>if the keyword was missing. - Allocated bytes — the sum of
AllocationAmount64for that interval and type.
Sort by allocated bytes descending within each interval. Do not sort globally; the point of the interval dimension is to see the type that dominates now, not the type that dominated the whole trace. A type that allocates 4GB over ten minutes but nothing in the last thirty seconds is not your incident. A type that allocates 200MB in the last thirty seconds is.
If the trace has the type-name keyword, also emit a second view: top types by total allocated bytes across the whole trace, with the interval of their peak. That is the view that tells you whether the spike is a new type or an existing type that changed rate.
When the script is the wrong tool
Two cases where you should stop scripting and use something else:
- The trace has no allocation events. If it was collected with
gc-collector without the allocation keywords, the script has nothing to aggregate. Recollect with--profile gc-verboseor with explicit--providers "Microsoft-Windows-DotNETRuntime:0x200000:4"and accept the overhead. Thedotnet-tracedocs list the keyword hex values so you can construct the provider string without guessing. - The trace is truncated. The
dotnet-tracedocumentation states that an unhandled exception during collection results in an incomplete trace, that the trace is truncated when the runtime shuts down, and that Rundown information will be missing, so stacks may be unresolved. Allocation ticks near the end of such a trace may be present but type names may not be. The script should detect the missing type-name events and report the trace as incomplete rather than emitting a report that silently attributes bytes toTypeID:0.
If you need stacks rather than types, TraceEvent can give you them, but the cost is different: stack events are larger, and the aggregation key becomes a frame list rather than a TypeID. That is a separate report and a separate memory budget. Do not try to produce both from one pass unless you have measured the working set.
Verification before you trust the numbers
Cross-check the script’s total against a counter. dotnet-counters exposes dotnet.gc.heap.total_allocated from the System.Runtime meter, and the documentation shows it as a cumulative byte counter. If your script’s total allocated bytes for the trace window is within the same order of magnitude as the delta of that counter over the same window, the script is reading the right events. If it is off by 100x, you are reading the wrong keyword or double-counting a merged session.
One more check: allocation sampling is sampling. The runtime does not emit a tick for every allocation. The report is a distribution, not an accounting ledger. Use it to rank types and intervals, not to assert that a type allocated exactly N bytes.
FAQ
Can I read a .nettrace with TraceLog? No. TraceLog is the ETW reader. Use the EventPipe reader for .nettrace, or convert with dotnet-trace convert and accept the conversion’s limitations. The dotnet-trace docs warn that convert and report may not work with the updated .nettrace format produced by collect-linux.
Why is my type name null? The trace was collected without the gcheapandtypenames keyword. The allocation tick carries a TypeID; the name comes from a separate event family. Recollect with the keyword enabled, or accept TypeID-only output.
How do I handle a trace with multiple sessions? Detect the session count before processing. If it is greater than one, split the file or offset timestamps at the seam. Bucketing on relative milliseconds across sessions produces a false spike.
Is there a command-line alternative to scripting? PerfView has command-line modes, and dotnet-trace report exists, but the report verbs are limited and the documentation notes they may not work with newer .nettrace variants. For a custom top-allocators-by-interval view, the TraceEvent library is the supported path.
Sources
- Microsoft, dotnet-trace performance analysis utility, https://learn.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-trace — CLR provider keyword table,
gc-verboseandgc-collectprofile descriptions, buffer overflow warning, truncated-trace and missing-Rundown behavior,collect-linuxformat caveat. - Microsoft, Investigate performance counters (dotnet-counters), https://learn.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-counters —
dotnet.gc.heap.total_allocatedcounter andSystem.Runtimemeter. - Microsoft, Dump collection and analysis utility (dotnet-dump), https://learn.microsoft.com/en-us/dotnet/core/diagnostics/dotnet-dump — dump types and SOS command surface, for the cases where the trace does not answer the question and a heap dump does.
