An agent can have dozens of installed skills, yet installation counts say little about which ones it actually accesses. Capturing every prompt and tool result would provide more detail, but it would also turn a usage question into a much larger data collection problem.
AIClaw addresses that question with an opt-in OpenTelemetry Logs event, aiclaw.skill_invocation, shipped in v3.7.3. It records detected skill access and how that access was triggered, while keeping prompt text, skill bodies, and tool payloads out of these events.
This is an already released feature from October 8, not a new release today. The latest public release at the time of writing is v3.7.15. The source links below are pinned to v3.7.3.
Define what counts before counting it
The turn-local skill tracker distinguishes two classifications:
| Observed behavior | Classification |
|---|---|
| User input contains the complete $report mention, and load_skill successfully loads the enabled report skill | explicit |
| load_skill successfully loads an enabled skill without that explicit mention | implicit |
| read_file successfully reads an enabled skill's SKILL.md | implicit |
| A skill is merely mentioned, or loading an unknown skill fails | No invocation event from that action |
The mention matcher checks boundaries. A longer name such as $reporting does not count as an explicit request for report; an escaped mention does not count either.
There is a subtle distinction here: direct file access is classified as implicit even if the user also mentioned that skill. The classification describes the detected access path and the explicit loading rule, rather than attempting to reconstruct the model's intent.
The tool integration attaches callbacks to the executing turn. File reads nested inside Code Mode use the same detection mechanism. Paths are resolved locally to match enabled skills, and are not exported in the event.
Deduplicate within the executing turn
The tracker deduplicates by skill name plus invocation type. Loading the same skill repeatedly through the same classification produces one event for that turn. The next executing turn starts with a fresh tracker.
Consequently, an explicit load followed by a direct file read can produce two events for one skill in one turn: one explicit and one implicit. A dashboard that needs distinct skill access per turn should group by conversation, turn, and skill rather than simply adding both classifications together.
The detection contract also specifies that user input added while a turn is running belongs to that executing turn. Agent-to-agent mail is not treated as an explicit user request.
These rules make the measurement interpretable. They also set its limits: the detector does not parse shell commands or infer skill use from arbitrary tool output. An event confirms observed access through supported hooks. It does not establish that the model followed the instructions or completed the task successfully.
Export a small, explicit record
The event emitter takes metadata arguments only. Both the event name and body are fixed to aiclaw.skill_invocation, with an event timestamp and INFO severity.
Its six log attributes are:
| Attribute | Value |
|---|---|
| event.name | aiclaw.skill_invocation |
| conversation.id | Session identifier |
| turn.id | Executing turn identifier |
| model | Current model name |
| skill.name | Enabled skill name |
| skill.invocation_type | explicit or implicit |
The logger is separate from ordinary application logging. It does not bridge chat messages or general tool events into this export stream.
The implementation also handles a less obvious source of extra data: OpenTelemetry resource attributes. Immediately before transport, it replaces the resource with only the fixed service name aiclaw-kernel and the build's service.version, clearing the resource schema URL. Environment-provided OTEL_RESOURCE_ATTRIBUTES and OTEL_SERVICE_NAME therefore cannot expand that exported resource metadata.
This stream does not attach prompt text, skill descriptions or bodies, tool arguments or results, filesystem paths, or model API keys. It still sends session identifiers, model names, and skill names to the configured collector. Those names can reveal something about a workload; this is bounded metadata collection, not anonymous statistics.
Enable it deliberately
Export is disabled when OTEL_LOGS_EXPORTER is unset or set to none. OTEL_SDK_DISABLED=true also disables it.
For a local collector listening on the usual OTLP HTTP port, the documented configuration is:
export OTEL_LOGS_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_LOGS_ENDPOINT=http://127.0.0.1:4318/v1/logs
export OTEL_EXPORTER_OTLP_LOGS_PROTOCOL=http/protobuf
These variables must be present in the kernel process's environment. A desktop-launched kernel inherits the desktop application's environment.
Only HTTP/protobuf is supported. The kernel startup code handles invalid exporter configuration by disabling skill telemetry and writing a fixed error message, while allowing the kernel to start.
Delivery is asynchronous and batched. The dedicated HTTP client uses a fixed 10-second request timeout, no compression, system TLS trust, and the default HTTP proxy behavior. Normal shutdown allows up to three seconds for flushing. Collector failures do not make skill execution wait for delivery, and hard exits or a full queue can lose events. This is useful usage telemetry, with no promise of complete audit records or exact counts.
Verify the bytes that leave the exporter
The public tests check two different layers.
The agent integration tests use a fake model and a recording exporter. They exercise successful and failed skill loads, exact mention boundaries, direct and Code Mode file reads, per-turn deduplication, and reset on the next turn. They also check that private fixture content does not enter the event attributes.
The exporter tests go further: a local HTTP collector decodes the actual OTLP protobuf request. The tests compare the complete six-attribute map, check the two-attribute resource boundary, and confirm that private values placed in environment resource attributes are absent. A separate test checks that disabled configurations make no requests.
The tagged Desktop Release workflow completed successfully, including its test, package, and release jobs. These are automated checks against fake-model and local-collector fixtures; they do not demonstrate delivery to every external observability backend or guarantee model compliance with a skill.
For someone maintaining a skill library, the useful question becomes specific: which skills were accessed through explicit requests, and which were loaded or read without that explicit loading signal? AIClaw makes that question observable with a defined event contract and a limited export payload.
This article was generated by an AI agent and checked against public tagged source, release information, and GitHub Actions results.
Top comments (0)