"Rewrite these two bars, then let me hear them."
That is a reasonable request to make while working on a score. For a browser agent, though, the visible page is a difficult interface: notes are SVG shapes, several voices can occupy the same beat, and a small change in duration can break the measure.
I build ScoreTail, a browser-based sheet music editor with real-time collaboration. I recently added WebMCP tools so an agent can read and edit the score the musician already has open.
The interesting work was deciding what an edit means: how to describe music to the agent, validate its changes, and put them into the same document and undo history as a person's edits.
ScoreTail's public score viewer, showing Pachelbel's Canon in D and the playback controls.
The page exposes the operations
WebMCP is a proposed browser API for registering page-local tools with descriptions and input schemas. In ScoreTail, those tools call the application's existing score operations. The user's current selection and loaded document are already available in the tab.
There is no separate model call inside these tool handlers. The user's agent supplies the reasoning; ScoreTail supplies the score representation, validation, editing, and playback.
I split the tool surface by the page and its editing permissions:
| Surface | Available tools |
|---|---|
| Public score viewer |
get_score_info, read_measures, play, stop_playback
|
| Editor with view access | The viewer tools, plus get_selection and select_measures
|
| Editor with edit access | All of the above, plus rewrite_measures, transpose, undo, and redo
|
This lets someone explore a public score without signing in. Tools that change notation are only registered when the editor has edit access. Read operations also carry readOnlyHint; that hint describes behavior, while the application's access controls remain responsible for permissions.
The smaller surface is useful for the agent too. A public viewer has no reason to advertise a rewrite operation that it cannot perform.
Give the agent a representation it can read back
MusicXML is ScoreTail's interchange format. It is expressive, but it includes details an agent does not need for a small rewrite, including the sequential cursor movements used to encode simultaneous voices.
For reading and writing notes, I reuse a compact text representation from ScoreTail's built-in AI editing feature. I call it lanes-seq. A line identifies a part, staff, voice, and measure:
P1 s1 v1 m2: D5=1/4 E5=1/4 F5=1/2
P2 s1 v1 m2: G2=1
The first line contains a quarter-note D, a quarter-note E, and a half-note F. The second contains a whole-note G in another part. Durations are fractions of a whole note, and pitches are absolute: C4 means middle C. Chords use parentheses:
P1 s1 v1 m3: (C5 E5 G5)=1/2 r=1/2
read_measures returns this notation with a header describing the score's parts and musical context. The agent can then submit the same format to rewrite_measures:
{
"start_measure": 2,
"parts": [1],
"lanes_seq": "P1 s1 v1 m2: D5=1/4 E5=1/4 F5=1/2"
}
This example assumes a full 4/4 measure and the indicated existing part, staff, and voice. A useful workflow reads the score first rather than guessing them.
The replacement unit is one staff in one measure. Supplying its lines replaces all of that unit's voices; leaving a unit out keeps it unchanged. This matters when an agent is asked to change a melody while preserving an accompaniment on another staff.
The tool description includes the format guide generated by the same function used for the built-in AI feature. Maintaining one guide helps keep the two interfaces consistent as the notation format evolves.
There are also practical bounds: a read call accepts up to 64 measures, and a rewrite accepts up to 32. Measure insertion and deletion require an explicit allow_measure_count_change argument.
Parse, plan, then apply
A JSON schema can establish that lanes_seq is a string. Musical structure needs another layer of validation.
For example, this is valid JSON but an incomplete replacement for a full 4/4 measure:
{
"start_measure": 2,
"lanes_seq": "P1 s1 v1 m2: D5=1/4"
}
The rewrite handler uses this pipeline:
tool arguments
-> parse lanes-seq
-> build and validate a rewrite plan
-> start an undo gesture
-> apply through existing score operations
-> select the changed range
The planning stage checks things such as note lines outside the requested measure or part scope and durations that do not fill the measure correctly. It also handles notation rules such as tie and slur repair.
If parsing or planning fails, the tool returns an error starting with Nothing was changed, followed by the validation errors. The agent can correct its input and try again without leaving a partial rewrite behind.
After a successful edit, the changed range is selected in the editor. That gives the person a visible place to inspect before playing it back.
Tool calls need their own undo boundaries
ScoreTail already groups ordinary edits into undo gestures using input events such as pointerdown and keydown. A tool invocation arrives without either event.
That creates a subtle problem: consecutive agent edits need separate undo entries even when there was no human input between them.
The writing tools explicitly call beginUndoGesture() before applying a change. The rewrite then passes through the same operation layer as other edits, as one operation. A successful rewrite can be undone in one step.
One regression test specifically checks this sequence:
rewrite measure 2
save the document snapshot
transpose measure 3
undo once
assert that the document matches the saved snapshot
This catches accidental merging of two tool calls into one undo entry. Another test checks that a valid rewrite followed by one undo restores the original score.
In the current implementation, rewrites apply immediately. There is no separate preview-and-approve stage in the WebMCP handler. That makes clear tool descriptions, visible changes, and predictable undo especially valuable. The agent should still obtain whatever approval the user's request requires before making an edit.
One shared document
ScoreTail stores the live score in Yjs. WebMCP changes go through the existing operation layer into that same document, so they follow the existing synchronization path to collaborators.
Human UI ----\
-> score operations -> Yjs document -> rendering / sync
WebMCP -----/
There is no second AI-owned score that later needs to be merged into the editor.
There is still a limit to what this guarantees. Shared document convergence does not tell us whether a change is a good arrangement, or whether another musician wanted it. Validation protects structural rules; the musician still judges the musical result.
Lifecycle and audio details
The bridge feature-detects document.modelContext and waits for the score data to be ready before dynamically loading the tools. If the API is unavailable, the normal editor continues without registering them.
Registrations use an AbortSignal. The bridge aborts it on cleanup, including when its permission-related dependencies change. Playback callbacks use a ref so registered tools can call the latest playback function without re-registering on every render.
Playback also needs to respect browser audio policies. When the page has never received user activation, the handler returns an error asking for a click. Otherwise an AudioContext.resume() can remain pending and leave the agent waiting for a result. The tool reports that situation instead of pretending playback started.
What is verified, and what remains experimental
The current WebMCP score-tool suite has 15 passing tests. It checks permission-dependent tool sets, scoped reads, successful and rejected rewrites, explicit measure-count changes, undo grouping, transposition, selection, and playback status handling.
These tests call the handlers with a real in-memory score model; playback is mocked. They establish application behavior, rather than universal compatibility with every browser agent. The public demo also loads and enters playback through its normal UI.
WebMCP itself is still experimental. Chrome's setup documentation covers its origin trial and local testing flag. A supporting browser alone also does not guarantee that a particular assistant discovers and calls the tools. I would check the browser and agent combination before planning a demo.
Try a public score
Open Canon in D in ScoreTail with a compatible WebMCP setup. The public viewer exposes reading and playback tools without requiring an account.
A starting request is:
Read the first eight measures and explain the chord progression, then play them.
For editing, copy a score into the full editor with edit access. Read a small range first, make one bounded change, listen, and try undo. WebMCP is currently attached to the full editor and public score viewer; the guest quick-edit workspace does not expose it.
I'm also working toward a library where public-domain sheet music can be opened, played, corrected, and arranged together, with OMR helping bring scanned scores into that workflow. These tools give an agent a way to assist with a specific passage while the musician stays in the same editor.
If you are adding agent tools to a creative app, how are you handling changes to a document that another person may be editing at the same time?
Top comments (1)
WebMCP is particularly interesting in a collaborative editor because the agent needs a clear model of both the document and who may change it. I would make the tool contract expose selection, revision, and permission state together, then require confirmation for a destructive edit. That keeps an agent from treating a shared score like an isolated local file.