tldraw-offline

name tldraw-offline description Drive and script tldraw offline canvases with an agent. version 1.0.0 author Teknium + Hermes Agent license MIT platforms ["linux","macos","windows"] metadata {"hermes":{"tags":["tldraw","canvas","whiteboard","document-script","diagramming"],"category":"creative","related_skills":[]}} tldraw offline Skill Work with the tldraw offline desktop app (offline.tldraw.com): read the open canvas, make edits, and write document scripts — JavaScript embedded in a .tldraw file that runs on load and gives the file durable behavior. The app runs a local HTTP API (default localhost:7236 ) that a coding agent drives with plain curl from its terminal — this is exactly how the app's own homepage demo (Codex editing a canvas live) works. The agent does NOT use computer-use / GUI clicking, and does NOT hand-edit the .tldraw file directly. Keep tldraw offline open while you work. When to Use The user has tldraw offline open and asks you to build or modify a canvas (diagrams, wireframes, layouts). You want to add durable behavior to a drawing (reactive shapes, interactive buttons, animation, connection logic) via an embedded document script. Do NOT hand-place shapes to imitate a drawing — write the code that generates them. Agents are far better at scripting the canvas than at drawing on it. Prerequisites tldraw offline installed and running , with a document open. Releases: https://github.com/tldraw/tldraw-offline/releases/latest (macOS DMG, Windows x64/Arm64, Linux x86_64 / arm64 AppImage or amd64/arm64 .deb ). Agent skills installed in the app : Develop → Install Agent Skills . The app writes its own tldraw skill into ~/.codex/skills/ , ~/.claude/skills/ , ~/.cursor/skills/ , and ~/.gemini/skills/ — teaching that agent the curl recipes below. (This Hermes skill mirrors that guidance for Hermes.) The local control API. On launch the app writes server.json to its config dir (Linux ~/.config/tldraw/ , macOS ~/Library/Application Support/tldraw/ , Windows %APPDATA%\tldraw\ ) with port (default 7236 ), a bearer token , pid , and startedAt . Every request except GET / needs Authorization: Bearer <token> . A clean quit removes server.json ; if it's present but the port doesn't answer, the app quit uncleanly — treat as not running. Re-read port + token on EVERY shell call. Each terminal call is a fresh shell, so an export ed token does not persist — "export once and reuse" sends an empty token and 401s. Read both inline at the top of each call: PORT=$(jq -r .port <server.json>); TOKEN=$(jq -r .token <server.json>) . No account or network needed for local editing. How to Run Two distinct workflows. Pick by whether the change must survive a reload. A. One-off canvas edits ( /exec ) — layout, generating shapes, cleanup. This is a live edit, not saved script: BASE=http://localhost:7236 TOKEN=$(python -c "import json;print(json.load(open(' $HOME /.config/tldraw/server.json'))['token'])" ) # find the focused document id DOC=$(curl -s " $BASE /api/search" -X POST -H 'content-type: application/json' \ -H "Authorization: Bearer $TOKEN " \ -d '{"code":"return (await api.getFocusedDoc()).id"}' | python -c "import sys,json;print(json.load(sys.stdin)['result'])" ) # run code with the live `editor` + `helpers` in scope curl -s " $BASE /api/doc/ $DOC /exec" -X POST -H 'content-type: application/json' \ -H "Authorization: Bearer $TOKEN " \ -d '{"code":"const {createShapeId,toRichText}=await import(\"tldraw\"); editor.createShape({id:createShapeId(),type:\"geo\",x:0,y:0,props:{geo:\"rectangle\",w:200,h:100,color:\"blue\",fill:\"solid\",richText:toRichText(\"hello\")}}); return editor.getCurrentPageShapes().length"}' B. Durable behavior ( script/main.js ) — reactive/interactive logic that must survive reload. Edit the file on disk; the app's watcher applies it: # get the live script file path for the doc curl -s " $BASE /api/doc/ $DOC /script-workspace" -X POST \ -H "Authorization: Bearer $TOKEN " # -> result.mainJsPath, result.isDefaultScript # edit result.mainJsPath with read_file / patch / write_file (see scripts/main.js) # then confirm the watcher applied it: curl -s " $BASE /api/doc/ $DOC /script-status" -H "Authorization: Bearer $TOKEN " The ready-to-adapt document script is scripts/main.js . Quick Reference The document-script contract (verified against the app's bundled script-context.d.ts ): import { createShapeId, toRichText } from 'tldraw' // primitives: import, not globals export default function ( { editor, helpers, signal } ) { editor. run ( () => { // batch = one undo step helpers. createShapeIfMissing ({ // idempotent furniture id : createShapeId ( 'node-1' ), type : 'geo' , x : 0 , y : 0 , props : { geo : 'rectangle' , w : 200 , h : 100 , richText : toRichText ( 'hi' ) }, }) }) const stop = editor. store . listen ( () => { /* react */ }) // fires the tick AFTER a commit signal. addEventListener ( 'abort' , () => stop ()) // REQUIRED cleanup on rerun/close } ctx.editor — the live Editor ( createShape , updateShape , deleteShapes , getCurrentPageShapes , getShape , getBindingsFromShape , zoomToFit , on('tick'|'event', fn) , run(fn, { history: 'ignore' }) ). ctx.helpers — createShapeIfMissing , createShapesIfMissing , createArrowBetweenShapes(from, to, { arrowheadEnd }) , translateShapes , onShapeTranslate(id, fn, { signal }) , richTextToPlainText , boxShapes , getLints . ctx.signal — AbortSignal ; attach every listener/interval teardown to it. config.js (separate file) registers custom shape/tool/component utils and runs before mount; main.js runs against the mounted editor and reruns on save. Interactive UI (clickable buttons that drive state) Drawn shapes can behave like a real app — the thing a static whiteboard can't do. Full example: scripts/counter.js (a number display + MINUS/RESET/PLUS buttons). Verification boundary — read this before claiming interaction works or doesn't. The app's OWN agent playbook says to verify a clickable-UI script with "one simulated click and one state read" via /exec ( editor.dispatch a pointer event, await a tick, read the shape's state) — NOT by driving a real mouse. By that standard the counter is verified: dispatched clicks stepped it 0 → 1 → 2 → 1 → 0 . Two caveats worth writing down: The script only runs once the app's file-watcher applies it. On Linux that watcher uses inotify; a host with an exhausted fs.inotify.max_user_instances logs inotify_add_watch ... No space left on device , script-status shows state: "not-watching" / hasEntry: false , and the script never executes. This is a host limit, not a script bug — a normal desktop is unaffected. Driving the canvas with computer-use needs FOREGROUND delivery, not background. cua-driver's default background delivery returns background_unavailable against Electron's occluded renderer — but that is the first rung, not a wall. cua-driver returns an escalation: "foreground" hint; set delivery_mode: "foreground" (pair with bring_to_front ) and its X11 XTest path ( x11_xtest_fg ) clicks land — this dismisses the "Run Script" consent dialog and clicks the canvas. Verified by the Cua team on this exact v1.11.0 AppImage (Linux/X11). Do NOT conclude "Electron rejects synthetic clicks" and give up in background mode — climb to foreground. (The real product path is still /exec , not clicks; this note is for computer-use-driven testing.) The pattern: export default function ( { editor, helpers, signal } ) { // 1. Build buttons idempotently; tag each with meta so the handler finds them. // Give buttons a visible label AND a meta.action. // 2. Hit-test pointer_down in PAGE coordinates against the button bounds: const inside = ( b, p ) => p. x >= b. x && p. x <= b. x + b. w && p. y >= b. y && p. y <= b. y + b. h function onEvent ( info ) { if (!info || info. name !== 'pointer_down' ) return let p = null try { if (info. point && editor. screenToPage ) p = editor. screenToPage (info. point ) } catch {} p = p ?? editor. inputs ?. currentPagePoint if (!p) return const hit = editor. getCurrentPageShapes (). find ( ( s ) => s. meta ?. ui === 'button' && inside ({ x : s. x , y : s. y , w : s. props . w , h : s. props . h }, p) ) if (hit) runAction (hit. meta . action ) // mutate state; store it in a shape's meta } editor. on ( 'event' , onEvent) signal. addEventListener ( 'abort' , () => editor. off ( 'event' , onEvent)) // REQUIRED } Find buttons by meta (or visible label via helpers.richTextToPlainText ), not by hard-coded coordinates. One script owns both build and read. If the shapes are created by one code path (with meta.action: 'inc' ) and the handler reads another convention ( meta.action === 'PLUS' ), clicks silently do nothing. Ship the buttons built by the same script that handles them, or ship an empty canvas so the script builds them fresh — never pre-bake mismatched shapes into the file's db. Keep app state in a shape's meta (e.g. meta.count ) and render it as that shape's richText label, so it survives save and is readable for verification. Detach the listener on signal abort. Skipping this is not cosmetic: on the next save the old onEvent stays attached alongside the new one, so every click fires twice and a counter jumps by 2 instead of 1. For continuous motion use editor.on('tick', fn) ; for a moving anchor with attached pieces use helpers.onShapeTranslate(id, fn, { signal }) . Shipping a self-running scripted .tldraw A .tldraw is a zip of metadata.json + session.json + db.sqlite + assets/ script/ (only those entries are packable). For the script to auto-run without the "This document contains a script → Run Script" consent dialog: metadata.json must carry a script manifest: { "sha256": "<digest>" } , where the digest is sha256 over each sorted script/ path as `${path}\0${sha256hex(bytes)}\n` . A mismatch is rejected as tampered. Pre-trust the digest by adding it to ~/.tldraw/script-trust.json ( { "trusted": ["<digest>"] } , or $TLDRAW_SCRIPT_TRUST ). The app skips consent when isScriptTrusted(digest) is true. Procedure Read the current token/port from server.json . Find the target doc with api.getFocusedDoc() (or api.getDocs() ); name it explicitly if several are open. For layout/generation, use /exec . For durable behavior, edit script/main.js via /script-workspace . Make scripts idempotent: create durable shapes with helpers.createShapeIfMissing and stable createShapeId('name') ids. Scripts rerun on every load. Keep script-owned writes out of the user's undo stack: editor.run(fn, { history: 'ignore' }) (or helpers.translateShapes , which already does). For reactivity, editor.store.listen(cb) and tear it down on signal abort. For interaction, editor.on('event', h) (hit-test pointer_down in page coords); for animation, editor.on('tick', h) . For a single moving anchor + attached internals, prefer helpers.onShapeTranslate(anchorId, fn, { signal }) over a broad store listener — a broad listener can turn your own writes into feedback loops. Shape props (validated against tldraw SDK v5 schema) editor.createShape / createShapeIfMissing accept partial props (shape utils fill defaults). When building raw records for a file snapshot, every prop below is required (run scripts/validate_shapes.mjs ): Shape Required props note richText , color , labelColor , size , font , align , verticalAlign , growY , fontSizeAdjustment , url , scale , textLastEditedBy text richText , color , size , font , textAlign , w , scale , autoSize frame w , h , name , color geo geo , w , h , color , fill , richText (+ dash/size/etc. defaulted) richText must be toRichText('...') — a bare string is rejected. color enum: black grey light-violet violet blue light-blue yellow orange green light-green light-red red white . font enum: draw sans serif mono . Pitfalls store.listen fires on the tick AFTER a commit, not synchronously. If you write a shape and immediately read state expecting the listener to have run, it hasn't. Verified live: an in-turn read shows 0 fires; after one setTimeout tick it shows 1. Same reason the app notes editor.dispatch is async — await a tick before verifying. ctx , not globals. The entry is export default function ({ editor, helpers, signal }) . There is no bare editor global in a document script. createShapeId / toRichText / Vec come from import ... from 'tldraw' . richText , not text . Text/note/geo labels use richText: toRichText(s) . Raw records need every prop; createShape does not. In-app pass only the props you care about; a hand-built .tldraw snapshot needs the full set (table). Scripts rerun on every load — be idempotent. Use createShapeIfMissing with stable ids or you duplicate content and clobber user edits. Clean up on signal . signal.addEventListener('abort', () => stop()) for