lex-data

name lex-data description Represent, parse, serialize, hash, or convert AT Protocol data values with `@atproto/lex`. Use whenever code touches CIDs, raw bytes, `$link` / `$bytes` JSON, CBOR / DRISL encoding, blob references, or AT Protocol identifier and datetime strings — DIDs, handles, AT URIs, NSIDs, TIDs, record keys, `createdAt` / `indexedAt`. Applies when the ask sounds like "serialize this record", "hash these bytes", "why is this datetime rejected", "count the characters in this field", or when replacing `multiformats` CIDs, `new Date().toISOString()`, or hand-rolled base64. Read it before typing a DID, AT URI, or datetime as a bare `string`. disable-model-invocation false Data model: types, CIDs, JSON, CBOR, identifier strings, blobs The AT Protocol data model is JSON plus two primitives — CIDs (content-addressed links) and bytes (raw binary) — encodable as JSON or as CBOR. Which package exports what @atproto/lex re-exports lex-data , lex-json , lex-schema and lex-client , so a package that already depends on it needs no extra import path. @atproto/lex-cbor is not re-exported — depend on it explicitly. Symbols Defined in Reachable via @atproto/lex LexValue , Cid , blob refs, base64/utf8 helpers, LexError @atproto/lex-data yes lexParse , lexStringify , jsonToLex , lexToJson @atproto/lex-json yes Identifier & datetime string types + guards @atproto/syntax , re-exported by @atproto/lex-schema yes encode , decode , decodeAll , cidForLex @atproto/lex-cbor no Low-level packages ( repo , common , xrpc-server ) import @atproto/lex-data directly to avoid pulling in the client and schema layers. [!NOTE] This repo sets verbatimModuleSyntax: true , so type-only names need the type modifier — import { type DidString, isDidString } from '@atproto/lex' . All examples below follow that form. Lex value types import { type LexValue , type TypedLexMap , isLexValue, isTypedLexMap, } from '@atproto/lex' // LexScalar: number (integers only) | string | boolean | null | Cid | Uint8Array // LexValue: LexScalar | LexValue[] | { [key: string]?: LexValue } // LexMap: { [key: string]?: LexValue } // TypedLexMap: LexMap & { $type: string } isLexValue , isLexMap , isLexArray and isLexScalar take unknown ; isTypedLexMap takes a LexValue , so narrow with isLexValue first when the input is untyped. isLexValue walks the whole structure and rejects cycles, non-integer numbers, and non-plain objects ( Date , class instances, functions) — it is not a cheap check, so avoid it on hot paths where a schema already validates the value. CIDs Cid is an interface, not a class. @atproto/lex-data defines it rather than re-exporting multiformats ' CID , because that class breaks under node16 / bundler module resolution. Treat CID and asMultiformatsCID as deprecated escape hatches. import { type Cid , asCid, decodeCid, ifCid, isCid, parseCid, parseCidSafe, } from '@atproto/lex' parseCid (str) // Cid — throws on invalid parseCidSafe (str) // Cid | null decodeCid (bytes) // Cid from binary form — throws on invalid isCid (value) // type guard ifCid (value) // Cid | null asCid (value) // Cid — throws on invalid Each accepts a { flavor } option that additionally constrains the CID, which is how you enforce that a blob CID really is a blob CID rather than trusting the caller: 'raw' — v1 + raw multicodec (blob content) 'cbor' — v1 + dag-cbor + sha256 (records, MST nodes) 'dasl' — either codec, v1 + sha256 For strings you already hold, validateCidString(str, options) returns a boolean and also checks the string is the canonical encoding; ensureValidCidString throws instead. Creating CIDs — hashing is async because it goes through WebCrypto: import { cidForCbor, cidForRawBytes, cidForRawHash, isCidForBytes, } from '@atproto/lex' import { cidForLex } from '@atproto/lex-cbor' await cidForLex (record) // CBOR-encode then hash — the usual entry point await cidForCbor (cborBytes) // hash bytes that are already CBOR await cidForRawBytes (blobBytes) // raw CID for blob content cidForRawHash (sha256Digest) // sync, when you already streamed the hash await isCidForBytes (cid, bytes) // verify a CID matches its content In Lex JSON, CIDs are encoded as { "$link": "bafyrei..." } . Bytes Binary data is Uint8Array ; JSON-encoded as { "$bytes": "base64..." } . @atproto/lex-data ships the byte utilities so packages don't add their own base64 dependency: toBase64 / fromBase64 (both take an optional 'base64' | 'base64url' alphabet), utf8ToBase64 / utf8FromBase64 , utf8FromBytes , ui8Concat , ui8Equals , asUint8Array , ifUint8Array . JSON ↔ Lex Function Direction Input Output lexParse JSON string → Lex string LexValue (with Cid , Uint8Array ) lexStringify Lex → JSON string LexValue string jsonToLex parsed JSON → Lex plain JS object LexValue lexToJson Lex → plain JS LexValue plain JS object lexParse<T>() takes a type parameter, so no as cast is needed: import { lexParse, lexStringify } from '@atproto/lex' const sub = lexParse<app. bsky . notification . defs . SubjectActivitySubscription >(json) const out = lexStringify ({ ref : someCid, data : someBytes }) lexParse and jsonToLex accept { strict } , defaulting to false : malformed $link / $bytes objects and non-integer numbers pass through unchanged rather than throwing. Pass { strict: true } when the input is untrusted and a silently-wrong value would be worse than an exception. lexParseJsonBytes(bytes, options) is the same parse starting from UTF-8 bytes. Field-level encode/decode is rarely needed, but note the return contract — these signal failure by returning undefined , they do not throw: import { encodeLexBytes, encodeLexLink, parseLexBytes, parseLexLink, } from '@atproto/lex' parseLexLink ({ $link : 'bafy...' }) // Cid | undefined parseLexBytes ({ $bytes : 'SGVsbG8=' }) // Uint8Array | undefined encodeLexLink (someCid) // { $link: '...' } encodeLexBytes ( new Uint8Array ([ 1 , 2 , 3 ])) // { $bytes: '...' } CBOR (DRISL) Use CBOR for repo storage, signed records, CAR files, and event frames — anywhere DRISL deterministic encoding is required. Add @atproto/lex-cbor as a dependency; it is not reachable through @atproto/lex . import { type LexValue } from '@atproto/lex' import { cidForLex, decode, decodeAll, encode } from '@atproto/lex-cbor' const bytes : Uint8Array = encode (someLexValue) const value = decode<{ foo : LexValue }>(bytes) // Concatenated values (CAR blocks, subscription frames) for ( const frame of decodeAll (buffer)) { /* … */ } encode throws on anything the AT data model forbids: non-string map keys, undefined , and non-integer numbers. That strictness is the point — it is what makes the resulting CID stable. Datetime strings DatetimeString is the branded type for AT Protocol datetimes. Date.prototype.toISOString() is not guaranteed to conform (years outside 0–9999 serialize with a ±YYYYYY prefix), so route Date → string through the helpers rather than calling toISOString() directly: import { type DatetimeString , asDatetimeString, currentDatetimeString, ifDatetimeString, isDatetimeString, toDatetimeString, } from '@atproto/lex' currentDatetimeString () // now toDatetimeString (date) // throws InvalidDatetimeError on an out-of-range date asDatetimeString (str) // validates + brands a string, throws on invalid isDatetimeString (value) // type guard ifDatetimeString (value) // DatetimeString | undefined isDatetimeStringLenient accepts ISO-ish strings that the spec rejects (e.g. missing a timezone) — use it when reading legacy records, not when writing. For repairing bad historical createdAt values, normalizeDatetime / normalizeDatetimeAlways live in @atproto/syntax only; they are not re-exported from @atproto/lex . Use DatetimeString on record and DB row types so a raw string can't drift in: import type { DatetimeString , DidString } from '@atproto/lex' interface Row { did : DidString indexedAt : DatetimeString } Identifier strings Nominal string types keep unvalidated string s from being passed where a validated identifier is expected. Type Format Guard DidString did:method:specific-id isDidString HandleString DNS-style handle isHandleString AtIdentifierString DID or handle isAtIdentifierString AtUriString at://… isAtUriString UriString any scheme:… URI isUriString NsidString app.bsky.feed.post isNsidString TidString timestamp identifier isTidString RecordKeyString rkey (TID or literal) isRecordKeyString CidString CID in string form isCidString LanguageString BCP-47 tag ( en , pt-BR ) isLanguageString DatetimeString AT Proto datetime isDatetimeString Only AtUriString , AtIdentifierString and DatetimeString also ship as… / assert… / if… variants. For the rest, the generic format helpers cover the same ground: import { asStringFormat, ifStringFormat, isStringFormat } from '@atproto/lex' isStringFormat (value, 'did' ) // narrows to DidString asStringFormat (value, 'nsid' ) // throws TypeError on invalid ifStringFormat (value, 'handle' ) // HandleString | undefined isAtUriString , isLanguageString and isStringFormat accept { strict: false } for a lenient pass that tolerates non-conforming real-world data. [!NOTE] These types are defined in @atproto/syntax and re-exported through @atproto/lex . Import from @atproto/lex where the package already depends on it — that keeps identifier types and generated schemas coming from one place and avoids adding a @atproto/syntax dependency. Boundaries Protobuf messages, data-plane responses and Kysely rows all arrive as bare string . Brand them once at the boundary so downstream code is typed, instead of asserting at each use: dids : dids as DidString [], post : { uri : item. uri as AtUriString , cid : item. cid || undefined }, An as cast is appropriate where the value provably came from a validated source (your own DB, a schema-validated response). Where it did not — env vars, CLI args, user input — run the guard: assert(isDidString(serverDid)) . Blob references Two shapes coexist on the network: type TypedBlobRef = { $type : 'blob' ; ref : Cid ; mimeType : string ; size : number } type LegacyBlobRef = { cid : string ; mimeType : string } // no $type, no size type BlobRef = TypedBlobRef | LegacyBlobRef New uploads are always TypedBlobRef — the PDS uploadBlob endpoint returns one. Reading code has to tolerate both, which is what the format-agnostic accessors are for: import { type BlobRef , enumBlobRefs, getBlobCid, getBlobCidString, getBlobMime, getBlobSize, isLegacyBlobRef, isTypedBlobRef, } from '@atproto/lex' getBlobCid (blob) // Cid — parses the string for legacy refs, so it can throw getBlobCidString (blob) // string — skips the parse, prefer it when you only need the string getBlobMime (blob) // string getBlobSize (blob) // number | undefined — legacy refs carry no size // Deep-walk a record for its blobs (what the PDS does on record write/import) for ( const ref of enumBlobRefs (record, { allowLegacy : true , strict : false })) { /* … */ } enumBlobRefs skips legacy refs unless allowLegacy: true . BlobRef is an interface, so instanceof does not apply — use isBlobRef / isTypedBlobRef / isLegacyBlobRef . What strict means for blobs The word means different things at two layers, and conflating them causes confusing validation failures: On the guards ( isBlobRef , isTypedBlobRef , isLegacyBlobRef ,