img2threejs

name img2threejs description Turn an object or character reference image into a quality-gated, animation-ready procedural Three.js model built in code. Use for image-to-3D reconstruction, detail-accurate object rebuilds, stylized/likeness-maximized human characters, sculpt specs, and staged code generation. license Apache-2.0 version 2.0.0 img2threejs — Image to procedural Three.js Rebuild the object visible in a reference image as a code-only procedural Three.js model, gated by a staged sculpting pipeline and an AI-vision self-correction loop. This is reconstruction-by-code, not photogrammetry, mesh extraction, or downloaded art packs. That promise governs how the model is built — it says nothing about which file formats it can subsequently be exported to; an explicitly-selected emission target ( --target <kind> ) is a terminal, whole-artifact transform of the already-built model, verified to its own stated limit, never a second way to build one. Agent-agnostic: works under Claude Code, Codex, or OpenCode. Wherever this doc says "agent vision" or "agent browser tool", use whatever the host provides — native image reading, a browser MCP (playwright/chrome-devtools), the project preview, or a user-supplied screenshot. This file is the always-loaded router: it holds the order of operations and every hard rule as one line. The full contract behind each rule lives in the grimoire/ or docs/ file that rule names — read the named file at the moment you reach that stage, not before. Canonical shared checkout Keep one checkout of this repository and let every host enter it through a symlink, so Claude and Codex execute the same code instead of drifting apart: ~/.claude/skills/img2threejs -> <your checkout> ~/.codex/skills/img2threejs -> <your checkout> When To Use The user attaches/points to an object image and wants a procedural Three.js model, a reconstruction/animation/destruction plan, a sculpt spec, or code. Also for material studies, action-ready props, game objects, botanical/mechanical parts, and stylized reconstructions. Core Promise Sculpt from a photo, in order — never one-shot a mesh: Run python3 forge/next.py --state .img2threejs/state.json [<spec>] first , at every start, resume, and before every correction iteration. It reports the ordered checklist, exact next command, evidence status, and bounded correction-loop status; it never replaces the spec/pass gates. Obey a hard stop; never continue from memory. Validate the image is a suitable 3D target ( grimoire/intake/validation_rubric.md ). Assess object class + complexity, then write a qualityContract before any code. Spec it: component hierarchy, materials, lighting, pivots, sockets, action anchors. Build pass-by-pass from blockout → structure → form → material → lighting → interaction → optimization. Verify each pass with a screenshot compared against the reference; fail a pass if an identity-defining feature is wrong even when the global score looks fine. State explicitly when output is approximate/stylized/low-poly. A single image cannot reveal hidden sides or guarantee exact geometry — say so instead of faking confidence. Mandatory Local State Gate Conversation context is disposable; .img2threejs/state.json is the local checklist authority. Initialize once per reconstruction, then gate every step through it: python3 forge/state.py init --state .img2threejs/state.json --reference <img> --profile <generic|character|installed-domain> --spec object-sculpt-spec.json python3 forge/next.py --state .img2threejs/state.json [object-sculpt-spec.json] python3 forge/state.py mark <step-id> --state .img2threejs/state.json --evidence <path> next.py prints the current step, pass, incomplete mandatory steps, exact next command, and loop/max . Exit code 3 or status=stopped is a hard stop: report the reason and request input. Never bypass it by reconstructing progress from chat history. Every completed step needs evidence; mark a non-applicable step skipped only with --reason — silent omission is forbidden. Loop counts derive from reviewHistory actions ( refine-spec / refine-code ), not agent memory. Defaults: 3 corrections per pass, 6 total. A domain profile's steps, gates and reference material come from the registry : in-repo modules ( character ) and installed plugins ( cs2 , animated-character from plugin-character) register identically, and forge/state.py init names what is available. A profile adds mandatory gates without changing the core order -- a domain plugin typically requires an authoritative classification, an intake manifest, and a machine-readable domain review before AI review; character requires the character contracts and landmark evidence; animated-character (requires the installed plugin-character) adds all of character plus the nine Stage R steps ( grimoire/readiness/animation_contract.md ). Pick it whenever the rig must MOVE — on character the Stage R gates are absent and the build completes without ever running them, which is how animation used to ship broken. Its order is load-bearing: repair the mesh, freeze it, bind additively, then verify parity. Every profile records suitability, projection applicability, and material-evidence applicability. The state file is a resumability index, not visual evidence: renders, specs, review history, and deterministic gates remain the authoritative artifacts. Required Inputs one image path / screenshot / URL / attached image (if missing or unreadable, ask) intended use: prop, game object, hero render, playable/destructible object, animation rig (default: real-time browser prop with interactive performance) when a domain plugin serves the item, whatever authoritative record its intake step requires, or an explicit request for the user/vision provider to supply one; heuristic detection alone is not enough to select a geometry adapter The Loop (scripts do enforcement; agent vision does judgment) Run scripts from the skill root ( forge/... ). Pure Python 3.10+ stdlib, no pip installs. Full flags: grimoire/scripts.md . Never let a script score visuals — that is the agent's job. Analyze the image first (agent vision, before any script): work the layered observation protocol in grimoire/intake/image_analysis.md — identify/classify, decompose macro→meso→micro, map part relationships, name materials in PBR terms, list identity-defining features, and flag what the single view hides. Observation before inference; controlled 3D vocabulary; 3D object-space not 2D image-space. Then probe local images: forge/stage1_intake/probe_image.py <image> (metadata only, not a visual check). 1a. Local Spec Search — after image analysis, before writing or refining a spec, pull local domain evidence (anatomy/PBR/wear/geometry/runtime/physics) rather than inventing it: python3 forge/stage2_spec/new_pre_spec_assessment.py "Name" --image <img> --out assessment.json (auto-runs BM25 over the core_3d collection, or the collection a declared domain contributes -- the collection is NEVER guessed from the target name; writes a localSpecSearch bundle that new_sculpt_spec.py --assessment carries into the spec). Full query-expansion recipe (bilingual terms, focused search_specs.py retrieval, cache rules): grimoire/intake/local_spec_search.md . MUST read it before retrying an incomplete or domain-specific query. 1b. Domain intake — when a domain plugin serves the item, complete its intake steps before pre-spec authoring (admission, heuristic signal, classification, family/route resolution). MUST read the contract its step names, completely, before creating the manifest or running pre-spec assessment. 1c. Optional fidelity evidence adapters — only when they improve an observed weak point; the stdlib core remains authoritative. Thin/complex masks → local SAM2; character face/pose → MediaPipe; weak front/back cues → Depth Anything V2 ( forge/stage1_intake/run_vision_adapter.py <segment|landmarks|depth> ... ; every adapter emits provenance; monocular depth is relative only). MCP-only scene mutations never count as implementation — write the proven change back to the spec or TypeScript, rebuild, recapture. Full adapter + MCP routing and authority boundaries: docs/integrations/reference_fidelity_tooling.md . Pre-Spec Assessment Gate — classify + score complexity + write the quality contract: forge/stage2_spec/new_pre_spec_assessment.py "Name" --image <img> --complexity <simple|moderate|complex|ultra-complex> --out assessment.json . Rules: grimoire/intake/quality_contract.md . Set objectClass.primaryDomain ( object | character | hybrid ) and fill the seeded detailInventory (its targetMinDetails scales with complexity). A domain plugin may raise these floors through its augmentation -- the merge clamps, so a plugin can never lower one (a skin's finish/wear/hardware IS the item, so such a domain is held to the top fidelity bar. Author procedural GEOMETRY but route the FINISH through the projection path in step 2c — a procedural finish for a patterned skin (Doppler/Gamma/Marble/Fade) reads visibly wrong against the reference. A domain plugin ships its own finish rulebook and texture-acquisition guide; read what its checklist steps name. 2b. Detail inventory (do not skip for detailed subjects) — scan zones and enumerate every identity-defining small detail (gloss, bevel, fasteners, linework, contours, stains): forge/stage1_intake/build_detail_inventory.py <image> --mode grid-3x3 --out-dir <dir> --out di.json . Each detail MUST map to a component.localFeatures or material.localOverrides entry — never prose only. Taxonomy + 3D-term recipes: grimoire/intake/detail_inventory.md . 2c. Projection-first fidelity (characters AND reference-matched surfaces — painted skins, decals, painted patterns) — when the goal is matching a specific reference's surface, put the photo's own pixels on the mesh instead of approximating them procedurally. This is the single biggest fidelity lever; a procedural material for a patterned surface is the #1 reconstruction failure. Recipe ( grimoire/character/likeness_maximization.md — its two levers generalize past characters): solve the camera ( stage1_intake/solve_camera_pose.py → referenceCamera ), de-light the reference ( stage1_intake/delight_albedo.py , hard requirement — de-lighting is what makes projection safe), then project the de-lit crop and bake it into UVs ( stage3_build/bake_projected_texture.py --mesh-id <id> ). For a painted skin the projected de-lit crop IS the finish — no procedural Doppler material. For characters, first capture landmarks ( stage1_intake/extract_landmarks.py --out anatomy.json ), fill preSpecAssessment.anatomy , route grimoire/character/reconstruction.md . A single view cannot show hidden sides — report per-region confidence and request more views when it matters. Character sub-routes, in order — decide what parts exist before shaping any, and shape the head before the hair that sits on it: Parts — grimoire/character/structure_decomposition.md Head — grimoire/character/head_construction.md (what the likeness gate reads against) Hair — grimoire/character/stylized_hair_threejs.md + parameter contract in grimoire/character/threejs_hair_parameter_contract.json . Lock topology only after the silhouette review passes: material tuning cannot repair wrong lock topology. 2d. Reference-free humanoid — a generic figure with no reference image has nothing to measure, so fill anatomy from public canon: forge/stage2_spec/humanoid_proportions.py <spec> --style-heads 8 --in-place . It writes anatomy.source: "canon-table" so canon is never mistaken for measurement, refuses to run when the spec names a reference image, and names anything the corpus does not supply rather than interpolating it. Author the spec from the assessment: forge/stage2_spec/new_sculpt_spec.py "Name" --image <img> --assessment assessment.json --augmentation spec-augmentation.json --domain <profile> --out object-sculpt-spec.json (the checklist step carries the resolved flags). Replace generic starter featureReviewTargets with the object's real identity-defining systems (≤5 critical, ≤3 important per pass); for characters add anatomy-proportion , face-landmark-placement , pose-silhouette , outfit-and-palette . Use 3D-graphics terms only ( grimoire/glossary/3d_vocabulary.md ), never "nice/smooth/shiny". Classify every component's topologyClass / topologyRationale per grimoire/intake/surface_topology.md before picking a primitive — this is what prevents a continuous organic form from being picked as a box. When material fidelity matters and a source image exists, analyze each material's finish then extract reference PBR evidence, both per crop (verify the crop is on the part you think it is): forge/stage1_intake/analyze_texture.py <crop> --spec spec.json --material-id <id> --in-place classifies the finish, extracts the gradient palette, and writes doc-grounded MeshPhysicalMaterial scalars onto the material. Recipes + Three.js texture/PBR rules: grimoire/build/threejs_texture_reference.md . Rule of thumb: solid albedo for flat paint, real reference crop for patterned finishes . forge/stage1_intake/extract_pbr_evidence.py <crop> --out-dir <dir> --material-id <id> --target-threshold 0.7 . Confidence < 0.7 is a stop/refine-input signal, not a pass. It is inference, not inverse rendering. For multiple named regions: forge/stage1_intake/material_region_analysis.py --manifest regions.json --out-dir material-evidence --out material-analysis.json , resolve each assignment from docs/materials/material-reference.json , wire it in with forge/stage2_spec/apply_material_analysis.py . Emit the controlled material camera/crop contract ( forge/stage4_review/material_views.py ), compare visible-footprint crops ( material_comparator.py ), apply only bounded material-scoped corrections ( material_feedback.py ), and record the blocking result ( material_gate.py ). Validate, then strict-validate before generating code: forge/stage2_spec/validate_sculpt_spec.py object-sculpt-spec.json then --strict-quality . Strict blocks shallow specs (a complex object with one root, no repetition systems, no local overrides, no micro groups is NOT implementation-ready even if JSON validates). Locked build passes — only touch the currently unlocked pass: forge/stage3_build/orchestrate_passes.py status object-sculpt-spec.json forge/stage3_build/generate_threejs_factory.py object-sculpt-spec.json --out src/createObjectModel.ts The generator is fail-closed: strict-quality must pass before it writes any factory, and a future --pass-id fails until prior passes are reviewed continue . If blocked, preserve the BLOCKED artifact and refine the subject-specific spec; do not substitute a generic template. The local state adds --force only for a new pass or refine-spec ; refine-code edits the current artifact without regenerating it. Before overwriting, carry valid hand refinement back into the spec; generated code must not be the only copy of reconstruction decisions. 6a. Hitting a triangle budget. performanceBudget.targetTriangles selects a tessellation tier for every primitive with segment counts (low ≤6k, standard ≤60k, else hero) and caps implicit-surface sampling grids. Where a tier is not precise enough, add geometryDescriptor.decimate: {"targetRatio": 0.4} to that component — a quadric collapse in the generated factory, run before skin binding so weights are computed on surviving vertices. It keeps position only (normals recomputed), so it is refused on an authored/unwrapped uvStrategy . Offline LOD tiers: forge/stage3_build/decimate.py <mesh.json> --ratio <r> --json . Render the current pass in a browser/preview, capture a screenshot at a review viewpoint. 7a. Off-axis and placement gates — a single review viewpoint is not evidence about the model. Capture a turntable, not one frame, and run all three; each catches a defect class the older gates pass by construction (a hole through a skull, a hat at hip height and a floating charm all survived eight front-only review rounds): forge/stage4_review/turntable_gate.py --capture 0=front.png --capture 90=right.png --capture 180=rear.png --capture 270=left.png --json node runtime/scripts/export_mesh_geometry.mjs --url <preview> --out meshes.json then forge/stage4_review/self_intersection.py meshes.json --json forge/stage4_review/attachment_anchor.py object-sculpt-spec.json --measured measured.json --json All three exit 0 clean / 1 gate failure / 2 error. A failure blocks continue even when the global fidelity score passes. Read sampledVertexCount / unmeasuredAttachments / missingAzimuths before believing a clean verdict: each names what the gate did not look at. Run deterministic gates before AI vision. MUST read grimoire/review/gates_reference.md and grimoire/review/self_correction.md completely. Run forge/stage4_review/diagnose_render.py and record the passing Tier 1 result with --spec object-sculpt-spec.json --pass-id <pass> --in-place ; for non-planar forms also run forge/stage4_review/diagnose_render_multi_angle.py with the fixed view and at least two meaningful orbit views. Then run forge/stage3_build/orchestrate_passes.py check object-sculpt-spec.json --pass-id <pass> . Package one side-by-side sheet, then inspect it with agent vision: forge/stage4_review/make_comparison_sheet.py --reference <img> --render <shot> --out cmp.png --json . Record the review (overall + per-layer + per-feature scores + decision): forge/stage4_review/append_review.py object-sculpt-spec.json --pass-id <pass> --fidelity <0-1> --action <continue|refine-spec|refine-code|request-input|stop> --summary "..." --render-screenshot <shot> --comparison-image cmp.png --ai-vision-score <0-1> --layer-scores-json '{...}' --feature-reviews-json <f.json> --in-place . When a domain plugin contributes a review gate, produce its versioned report first with the command that plugin's review step names, then attach it with --domain-review-json <report>.json --review-scene-json <the plugin's scene fixture> . The checklist step carries the resolved paths. A failed family, painted-region, projection-coverage, critical-detail, or orbit gate blocks continue even when the global score passes. See the plugin's own review-gate documentation. Sync pipeline state after manual review edits, record checklist evidence, then re-run the local state gate before another correction or pass: forge/stage3_build/orchestrate_passes.py sync object-sculpt-spec.json --in-place python3 forge/next.py --state .img2threejs/state.json object-sculpt-spec.json . Before declaring completion, run forge/stage4_review/check_part_coverage.py --spec object-sculpt-spec.json --manifest parts.json and verify the action-ready hierarchy. Mark part-coverage and action-ready only with evidence. GLB-mediated v2 render-fidelity track (1.5 alpha) When the user supplies a GLB as an intermediate reference, the browser-rendered GLB is the structural and visual baseline for an independently authored procedural factory. The raw GLB is never pixel evidence and its topology/materials are never copied into the factory. Before any