superpowers-rest-automation

name superpowers-rest-automation description Builds reliable automations that integrate with REST APIs: auth, pagination, retries, rate limits, idempotency, webhooks, data mapping, and safe error handling. Use when calling external APIs, syncing systems, or building ETL-style workflows. REST Automation Skill This skill enforces reliability and safety when building automations that call REST APIs. When to use this skill Use whenever the task involves: calling external REST APIs (CRUD, search, sync) integrating 2+ systems (ETL, iPaaS-like flows) webhooks, polling, or scheduled jobs data ingestion, normalization, enrichment, deduplication Default design principles Idempotent by design : repeats should not create duplicates or corrupt data. Observable : logs/metrics correlate each run and each API call. Fail safe : handle partial failures; avoid silent data loss. Rate-limit aware : backoff and respect vendor limits. Least privilege : handle secrets safely, avoid overbroad scopes. Checklist (apply unless irrelevant) 1) Define the contract Inputs (format, required fields, validation) Outputs (where data goes, expected shape) Success criteria (what “done” means) Non-goals (what the automation will not do) 2) Authentication & secrets Identify auth type: API key, OAuth2, JWT, mTLS Never hardcode secrets in code or logs Support secret injection via env vars / secret manager Plan token refresh if applicable (OAuth2) 3) Idempotency & deduplication Pick at least one: Use provider idempotency keys (if supported) Use stable external IDs (e.g., external_id field) for upserts Keep a local/state store mapping source IDs -> target IDs Use deterministic hashes for dedupe when no stable ID exists Document the idempotency strategy explicitly. 4) Pagination & incremental sync Detect pagination style: next link, cursor, page+limit, offset+limit Ensure loops terminate safely (max pages / max time) Prefer incremental sync using updated_since /ETag/If-Modified-Since when possible Handle out-of-order updates and late-arriving events 5) Retries, backoff, and timeouts Set timeouts for connect/read Retry on transient errors: network failures, 429, 5xx (with limits) Use exponential backoff with jitter if possible Do not retry on most 4xx (except 408/409/429 depending on semantics) Cap retries and surface failures clearly 6) Rate limits & quotas Respect Retry-After and rate-limit headers Implement adaptive backoff on 429 Consider batch endpoints to reduce call volume Avoid bursty concurrency unless explicitly safe 7) Data mapping & validation Explicit mapping layer (source -> normalized -> target) Validate required fields and types Normalize common formats (dates, enums, currency, locales) Handle nullability and partial payloads Record rejected records with reasons (don’t silently drop) 8) Error handling strategy Choose and document per error class: Skip with log (non-critical record) Retry (transient) Quarantine (store failing payload for later) Fail the run (systemic issue) Ensure the workflow reports a clear summary at the end. 9) Observability & audit trail Minimum: Run ID / correlation ID Per-request logs: method, path (not full secrets), status, latency, attempt count Counters: processed, created, updated, skipped, failed Prefer structured logs (JSON) if possible. 10) Webhooks (if involved) Verify signature (if provided) Handle replay (idempotency for event IDs) Respond quickly; process async if needed Store raw event payloads (optional but recommended) 11) Safety controls Dry-run mode (no writes) Limit scope (max records per run) “Kill switch” config flag Backups/rollback plan for destructive operations Output requirements (when producing a solution) Include: Idempotency strategy (1–3 bullets) Retry/backoff policy Pagination/incremental sync approach (if relevant) Error handling strategy + what gets logged/quarantined Verification plan (tests or a safe sandbox run plan)