powertoys-verification

name powertoys-verification description Verify PowerToys behavior end-to-end with the winapp CLI across two scenarios: (A) a module's release checklist against the installed build; (B) PR validation — derive each PR's checklist from its description + diff, then drive it against the installed build (a merged/shipped PR, or a whole release/hotfix set) or by building + sideloading the module when the PR isn't in the build yet (unmerged or not-yet-released). Drive each item via UIA invoke / Named Events / settings.json edits / clipboard / GPO / SendInput, and emit a structured PASS / FAIL / BLOCKED verdict per item with evidence (FAIL distinguishes product defects from stale/ambiguous checklist items). Use when asked to verify a module checklist, validate a PR, sign off a release/hotfix's PRs, or QA installed/sideloaded PowerToys bits. Combines generic winapp ui mechanics (references/winapp-ui-testing.md) with PT-specific recipes, per-scenario playbooks (references/scenarios/), and the helper .ps1 files shipped with this skill. license Complete terms in LICENSE.txt When to use this skill Use this skill whenever you need to verify PowerToys behavior with the winapp CLI and emit a structured PASS / FAIL / BLOCKED verdict with evidence (UIA enumeration, log line, settings.json diff, screenshot, etc.). It runs in two scenarios that share the same drive techniques, helpers, taxonomy and report format — and differ only on what the checklist is and what bits you test: Scenario Trigger Checklist source Bits under test A — Module checklist "verify all <Module> items", "sign off Color Picker" Supplied file ( references/release-checklist/<module>.md ) Installed shipped artifact (read-only) B — PR validation "validate PR #N" (open or merged), "build it and test the fix", "verify the PRs in this release/hotfix", "sign off 0.X.Y" Derived from each PR's description + diff Installed shipped artifact if the code is already in the build; build + sideload if it isn't (unmerged / not-yet-released) Step 0 for every run — pick the scenario. Read references/scenarios/index.md (the router + the "bits under test" contract + the verdict-vocabulary mapping), then read the one matching scenario doc ( references/scenarios/{module-checklist,pr-validation}.md ). For B , also resolve the bits sub-decision ("is the PR's code in the build under test?") before driving. This SKILL.md is the shared engine (drive techniques, helpers, taxonomy, pitfalls) common to both. Each item produces a PASS / FAIL / BLOCKED verdict with evidence. For A the checklist is supplied; for B you derive it from the PR(s). The skill is the how — independent of any specific checklist. Required reads (in order) references/scenarios/index.md — read FIRST : the scenario router (A/B), the "bits under test" contract (installed-and-immutable, vs build-and-sideload for a PR whose code isn't in the build under test), and the verdict-vocabulary mapping. Then read the one matching scenario doc: references/scenarios/module-checklist.md (A) · pr-validation.md (B). references/winapp-ui-testing.md — the prerequisite UIA mechanics doc (winapp ui verbs, scripted batch testing, file pickers, accessibility audits, screenshots, click-vs-invoke, PostMessage, SendInput cb=40, stunted-UIA recovery, settings-mutation safety contract). Read this first — this skill assumes you know its content and only adds PT-specific extensions. This SKILL.md — the shared engine: the 3-bucket drive-technique selector (Step 2), classification taxonomy, critical pitfalls, helper-script catalog. references/modules/<module>.md IF IT EXISTS — per-module entry-paths, item-by-item recipes, common BLOCKED traps, fixture lists, source citations. Always check references/modules/ first. If no profile exists, fall back to this SKILL.md and create one after you finish (template in references/modules/README.md ). references/explorer-context-menu-flow.md IF your module registers an Explorer right-click entry (PowerRename, File Locksmith, Image Resizer, New+, Preview Pane, RegistryPreview) — shared synthetic-right-click + UIA-invoke + multi-file-selection flow + module-caption table. Helper: scripts/pt-explorer-contextmenu.ps1 . references/pre-flight.md — pre-flight checks, bootstrap snippet, state-hygiene cleanup, final wrap-up, hard rules. references/reporting-format.md — per-item table template, top-of-report summary, step-table rules, anti-patterns, worked example. references/environment-setup.md — RDP/sleep/screensaver/session-attachment gotchas. Cite in BLK-ENV verdicts. references/release-checklist/<module>.md — SCENARIO A ONLY — the supplied checklist for the module under test (one file per module; see references/release-checklist/index.md ). Each item carries [ADMIN: …] + [CLARITY: …] metadata. This file IS the set of items to verify. For B the checklist is derived from the PR(s) instead (see the scenario doc). Helper scripts shipped with this skill File Purpose scripts/pt-shared-events.ps1 Invoke-PtSharedEvent , Test-PtSharedEvent , Get-PtSharedEventCatalog — 56-entry friendly-name map for PT Named Events (CmdPal.Show, AOT.Pin, PowerLauncher.Invoke, LightSwitch.Toggle, ZoomIt.Draw, ...). The deterministic, foreground-free, UIPI-immune way to trigger a module. scripts/pt-sendinput-chord.ps1 Send-PtChord , Wait-PtHotkeyAccepted — last-resort SendInput hotkey injection with the cb=40 fix. Use only when the module has no Named Event and the hotkey itself is the test subject. scripts/pt-foreground-guard.ps1 Test-PtForeground , Force-PtForeground , Assert-PtForegroundOrAbort — guard helpers to ensure target window IS foreground before SendInput, so keys don't leak to caller's terminal. scripts/pt-cmdpal-recycle.ps1 Reset-CmdPalAppX , Reset-CmdPalToHome , Test-CmdPalDegraded , Invoke-CmdPalQuery — CmdPal-specific lifecycle (handles TextChanged-broken state, BackButton navigation, AppX recycle). scripts/pt-admin-probe.ps1 Test-PtAdmin , Test-ProcessElevated , Test-PtRunnerAdmin — TokenElevation probes to verify your session and the PT runner have the right elevation for the test. scripts/pt-clipboard-diff.ps1 Get-PtClipboardFormats , Compare-PtClipboardFormatDiff , Set-PtClipboardRich — multi-format clipboard inspection for Advanced Paste tests. scripts/pt-explorer-com.ps1 Get-PtExplorerWindows , Open-PtExplorerAtPath , Select-PtExplorerFiles , Invoke-PtPeekWithExplorerSelection , Test-PtInteractiveDesktop — drive Explorer via Shell COM to set up multi-file selections, then trigger Peek/FZ/PowerRename/Image Resizer/Workspaces via their hotkeys. Use this for Peek L706-L709, L719-L720 and any test that needs an Explorer file selection. scripts/pt-explorer-contextmenu.ps1 Test-PtDesktopInteractive , Open-PtExplorerContextMenu , Open-PtBackgroundContextMenu , Invoke-PtContextMenuItem , Get-PtContextMenuItems — open Win11's real context menu coordinate-free (COM-select the file → focus it → Shift+F10; coordinate right-click only as fallback), then UIA-invoke a menu item by name. Canonical user-flow path for File Locksmith / Image Resizer / PowerRename / New+ menu-presence + launch tests — prefer selecting a file ( Open-PtExplorerContextMenu -FileName ) over the background menu; open one menu per fresh window; assert menu TYPE + focus, not module presence. Needs an unlocked interactive desktop. See references/explorer-context-menu-flow.md for the full write-up, stability notes, and per-module captions. scripts/pt-shell-verbs.ps1 Get-PtShellVerbs , Invoke-PtShellVerb , Reset-PtShellComCache — enumerate + invoke CLASSIC HKCR shell verbs via Shell.Application COM. NOT for PT context-menu modules on Win11 (PT registers via IExplorerCommand, not classic — use pt-explorer-contextmenu.ps1 for those). Useful for non-PT verbs (Open/Edit/Send-to/third-party) and as a negative check that PT verbs are NOT classic-shadowed. scripts/pt-state.ps1 Get-PtSettings , Get-PtModuleSettings , Get-CmdPalSettings , Get-PtRunnerLogTail , Test-PtModuleEnabled , Test-PtModuleProcess , Restart-PtRunner , Backup-PtModuleSettings , Restore-PtModuleSettings — common state checks. scripts/pt-nonelevated.ps1 Start-PtNonElevated , Invoke-PtNonElevatedCapture — launch an exe at Medium IL (non-elevated) from an elevated agent shell via a one-shot RunLevel Limited scheduled task. Required for elevation-visibility tests (a non-elevated module must NOT see higher-integrity processes; e.g. File Locksmith L649/L650). Verify the result with Test-ProcessElevated . Dot-source them all at once in your bootstrap (the Get-ChildItem loop loads every helper — see Step 1 — Bootstrap ): $skill = '<this skill folder>' # the folder containing SKILL.md, e.g. <PT-repo>\.github\skills\powertoys-verification Get-ChildItem "$skill\scripts" -Filter '*.ps1' | ForEach-Object { . $_.FullName } Step 1 — Bootstrap $module = 'AdvancedPaste' # or 'CmdPal', 'FZ', 'Peek', ... # Work out of %TEMP% during the run (keeps screenshots/scratch off OneDrive); move to the # sign-off archive at the very end (see Step 7). $workspace = "$env:TEMP\verify-$module-$(Get-Date -Format yyyyMMdd-HHmmss)" New-Item -ItemType Directory -Path $workspace, "$workspace\artifacts" -Force | Out-Null $report = "$workspace\verify-$module.md" # Dot-source helpers $skill = '<this skill folder>' # set once at top of your script (the folder containing SKILL.md) Get-ChildItem "$skill\scripts" -Filter '*.ps1' | ForEach-Object { . $_.FullName } # Verify environment "=== Environment ===" | Tee-Object $report -Append "IsAdmin: $(Test-PtAdmin)" | Tee-Object $report -Append $rn = Test-PtRunnerAdmin "PT runner: PID=$($rn.Pid) Elevated=$($rn.Elevated)" | Tee-Object $report -Append # The checklist source depends on the scenario (see references/scenarios/): # A - read the supplied references/release-checklist/<module>.md # B - derive 1-3 items from each PR's `gh pr view/diff`; then drive the installed bits, OR # build+sideload if the code isn't in the build under test (pr-validation.md) # Then iterate the items (see Step 6 - Verifier loop). Step 2 — Drive techniques Every checklist item boils down to ONE of three intents. Pick the bucket from the verb in the item, then use the best technique inside it. Stop at the first technique that works. Intent Verb-cues in the checklist item Bucket Change a setting "default is X", "setting persists", "is enabled/disabled by default", "value Y is accepted" §2.A Interact with a UI element "click X", "toggle X", "type into Y", "X button is visible", "selecting Z does W" §2.B Trigger a module action "pressing hotkey X opens Y", "module launches", "Z happens when invoked" §2.C §2.A — Change a setting (single technique) Edit the JSON file the module reads, wait for the file-watcher debounce, assert, then restore from backup. Zero external tools. $bk = Backup-PtModuleSettings -ModuleDir AdvancedPaste try { $j = Get-PtModuleSettings -ModuleDir AdvancedPaste $j.properties.IsAdvancedAIEnabled.value = $false $j | ConvertTo-Json -Depth 12 | Set-Content "$env:LOCALAPPDATA\Microsoft\PowerToys\AdvancedPaste\settings.json" Start-Sleep -Seconds 4 # debounce — runner re-reads via file-watcher # ... assertion ... } finally { Restore-PtModuleSettings -ModuleDir AdvancedPaste -BackupPath $bk } For shell-extension modules (PowerRename, File Locksmith, Image Resizer, New+) edit the module-owned file under %LOCALAPPDATA%\Microsoft\PowerToys\<Module>\ , then Restart-PtRunner (and on stubborn handlers, restart Explorer). See pitfall #12 below. If you need to flip the enabled bit for a whole module, debounce isn't enough — call Restart-PtRunner after the write. §2.B — Interact with a UI element (2 techniques, most-reliable first) B1. UIA invoke / set-value — always try first winapp ui invoke 'SubmitButton' -a PowerToys.Settings winapp ui set-value 'QueryTextBox' '=2+3*4' -a PowerToys.PowerLauncher Start-Sleep -Milliseconds 600 winapp ui inspect -a PowerToys.PowerLauncher --depth 7 -i 2>$null Invoke goes through UIA InvokePattern COM IPC — no foreground steal, no UIPI. See references/winapp-ui-testing.md §CRITICAL — invoke vs click. B2. PostMessage WM_KEYDOWN/CHAR — when UIA can't reach the target For elevated targets, AppX windows with stunted UIA trees, or keystrokes that UIA set-value can't dispatch (arrow-key ListView nav, Enter to commit). See references/winapp-ui-testing.md §CRITICAL — Keystroke input that bypasses UIPI (PostMessage). Esc is often filtered by WinUI 3 raw-input hook — use BackButton invoke instead. §2.C — Trigger a module action (2 techniques, most-reliable first) C1 Named Event C2 SendInput chord Proves The action fires (the path downstream of the hotkey). Not that the chord is bound. The full path: real keys → runner hook → action. The only method that proves the chord binding itself. Robustness Highest — no foreground, no input desktop, UIPI-immune; works headless / RDP-minimized. Lowest — needs an attached input desktop (else BLK-ENV ), steals foreground, can't inject OS-reserved chords (Win+L / Win+Tab). Precondition Owning module process is running (the event only exists while it is). Attached input desktop + foreground. Pick by what the item asserts: for "does action Y happen" use C1; for "pressing chord X triggers Y" or "the rebind takes effect", C1 is insufficient (it bypasses the chord) — use C2, or C1 plus a runner-log line proving the chord was accepted. C1. Named Event signal — preferred Invoke-PtSharedEvent -Name 'CmdPal.Show' # opens CmdPal without keyboard Invoke-PtSharedEvent -Name 'AOT.Pin' # pins foreground window via AOT Invoke-PtSharedEvent -Name 'PowerLauncher.Invoke' # opens PT Run Invoke-PtSharedEvent -Name 'LightSwitch.Toggle' # toggles theme Get-PtSharedEventCatalog | Format-Table # full list No synthetic input — it's a SetEvent on the kernel event the module waits on, the same downstream path the runner's hotkey handler signals. Verify the side effect via UIA ( winapp ui list-windows -a <module> ), a log line ( Get-PtRunnerLogTail ), or settings.json diff ( Get-PtModuleSettings ). The event only exists while the owning process runs, so Test-PtSharedEvent doubles as an "is the module alive" check. C2. SendInput chord — last resort / chord-binding verification Real synthetic keys. Loud (steals foreground) and fragile, but the only way to prove the activation chord is actually bound. The runner's global keyboard hook catches the chord regardless of focus, so the precondition is just an attached input desktop (pitfall #7; on a detached desktop SendInput returns ACCESS_DENIED and the keys vanish → mark BLK-ENV ). # Precondition: input desktop attached? 0 = detached → don't bother sending, mark BLK-ENV (pitfall #7) if ([PtFg]::GetForegroundWindow() -eq [IntPtr]::Zero) { throw 'No input desktop — BLK-ENV (pitfall #7)' } Send-PtChord -Mods 0x5B,0x10 -Key 0x43 # Win+Shift+C → Color Picker (cb=40 fix is inside the helper) $line = Wait-PtHotkeyAccepted -ModuleHint 'Color' -TimeoutSec 3 if (-not $line) { throw 'Runner did not log hotkey invocation' } Rare fallback — a module that uses its own RegisterHotKey and exposes no Named Event. Post WM_HOTKEY ( 0x312 ) straight to its message window (find the HWND via EnumWindows + GetClassName through Add-Type — same P/Invoke pattern as pt-foreground-guard.ps1 ). No current PT module needs this: ZoomIt — the obvious candidate — also waits on Named Events ( ZoomIt.Zoom , ZoomIt.Draw , …; source: Zoomit.cpp CreateEventW(ZOOMIT_ZOOM_EVENT) ), so drive it with C1. Different case — sending keys into a specific focused window (e.g. a CmdPal alias like = / < / > that winapp ui set-value can't trigger because it bypasses TextChanged; see references/modules/command-palette.md ). Here the keystrokes go to whatever currently has focus, so you must bring the target window foreground first: Assert-PtForegroundOrAbort -AppId Microsoft.CmdPal.UI # -AppId = the window you're typing INTO