Deployment test checklist

Use this checklist to validate the Windows, macOS, or Linux enterprise installer on representative workstations before expanding a managed deployment. It covers clean installation, per-device...

Use this checklist to validate the Windows, macOS, or Linux enterprise installer on representative workstations before expanding a managed deployment. It covers clean installation, per-device enrollment, runtime health, live policy evidence, protection-state classification, and rollback.

Select the current package from its artifact feed. For Windows, verify the signed IntuneWin metadata. For macOS, verify the PKG signature, notarization, stapling, and checksum. For Linux, verify the RPM and repository signatures, detached checksum signature, SBOM, provenance, and packaged Gateway metadata. Keep this evidence with the deployment change record.

Test scenarios

Story	Acceptance outcome
Windows clean-state install	A tester can remove prior AgentKeeper runtime state, install from the Windows artifact, enroll with an MDM token, and see fresh Workstations health.
macOS clean-state install	A tester can remove prior runtime state, deploy the device profile with no user mapping, install the approved signed PKG, stage the token with the provider's root After-install script before login, and see fresh Workstations health.
macOS first login and user switch	Machine enrollment completes without a signed-in user; after each regular user signs in or becomes the active console user, the runtime reconciles that account without scanning `/Users`.
Linux clean-state install	A tester can verify and install the signed RPM on a supported RHEL/Rocky host, enroll explicit users with an Enterprise Installer token over standard input, and see Workstations plus AI Discovery evidence.
Per-device credential exchange	A deployment-scoped enrollment token is used only for enrollment, and the installed runtime reports a valid per-device credential.
Runtime heartbeat visibility	Workstations and Setup show runtime mode, install source, version, hook matrix, credential state, policy fetch, self-test, live event, repair, and restart-required state where available.
Honest protection state	Self-test alone does not produce Full Integrity. Full Integrity requires a recent live pre-execution event from a supported hook or gateway path.
Rollback and cleanup	The tester can uninstall or roll back without replacing unrelated customer hooks or agent configuration.

Before testing

For each machine:

Log in to the validation org.
Confirm /api/version reports the expected image.
Create a fresh scoped Enterprise Installer token for that machine or cohort; use the matching MDM provider or Native Linux RPM scope.
Keep Workstations and Activity open in browser tabs.
Record any existing workstation state before cleanup.
Close Claude Code, Codex, Cursor, Windsurf, Copilot, Gemini, Claude Desktop, and other selected target apps before install.

Do not paste raw enrollment tokens, fleet API keys, Microsoft Graph credentials, support bundles, or unredacted runtime logs into reports, tickets, or chat.

Test matrix

Area	Windows evidence	macOS evidence	Linux evidence
Artifact	ZIP URL, expected SHA256, actual SHA256, package metadata, signature state	PKG URL, expected SHA256, actual SHA256, `version.json`, `pkgutil --check-signature`, `spctl`, and stapler result	Exact RPM/NEVRA, `rpm --checksig --verbose`, signed checksums/repository metadata, SBOM, and provenance
Cleanup	Backup path, service removal result, removed/moved AgentKeeper paths	Backup path, LaunchDaemon bootout result, removed/moved AgentKeeper paths	`prepare-image --json`, uninstall/purge output, and preserved customer configuration
Install	`install.ps1` output, service install output, signed heartbeat output	Release-scope metadata, `installer` output, postinstall log	`dnf` output, RPM scriptlets, owners/modes, SELinux state, system/user units, and journal
Local health	`Get-Service AgentKeeperRuntime`, `status --json`, `doctor`, `detect.ps1`, `State\last-install.json`	`launchctl print`, `status --json`, `doctor`, detect script, `State/last-install.json`, targeting mode, and active console user	`systemctl`, `rpm -V`, `status --json`, `doctor --json`, reconcile/repair, and nine-surface state
Dashboard health	Workstation id, protection state, runtime mode, install source, version, credential state, policy fetch health	Workstation id, protection state, runtime mode, install source, version, credential state, policy fetch health	Unique workstation id, Native Linux RPM provenance, target user, distribution, package/service state, and AI Discovery parity
Live proof	Last command preflight, last HTTP evaluate, last live event, Activity event id	Last command preflight, last HTTP evaluate, last live event, Activity event id	Real Claude Code allow/deny events and real routed MCP traffic; fixture/self-test evidence stays non-active
Rollback	Uninstall output and preserved non-AgentKeeper hook config	Uninstall output and preserved non-AgentKeeper hook config	Signed downgrade/uninstall output, service recovery, and preserved customer-owned config

Expected states

Moment	Expected result
After cleanup	Local AgentKeeper runtime files and service/LaunchDaemon are absent or backed up. Old server-side workstation records may still exist.
After install	Package/service is present. This proves installation only. After profile delivery and token staging, enrollment should issue the per-device credential and the Workstation should appear or refresh.
After macOS pre-login enrollment	`enrollment_state=enrolled`, `credential_state=valid`, and `targeting_mode=active_console_user` can coexist with no `active_console_user` and `maintenance_state=waiting_for_user`.
After macOS sign-in or user switch	The next one-minute maintenance tick resolves only the active regular console account through Directory Services and reconciles its selected integrations. No per-device username mapping or `/Users` scan is used.
After self-test	Last self-test timestamp updates. This proves reachability only, not live protection.
Before app restart	Standby, Basic Coverage, Mixed, Degraded, or Unknown can be valid depending on fresh Runtime evidence and explicit failures. `restart_required` alone does not make the presentation state Degraded.
After target app restart and harmless action	Last live event or command preflight updates. Full Integrity can be valid only if the event is from a supported pre-execution path.
After controlled block test	BLOCK appears only for a supported pre-execution hook or gateway path. After-only telemetry must not claim blocking.

Severity rubric

Severity	Use when
P0	Install cannot complete on a supported OS, the service/LaunchDaemon cannot start, or the installer destroys unrelated customer configuration.
P1	Enrollment fails, no workstation appears, credential state is invalid, policy fetch is broken, or uninstall cannot remove the runtime.
P2	Workstations shows the wrong protection state, self-test is confused with live protection, restart-required is missing, or health fields are stale/inconsistent.
P3	Copy, docs, CLI output, browser instructions, or QA evidence is confusing but the install path works.

Windows report

Windows enterprise install QA report

Machine:
- Hostname:
- Windows version:
- Architecture:
- Tester:
- Date/time:
- Image tag:

User story result:
- Dirty-machine cleanup: PASS/FAIL
- Artifact checksum verified: PASS/FAIL
- Runtime install completed: PASS/FAIL
- Per-device credential valid: PASS/FAIL
- Runtime heartbeat visible: PASS/FAIL
- Self-test recorded as reachability only: PASS/FAIL
- Live pre-execution event recorded after app restart: PASS/FAIL/NOT TESTED
- Protection state classification correct: PASS/FAIL
- Rollback/uninstall preserves unrelated config: PASS/FAIL/NOT TESTED

Artifact:
- URL:
- Expected SHA256:
- Actual SHA256:
- Signature state:
- Package metadata version:
- Backup path:

Local evidence:
- Get-Service AgentKeeperRuntime:
- status --json redacted:
- doctor redacted:
- self-test --json redacted:
- detect.ps1 output:
- last-install marker redacted:

Dashboard evidence:
- Workstation id:
- Protection state:
- Runtime mode:
- Install source:
- Package/runtime version:
- Hook matrix:
- Credential state:
- Policy fetch health:
- Last command preflight:
- Last HTTP evaluate:
- Last self-test:
- Last live event:
- Last repair result:
- Restart required:

Defects:
- Severity:
- Expected:
- Actual:
- Reproduction:
- Screenshot/log references:

macOS report

macOS enterprise install QA report

Machine:
- Hostname:
- macOS version:
- Architecture:
- Tester:
- Date/time:
- Image tag:

User story result:
- Dirty-machine cleanup: PASS/FAIL
- Artifact checksum verified: PASS/FAIL
- Approved PKG downloaded from release feed: PASS/FAIL
- Token provider and deployment ID match the MDM assignment: PASS/FAIL
- Device profile contains no username or per-user variable: PASS/FAIL
- Enrollment completed before login: PASS/FAIL/NOT TESTED
- First regular user reconciled automatically after sign-in: PASS/FAIL
- Newly active regular user reconciled automatically after user switch: PASS/FAIL/NOT TESTED
- Runtime install completed: PASS/FAIL
- Per-device credential valid: PASS/FAIL
- Runtime heartbeat visible: PASS/FAIL
- Self-test recorded as reachability only: PASS/FAIL
- Live pre-execution event recorded after app restart: PASS/FAIL/NOT TESTED
- Protection state classification correct: PASS/FAIL
- Rollback/uninstall preserves unrelated config: PASS/FAIL/NOT TESTED

Artifact:
- URL:
- Expected SHA256:
- Actual SHA256:
- pkgutil signature result:
- version.json signed/notarized/stapled values:
- Package metadata version:
- Backup path:

Local evidence:
- launchctl print summary:
- status --json redacted:
- targeting mode:
- active console user after sign-in:
- doctor redacted:
- heartbeat --json redacted:
- detect script output:
- last-install marker redacted:

Dashboard evidence:
- Workstation id:
- Protection state:
- Runtime mode:
- Install source:
- Package/runtime version:
- Hook matrix:
- Credential state:
- Policy fetch health:
- Last command preflight:
- Last HTTP evaluate:
- Last self-test:
- Last live event:
- Last repair result:
- Restart required:

Defects:
- Severity:
- Expected:
- Actual:
- Reproduction:
- Screenshot/log references:

Linux report

Linux enterprise install QA report

Machine:
- Hostname:
- Distribution/version:
- Architecture:
- Deployment mode:
- Target user and UID:
- SELinux/FIPS posture:
- Proxy/custom CA:
- Tester and date/time:

Artifact:
- Feed phase:
- RPM filename and NEVRA:
- Expected/actual SHA256:
- RPM signature and repository-signature result:
- Source SHA, Gateway version/SHA, and catalog version:
- SBOM/provenance references:

Local evidence:
- rpm -V:
- systemctl status:
- systemd-analyze security:
- status --json redacted:
- doctor --json redacted:
- reconcile/repair output:
- credential owner/mode:
- recent SELinux AVC and journal redaction result:

Universal coverage:
- Claude Code real allow/deny event ids:
- MCP Gateway real routed event id:
- Each installed secondary surface presence/config/runtime/limitation:
- Each absent surface reports not_seen:
- Cowork reports unsupported/discovery-only on RHEL/Rocky:

Dashboard evidence:
- Workstation id and Native Linux RPM provenance:
- AI Discovery target-user row and catalog version:
- Package/service/reconciliation state:
- No duplicate workstation after re-enrollment/rebuild:

Lifecycle:
- prepare-image clean/contamination/clean result:
- reinstall/upgrade/downgrade result:
- uninstall retains enrollment state:
- purge removes only AgentKeeper-owned state:
- rollback restores service and owned hooks:

Defects:
- Severity, expected, actual, reproduction, and evidence:

Pass criteria

An OS deployment is ready for its assigned device group when every applicable item shows:

Clean install from dirty machine succeeds.
Checksum verification succeeds.
Runtime service or LaunchDaemon starts reliably.
Root/admin-only install marker exists, matches the pinned package/provider, and contains no enrollment token or fleet API key.
Enrollment token exchange produces a valid per-device credential.
Workstations shows fresh runtime heartbeat and health fields.
Self-test appears as reachability proof only.
A real live event appears after target app restart.
Full Integrity appears only when live pre-execution proof exists.
Rollback or uninstall preserves unrelated customer-owned hooks and config.
Windows metadata reports signed: true and final_intunewin_ready: true; WDAC customers have accepted the required trust rule.
macOS version.json reports signed: true, notarized: true, stapled: true, and final_pkg_ready: true; the PKG passes pkgutil --check-signature, spctl -a -vv -t install, and xcrun stapler validate; the device profile contains no username; pre-login enrollment and automatic active-console-user reconciliation pass; and the assigned device group passes install, live-event, and rollback checks.
Linux RPM (Beta) metadata matches the installed NEVRA and verifies the RPM, repository, checksum signature, packaged Gateway, SBOM, and provenance; the canary also proves upgrade/rollback, proxy/custom-CA/SELinux behavior, AI Discovery, a controlled Claude Code policy decision, and routed MCP evidence.
All P0/P1 defects are closed or explicitly accepted before broad assignment.

Use Windows enterprise install, macOS enterprise install, Linux RPM enterprise install, Validate deployment, and Troubleshooting for the companion procedures.

Was this page helpful?