diff --git a/manifest.json b/manifest.json index 463512de..fb6fd63b 100644 --- a/manifest.json +++ b/manifest.json @@ -73,6 +73,10 @@ "title": "Deploy to Linux", "path": "/platform/smallstep-agent.mdx" }, + { + "title": "Troubleshooting the Agent", + "path": "/platform/troubleshooting-agent.mdx" + }, { "title": "Configure Browser Certificates", "path": "/tutorials/browser-certificate-setup-guide.mdx" diff --git a/platform/smallstep-agent.mdx b/platform/smallstep-agent.mdx index c89047b8..80fa7618 100644 --- a/platform/smallstep-agent.mdx +++ b/platform/smallstep-agent.mdx @@ -26,11 +26,6 @@ In this document, we will install, configure, and start the Smallstep Agent on a - runtime state in `/run/step-agent` - configuration in `/etc/step-agent` - certificates in`/var/lib/step-agent` and in your configured locations -- The agent will connect to the following Smallstep hosts: - - Your CA: `.ca.smallstep.com` and subdomains - - Agent API: `control.infra.smallstep.com` - - Smallstep API: `gateway.smallstep.com` - - TPM Attestation CA: `att.smallstep.com` # Quick Install diff --git a/platform/smallstep-app.mdx b/platform/smallstep-app.mdx index cb785bab..a6e8ff0c 100644 --- a/platform/smallstep-app.mdx +++ b/platform/smallstep-app.mdx @@ -1,20 +1,22 @@ --- updated_at: September 17, 2025 -title: The Smallstep App -html_title: Smallstep App User Documentation Guide -description: Complete guide to the Smallstep App for enterprise security workflows. Manage certificates, devices, and identity from a unified interface. +title: The Smallstep Agent +html_title: Smallstep Agent User Documentation Guide +description: Complete guide to the Smallstep Agent for enterprise security workflows. Manage certificates, devices, and identity from a unified interface. --- -Smallstep ensures that access to financial data, code repositories, PII, and other sensitive resources is only possible from trusted devices. +Smallstep ensures that access to financial data, code repositories, PII, and other sensitive resources is only possible from trusted devices. -The Smallstep desktop app offers a uniform experience for device identity across macOS, Windows, and Linux, and is foundational to Smallstep's high-assurance device attestation workflow, automating the enrollment and delivery of client certificates, and configuring the components that depend on them. +The Smallstep Agent offers a uniform experience for device identity across macOS, Windows, and Linux, and is foundational to Smallstep's high-assurance device attestation workflow, automating the enrollment and delivery of client certificates, and configuring the components that depend on them. -The Smallstep app operates differently for Linux. For Linux specific instructions, see [Smallstep Agent for Linux](./smallstep-agent.mdx). +The agent runs as a background service on all platforms. On macOS and Windows, the agent includes an optional desktop app that provides visibility into the agent's status and aids in troubleshooting. + +The Smallstep Agent operates differently for Linux. For Linux specific instructions, see [Smallstep Agent for Linux](./smallstep-agent.mdx). ## Download -The Smallstep App includes the Smallstep Agent, -which runs in the background. +On macOS and Windows, the Smallstep Agent includes an optional desktop app UI for transparency and troubleshooting. +The agent runs as a background service on all platforms. Installers for macOS, Windows and Linux can be also be downloaded from [GitHub releases](https://github.com/smallstep/smallstep-desktop/releases). Releases are signed with, and can be verified, by cosign. @@ -53,20 +55,29 @@ All platforms require an internet connection for normal operation. ### Windows -- *Administrator privileges* - the Smallstep app requires privilege escalation to be able to communicate to the TPM +- *Administrator privileges* - the Smallstep Agent requires privilege escalation to be able to communicate to the TPM ### macOS -- *Location permission* - to enable management of Wifi networks, the Smallstep app needs location permission -- *Keychain access* - the Smallstep app uses the macOS keychain to store both keys and certificates it manages -- *Network Extension entitlement* - the Smallstep app requests the *Network Extension* entitlement so that it can manage VPN connections +- *Location permission* - to enable management of Wifi networks, the Smallstep Agent needs location permission +- *Keychain access* - the Smallstep Agent uses the macOS keychain to store both keys and certificates it manages +- *Network Extension entitlement* - the Smallstep Agent requests the *Network Extension* entitlement so that it can manage VPN connections ### Linux -- *TPM read/write permission* - the Smallstep app communicates to the TPM from user-space using `tpm-tss2`, and the running user must have read/write permissions to the TPM resource manager (typically `/dev/tpmrm0`) +- *TPM read/write permission* - the Smallstep Agent communicates to the TPM from user-space using `tpm-tss2`, and the running user must have read/write permissions to the TPM resource manager (typically `/dev/tpmrm0`) + +## Connectivity Requirements + +The Smallstep Agent connects to the following Smallstep hosts: +- Your CA: `.ca.smallstep.com` and subdomains +- Agent API: `control.infra.smallstep.com` +- Smallstep API: `gateway.smallstep.com` +- TPM Attestation CA: `att.smallstep.com` ## File Access -On all platforms, the Smallstep app creates and manages a directory on the filesystem in a well-known location for management of keys and certificates. However, it does not access any other file on a device except the one it creates. + +On all platforms, the Smallstep Agent creates and manages a directory on the filesystem in a well-known location for management of keys and certificates. However, it does not access any other file on a device except the one it creates. - On macOS: `$HOME/Library/Application Support/Smallstep` - On Windows: `%LOCALAPPDATA%/Smallstep` @@ -74,7 +85,7 @@ On all platforms, the Smallstep app creates and manages a directory on the files ## Telemetry -The Smallstep app collects and reports some data from the host device as part of its normal operation. These are: +The Smallstep Agent collects and reports some data from the host device as part of its normal operation. These are: - Device Identifiers from TPM-enabled platforms - Device/Computer Name diff --git a/platform/troubleshooting-agent.mdx b/platform/troubleshooting-agent.mdx new file mode 100644 index 00000000..77bbb3b7 --- /dev/null +++ b/platform/troubleshooting-agent.mdx @@ -0,0 +1,366 @@ +--- +updated_at: December 05, 2025 +title: Troubleshooting the Smallstep Agent +html_title: Smallstep Agent Troubleshooting Guide +description: Troubleshoot common issues with the Smallstep Agent. Learn how to use the doctor command and resolve connectivity, certificate, and configuration problems. +--- + +This guide helps you diagnose and resolve common issues with the Smallstep Agent. The agent includes built-in diagnostic tools to help identify problems quickly. + +## Prerequisites + +Before troubleshooting, ensure your system meets the requirements: +- Review [System Requirements](./smallstep-app.mdx#system-requirements) +- Check [Runtime Requirements](./smallstep-app.mdx#runtime-requirements) +- Verify [Connectivity Requirements](./smallstep-app.mdx#connectivity-requirements) + +## Using the Doctor Command + +The Smallstep Agent includes a `doctor` command that performs automated health checks and provides diagnostic information. This is the recommended first step when troubleshooting issues. + +### Running the Doctor Command + +**On macOS and Windows:** +```bash +step-agent-plugin doctor +``` + +**On Linux:** +```bash +sudo step-agent-plugin doctor +``` + +The doctor command runs a series of checks and displays results in a table format: + +``` ++------------------------------------------+--------+-------------+ +| Check | Status | Description | ++------------------------------------------+--------+-------------+ +| Check runtime requirements | PASS | | +| Ping the agent | PASS | | +| Check attestation CA connectivity | PASS | | +| Check API gateway connectivity | PASS | | +| Check mission control connectivity | PASS | | +| Bootstrap certificate authorities | PASS | | +| Verify issued certificates | PASS | | +| Sign with endpoint keys | PASS | | +| Enumerate PKCS#11 objects | PASS | | ++------------------------------------------+--------+-------------+ +``` + +### Understanding Doctor Output + +Each check validates a specific aspect of the agent's functionality. A **PASS** status means the check succeeded, while **FAIL** indicates a problem that needs attention. + +#### Check Runtime Requirements + +**What it checks:** Verifies that your system has the necessary components for the agent to function: +- **TPM 2.0 availability** - Ensures the Trusted Platform Module is accessible and can create/delete keys +- **systemd** (Linux only) - Confirms systemd is running for managing network connections + +**Common failure reasons:** +- No TPM 2.0 module present or enabled in BIOS/UEFI +- TPM is locked or inaccessible +- User lacks permissions to access the TPM (Linux) +- systemd is not the init system (Linux) + +**Troubleshooting steps:** +1. Verify TPM 2.0 is enabled in your system BIOS/UEFI settings +2. On Linux, check TPM permissions: `ls -l /dev/tpm*` +3. On Linux, ensure the user has read/write access to `/dev/tpmrm0` +4. On Windows, verify TPM is enabled: Run `tpm.msc` or check Device Manager +5. On Linux, confirm systemd is running: `ps -p 1 -o comm=` + +#### Ping the Agent + +**What it checks:** Verifies the agent background service is running and responding to IPC requests. + +**Common failure reasons:** +- Agent service is not running +- Agent service crashed or failed to start +- IPC socket is inaccessible + +**Troubleshooting steps:** +1. Check if the agent service is running: + - **macOS:** Look for Smallstep in Activity Monitor + - **Windows:** Check Task Manager for the Smallstep Agent process + - **Linux:** Run `sudo systemctl status step-agent` +2. Restart the agent service: + - **macOS/Windows:** Restart from the desktop app or System Settings + - **Linux:** Run `sudo systemctl restart step-agent` +3. Check agent logs for error messages: + - **Linux:** Run `sudo journalctl -u step-agent -n 50` + +#### Check Attestation CA Connectivity + +**What it checks:** Tests network connectivity to `att.smallstep.com` (TPM Attestation CA). + +**Common failure reasons:** +- Firewall blocking outbound HTTPS connections +- Proxy configuration required but not set +- Network connectivity issues +- DNS resolution failure + +**Troubleshooting steps:** +1. Test connectivity manually: `curl -v https://att.smallstep.com` +2. Check firewall rules allow HTTPS to `att.smallstep.com` +3. If behind a proxy, ensure proxy settings are configured +4. Verify DNS resolution: `nslookup att.smallstep.com` + +#### Check API Gateway Connectivity + +**What it checks:** Tests network connectivity to `gateway.smallstep.com` (Smallstep API). + +**Common failure reasons:** +- Same as attestation CA connectivity issues + +**Troubleshooting steps:** +1. Test connectivity manually: `curl -v https://gateway.smallstep.com` +2. Check firewall rules allow HTTPS to `gateway.smallstep.com` +3. Review network and proxy configuration + +#### Check Mission Control Connectivity + +**What it checks:** Tests gRPC connectivity to `control.infra.smallstep.com:443` (Agent API). + +**Common failure reasons:** +- Firewall blocking port 443 for gRPC traffic +- Deep packet inspection interfering with gRPC +- Proxy not configured for gRPC + +**Troubleshooting steps:** +1. Test connectivity manually: `curl -v https://control.infra.smallstep.com` +2. Check firewall rules allow HTTPS/gRPC to `control.infra.smallstep.com` +3. Some corporate firewalls may block gRPC - work with your network team + +#### Bootstrap Certificate Authorities + +**What it checks:** Verifies the agent can retrieve and validate CA root certificates from your Smallstep team's CA. + +**Common failure reasons:** +- Connectivity issues to `.ca.smallstep.com` +- Invalid or expired CA fingerprint in configuration +- Agent not properly registered with your team + +**Troubleshooting steps:** +1. Verify connectivity to your CA: `curl -v https://.ca.smallstep.com` +2. Check the agent configuration file for correct team name and fingerprint: + - **macOS/Windows:** Configuration in app settings + - **Linux:** Check `/etc/step-agent/agent.yaml` +3. Ensure the device is approved in your Smallstep console + +#### Verify Issued Certificates + +**What it checks:** Validates that all endpoint certificates are currently valid (not expired or not yet valid). + +**Common failure reasons:** +- Certificates expired due to agent being offline +- System clock is incorrect +- Certificate renewal failed + +**Troubleshooting steps:** +1. Verify system clock is accurate: `date` +2. If clock was wrong, correct it and restart the agent +3. Check when certificates will expire (look in agent logs or desktop app) +4. Run doctor with `--renew` flag to force certificate renewal (see below) + +#### Sign with Endpoint Keys + +**What it checks:** Tests that the agent can perform cryptographic signing operations with endpoint keys stored in the TPM/Secure Enclave. + +**Common failure reasons:** +- TPM/Secure Enclave access issues +- Keys were deleted or corrupted +- Permission issues accessing keys + +**Troubleshooting steps:** +1. Verify TPM/Secure Enclave is accessible (see runtime requirements above) +2. Check agent logs for key access errors +3. If keys are corrupted, the agent may need to re-register + +#### Enumerate PKCS#11 Objects + +**What it checks:** (Linux/macOS only) Verifies that certificates and keys are accessible via the PKCS#11 interface. + +**Common failure reasons:** +- PKCS#11 server not running +- `P11_KIT_SERVER_ADDRESS` environment variable not set correctly +- p11-kit not installed or configured + +**Troubleshooting steps:** +1. Verify the PKCS#11 socket exists: + ```bash + ls -l $XDG_RUNTIME_DIR/step-agent/step-agent-pkcs11.sock + ``` +2. Check the environment variable: + ```bash + echo $P11_KIT_SERVER_ADDRESS + ``` +3. Test PKCS#11 manually: + ```bash + pkcs11-tool --module /usr/lib/x86_64-linux-gnu/pkcs11/p11-kit-client.so --list-slots + ``` +4. See [PKCS#11 Support](./smallstep-agent.mdx#openssl-and-pkcs11-support) for configuration details + +### Advanced Doctor Options + +#### Force Certificate Renewal + +To test certificate renewal as part of the doctor checks, use the `--renew` flag: + +```bash +step-agent-plugin doctor --renew +``` + +This will attempt to renew all endpoint certificates and report any failures. This is useful when: +- Certificates are close to expiration +- Testing the renewal workflow +- Recovering from a certificate expiration issue + + +The `--renew` flag will trigger certificate renewal for all endpoints, which may briefly interrupt services using those certificates. + + +#### JSON Output + +For scripting or integration with monitoring tools, use the `--json` flag: + +```bash +step-agent-plugin doctor --json +``` + +This outputs check results in JSON format: + +```json +{ + "authorities": { + "Description": "", + "State": "PASS" + }, + "certificates": { + "Description": "", + "State": "PASS" + }, + "network-attestation": { + "Description": "", + "State": "PASS" + }, + "network-control": { + "Description": "", + "State": "PASS" + }, + "network-gateway": { + "Description": "", + "State": "PASS" + }, + "ping": { + "Description": "", + "State": "PASS" + }, + "preflight": { + "Description": "", + "State": "PASS" + }, + "sign": { + "Description": "", + "State": "PASS" + } +} +``` + +## Common Issues and Solutions + +### Agent Won't Start + +**Symptoms:** +- Agent service fails to start +- Desktop app shows "Agent not running" +- Doctor ping check fails + +**Solutions:** +1. Check system requirements are met +2. Run `step-agent-plugin doctor` to identify specific issues +3. Check agent logs for startup errors +4. Restart the system if the TPM is in an inconsistent state +5. On Linux, check for conflicting services using the same ports + +### Network Connectivity Issues + +**Symptoms:** +- All network checks fail in doctor output +- Agent shows "offline" status +- Certificate renewal fails + +**Solutions:** +1. Verify internet connectivity: `ping 8.8.8.8` +2. Test DNS resolution: `nslookup gateway.smallstep.com` +3. Review [Connectivity Requirements](./smallstep-app.mdx#connectivity-requirements) +4. Check corporate firewall and proxy settings +5. Ensure all required Smallstep hosts are allowlisted + +### Certificate Expired or Invalid + +**Symptoms:** +- "Verify issued certificates" check fails +- Applications can't authenticate +- Certificate errors in browser or VPN + +**Solutions:** +1. Check system clock is accurate +2. Run `step-agent-plugin doctor --renew` to force renewal +3. If agent was offline for extended period, certificates may have expired +4. Check agent logs for renewal errors +5. Verify connectivity to your team's CA + +### TPM/Secure Enclave Access Issues + +**Symptoms:** +- Runtime requirements check fails +- "Sign with endpoint keys" check fails +- Agent can't create or access keys + +**Solutions:** +1. **Windows:** + - Run agent with administrator privileges + - Verify TPM is enabled in Device Manager + - Check TPM status: `Get-Tpm` in PowerShell +2. **macOS:** + - Ensure app has keychain access permission + - Check System Settings > Privacy & Security > Keychain +3. **Linux:** + - Check TPM permissions: `ls -l /dev/tpmrm0` + - Add user to tss group: `sudo usermod -a -G tss $USER` + - Verify tpm2-tss is installed + +### PKCS#11 Not Working (Linux/macOS) + +**Symptoms:** +- Applications can't access certificates via PKCS#11 +- Chrome/Firefox don't see Smallstep certificates +- NetworkManager can't use agent certificates + +**Solutions:** +1. Verify PKCS#11 socket exists and is accessible +2. Set environment variable correctly: + ```bash + export P11_KIT_SERVER_ADDRESS=unix:path=$XDG_RUNTIME_DIR/step-agent/step-agent-pkcs11.sock + ``` +3. Install p11-kit if not present +4. Test with `pkcs11-tool --list-slots` +5. See [PKCS#11 configuration guide](./smallstep-agent.mdx#openssl-and-pkcs11-support) + +## Getting Help + +If you've tried the troubleshooting steps above and still have issues: + +1. **Run doctor command** and save the output +2. **Collect agent logs:** + - **Linux:** `sudo journalctl -u step-agent -n 200` + - **macOS/Windows:** Export logs from the desktop app +3. **Note your configuration:** + - Operating system and version + - Smallstep Agent version (`step-agent-plugin version`) + - Any error messages or failure descriptions +4. **Contact Smallstep support** with the information above + +For security issues or private questions, email support@smallstep.com.