1. Resources
  2. Detection rules
  3. Commands Run on EC2 Instances Through Systems Manager

Commands Run on EC2 Instances Through Systems Manager detection rules

An attacker who controls an identity with Systems Manager permissions runs scripts or opens shell sessions on managed instances, with root or SYSTEM rights, and reaches the instance roles and data without any network access to the hosts.

Platform
AWS
Severity
High
Log source
AWS CloudTrail
Last updated
MITRE ATT&CK technique
T1651Cloud Administration Command
Tactic
Execution
Rule formats
Sigmaand queries for: SplunkMicrosoft SentinelIBM QRadar
What an attacker needs
Credentials for a principal that may call SendCommand or StartSession on instances that are managed by Systems Manager.
Impact
Code execution as root or SYSTEM on the targeted instances, which also exposes the credentials of their instance roles and their data.
Response
Cancel the command and terminate the sessions, contain the calling principal, and treat every targeted instance as compromised.
Mitigation
Grant SendCommand and StartSession only to operators, limited by instance tags, and log session activity to S3 or CloudWatch Logs.
CloudTrail events
SendCommandStartSession
Threat groups
APT29VOID MANTICORE

How the attack works

AWS Systems Manager manages instances through an agent that is installed on them and that connects out to the service. Two API calls turn an IAM permission into code execution on a host. SendCommand (Run Command) runs a Systems Manager document on one instance, a list of instances or everything that matches a tag, and with the AWS documents AWS-RunShellScript and AWS-RunPowerShellScript the caller supplies the script. StartSession (Session Manager) opens an interactive shell, or a port forwarding tunnel, to a managed node. Neither needs an open inbound port, an SSH key or a password on the host.

AWS is explicit about what this means. The agent runs with root permissions on Linux and SYSTEM permissions on Windows Server, and AWS states that any principal that has been granted permission to send commands to the agent has root or SYSTEM permissions, and that a user who may run any of the predefined documents has administrator access to the node. An attacker who has taken over such a principal therefore owns the instances it can reach: local data, credentials on disk, and the temporary credentials of the instance role from the instance metadata. With a tag-based target, one call reaches a whole fleet.

Systems Manager logs all control plane operations as CloudTrail management events under ssm.amazonaws.com, and AWS names SendCommand and StartSession among them. The events show who started a command or a session and on what. What was typed inside a session is not in CloudTrail: it is only recorded when session logging to Amazon S3 or CloudWatch Logs is configured. AWS publishes no example record for the two calls, so the rules that read a request field say that the field follows the API reference. No incident report was used for this page.

What an attacker needs

  • Credentials for a principal allowed ssm:SendCommand or ssm:StartSession on the target instances.
  • Instances that are managed nodes, which means that the agent is running and the instance has a role that lets it register with Systems Manager.

Impact

Code execution as root or SYSTEM on every instance the principal can target. From there the attacker reads application secrets and data, takes the temporary credentials of the instance role and acts with its permissions in the account, and moves to whatever the hosts can reach on the network. A port forwarding session gives a tunnel into the VPC.

Detection rules

Systems Manager Command Sent or Session Started

Detects SendCommand and StartSession calls to AWS Systems Manager. The first runs a Systems Manager document on managed nodes, the second opens an interactive shell or a port forwarding tunnel to one node. AWS states that the agent on the node runs with root or SYSTEM permissions and that any principal allowed to send commands to it has those permissions, and it names both calls among the control plane operations that are logged as management events. The rule matches the event names only and covers every document, including custom ones. It is a broad rule: scope it to callers outside your operations and automation roles, or use it as a hunting query. It also matches attempts that failed (errorCode present): check errorCode, because an unauthorized attempt is worth seeing.

LevelLowStatusExperimentalLog sourceAWS CloudTrail
title: Systems Manager Command Sent or Session Started
id: e4f8de3c-f772-47ce-8735-2e3ffe70e399
status: experimental
description: >-
  Detects SendCommand and StartSession calls to AWS Systems Manager. The
  first runs a Systems Manager document on managed nodes, the second opens
  an interactive shell or a port forwarding tunnel to one node. AWS states
  that the agent on the node runs with root or SYSTEM permissions and that
  any principal allowed to send commands to it has those permissions, and
  it names both calls among the control plane operations that are logged as
  management events. The rule matches the event names only and covers every
  document, including custom ones. It is a broad rule: scope it to callers
  outside your operations and automation roles, or use it as a hunting
  query.
  It also matches attempts that failed (errorCode present): check errorCode,
  because an unauthorized attempt is worth seeing.
references:
  - https://docs.aws.amazon.com/systems-manager/latest/userguide/monitoring-cloudtrail-logs.html
  - https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-restrict-root-level-commands.html
  - https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_StartSession.html
author: Forestall
date: 2026-11-18
tags:
  - attack.execution
  - attack.t1651
logsource:
  product: aws
  service: cloudtrail
detection:
  selection:
    eventSource: ssm.amazonaws.com
    eventName:
      - SendCommand
      - StartSession
  condition: selection
falsepositives:
  - Patch runs, maintenance windows, configuration management and operators who use Session Manager for shell access. Useful only after the automation and operations roles are excluded with a filter_* selection on userIdentity.arn.
level: low

False positives

  • Patch runs, maintenance windows, configuration management and operators who use Session Manager for shell access. Useful only after the automation and operations roles are excluded with a filter_* selection on userIdentity.arn.

Session Manager Sessions Started on Many Instances by One Principal

Counts, per calling principal (userIdentity.arn), on how many different targets it started a Session Manager session. Sessions on five or more different managed nodes within 30 minutes is the pattern of someone walking through a fleet, for example to collect the credentials of each instance role. An operator who works on an incident does the same, so the count and the window are starting points to tune. The target is read from requestParameters.target, which follows the Target request parameter of the StartSession API reference: AWS publishes no example record for this event, so confirm the field in your own events. Denied attempts are counted as well. For role sessions the principal is the session ARN, so one role used with different session names is counted per session.

Fires when one userIdentity.arn has at least 5 different requestParameters.target values within 30 minutes.

Threshold ruleLevelMediumStatusExperimentalLog sourceAWS CloudTrail
title: Session Manager Session Started (Many Targets Base Event)
id: 4e736f6e-178b-4dfe-835a-7f3112c983ef
name: aws_ssm_session_started
status: experimental
description: >-
  Base event of the threshold rule in this file. Any StartSession call to
  AWS Systems Manager, including attempts that were denied.
logsource:
  product: aws
  service: cloudtrail
detection:
  selection:
    eventSource: ssm.amazonaws.com
    eventName: StartSession
  condition: selection
level: low
---
title: Session Manager Sessions Started on Many Instances by One Principal
id: c78676ef-a12f-4b16-8ad6-9595ebfa8910
status: experimental
description: >-
  Counts, per calling principal (userIdentity.arn), on how many different
  targets it started a Session Manager session. Sessions on five or more
  different managed nodes within 30 minutes is the pattern of someone
  walking through a fleet, for example to collect the credentials of each
  instance role. An operator who works on an incident does the same, so
  the count and the window are starting points to tune. The target is read
  from requestParameters.target, which follows the Target request parameter
  of the StartSession API reference: AWS publishes no example record for
  this event, so confirm the field in your own events. Denied attempts are
  counted as well. For role sessions the principal is the session ARN, so
  one role used with different session names is counted per session.
references:
  - https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_StartSession.html
  - https://docs.aws.amazon.com/systems-manager/latest/userguide/monitoring-cloudtrail-logs.html
  - https://attack.mitre.org/techniques/T1651/
author: Forestall
date: 2026-11-18
tags:
  - attack.execution
  - attack.t1651
correlation:
  type: value_count
  rules:
    - aws_ssm_session_started
  group-by:
    - userIdentity.arn
  timespan: 30m
  condition:
    field: requestParameters.target
    gte: 5
falsepositives:
  - Operators who check many hosts during an incident or a rollout, and tools that open sessions to instances in turn. Exclude those roles or give them a higher count.
  - The values of 5 targets and 30 minutes are starting points, not tested limits. Measure what your own operators produce and adjust both.
level: medium

False positives

  • Operators who check many hosts during an incident or a rollout, and tools that open sessions to instances in turn. Exclude those roles or give them a higher count.
  • The values of 5 targets and 30 minutes are starting points, not tested limits. Measure what your own operators produce and adjust both.

Shell or PowerShell Script Sent to Instances with Systems Manager Run Command

Detects SendCommand calls to AWS Systems Manager that run the AWS documents AWS-RunShellScript or AWS-RunPowerShellScript. With these two documents the caller supplies the script, and AWS states that it runs on the managed node as if the caller were logged on locally, with administrative permissions. AWS names SendCommand among the Systems Manager calls that are logged as management events. The document is read from requestParameters.documentName, which follows the DocumentName request parameter of the Systems Manager API reference: AWS publishes no example record for this event, so confirm the field in your own events. Read the targets of the request (instance IDs or tag-based targets) to see how many hosts were reached. It also matches attempts that failed (errorCode present): check errorCode, because an unauthorized attempt is worth seeing.

LevelMediumStatusExperimentalLog sourceAWS CloudTrail
title: Shell or PowerShell Script Sent to Instances with Systems Manager Run Command
id: e37e60a3-b19a-48c6-ba4b-619b3d4df808
status: experimental
description: >-
  Detects SendCommand calls to AWS Systems Manager that run the AWS
  documents AWS-RunShellScript or AWS-RunPowerShellScript. With these two
  documents the caller supplies the script, and AWS states that it runs on
  the managed node as if the caller were logged on locally, with
  administrative permissions. AWS names SendCommand among the Systems Manager
  calls that are logged as management events. The document is read from
  requestParameters.documentName, which follows the DocumentName request
  parameter of the Systems Manager API reference: AWS publishes no example
  record for this event, so confirm the field in your own events. Read the
  targets of the request (instance IDs or tag-based targets) to see how many
  hosts were reached.
  It also matches attempts that failed (errorCode present): check errorCode,
  because an unauthorized attempt is worth seeing.
references:
  - https://docs.aws.amazon.com/systems-manager/latest/userguide/monitoring-cloudtrail-logs.html
  - https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_SendCommand.html
  - https://docs.aws.amazon.com/systems-manager/latest/userguide/walkthrough-cli.html
  - https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent-restrict-root-level-commands.html
  - https://attack.mitre.org/techniques/T1651/
author: Forestall
date: 2026-11-18
tags:
  - attack.execution
  - attack.t1651
logsource:
  product: aws
  service: cloudtrail
detection:
  selection:
    eventSource: ssm.amazonaws.com
    eventName: SendCommand
    requestParameters.documentName:
      - AWS-RunShellScript
      - AWS-RunPowerShellScript
  condition: selection
falsepositives:
  - Operators, configuration tools and maintenance windows that run scripts on instances. They use the same roles every time, so add those roles to a filter_* selection on userIdentity.arn and keep alerting on scripts sent by people outside the operations team, by workload roles or by unknown roles.
level: medium

False positives

  • Operators, configuration tools and maintenance windows that run scripts on instances. They use the same roles every time, so add those roles to a filter_* selection on userIdentity.arn and keep alerting on scripts sent by people outside the operations team, by workload roles or by unknown roles.

Detection logic

Patterns that need more than one event, a baseline or knowledge of your own environment. Each is a hypothesis to build and tune in your SIEM, not a ready rule.

Command or session on an instance followed by use of its role from elsewhere

Detection hypothesisSequence of eventsNeeds your own lists

The usual goal of a command on an instance is the instance role. Credentials taken from the instance metadata are then used from the attacker's own machine, so calls by the role's session arrive from an address that is not the instance.

  1. A SendCommand or StartSession event from ssm.amazonaws.com. Note the caller in userIdentity.arn and the target instance.
  2. Within the next hours, CloudTrail events whose userIdentity.arn is an assumed-role session of the instance role of that instance.
  3. Compare sourceIPAddress of those events with the addresses of the instance and of your NAT gateways, and alert when it is none of them.

EventsSendCommandStartSession

LimitsIt needs the instance role of each instance and your own address ranges. Calls that the attacker makes from the instance itself come from the expected address and are not found. The time window is a starting point.

First command or session by a principal that has never operated instances

Detection hypothesisNeeds a baseline

Operators and automation roles use Systems Manager daily. A workload role, an application user or a newly created identity that sends its first command or starts its first session is using a permission it was never meant to exercise.

  1. Keep a record, over at least 30 days, of the principals (userIdentity.arn, or the role in userIdentity.sessionContext.sessionIssuer.arn) that called SendCommand or StartSession.
  2. Alert when a principal that is not in the record makes either call.

EventsSendCommandStartSession

LimitsNew operators and new automation produce first-time events too. The 30 days are a starting point.

Triage

  • Identify the caller (userIdentity.arn), the source IP address and the user agent. Is it an operator, a maintenance window or an automation role that normally manages these instances?
  • Read the document name and the targets of the request. A shell script document sent to many instances, or to a tag that covers a fleet, by a principal that has not done this before is the pattern to look at.
  • For a session, read the target instance and check whether the caller is on the team that owns it, and whether the session used port forwarding.
  • Read the command invocations in Systems Manager and the session logs, where they are kept, for what was actually run.
  • Check what the instance role of the target did in CloudTrail after the event, and from which source address.

Response

Once the attack is confirmed:

  1. Cancel the running command and terminate the open sessions in Systems Manager.
  2. Contain the principal that sent the command or started the session, and take away its Systems Manager permissions until its use is explained.
  3. Treat every targeted instance as accessed with root or SYSTEM rights. Isolate it, keep its disks for review, and rebuild it from a known image.
  4. Revoke the active sessions of the instance roles of the targeted instances, and rotate the secrets stored on or readable from those hosts.
  5. Review CloudTrail for what the instance roles did after the event, and remove what was created.

Mitigation

  • Grant ssm:SendCommand and ssm:StartSession only to the principals that operate the instances, and limit them to instances with specific tags, as AWS recommends for these two actions.
  • Limit which documents a principal may run. Do not allow the predefined AWS- documents to everyone, because AWS states that they give administrator access to the node.
  • Configure Session Manager to log session activity to Amazon S3 or CloudWatch Logs, and send Run Command output there, so that what was executed can be reviewed.
  • Keep instance roles least-privilege, because every command on the host can use the role.
  • Remove Systems Manager permissions from workload roles and CI roles that do not operate instances.

Frequently asked questions

Does CloudTrail show which commands were run?

CloudTrail records that `SendCommand` or `StartSession` was called, by whom and against which target. It does not record what was typed in a session. Session content is only kept when Session Manager is configured to log to Amazon S3 or CloudWatch Logs. For Run Command, the status and output of a command can be read back through Systems Manager itself, for example with `ListCommandInvocations`.

Why is permission to send a command the same as being an administrator on the host?

AWS states that the agent runs with root permissions on Linux and SYSTEM permissions on Windows Server, and that any principal that may send commands to the agent has those permissions. AWS also states that a user who may run any predefined document whose name begins with `AWS-` has administrator access to the node.

Which legitimate activity looks the same?

Patch runs, maintenance windows, configuration tools and operators who use Session Manager instead of SSH all produce these events every day. The rules are useful after the automation roles are excluded. What remains is commands and sessions started by people, by workload roles or by principals that have never operated instances.

Has this been used in attacks?

AWS lists Systems Manager command execution in its own threat technique catalog under cloud administration command, with `SendCommand` and `StartSession` as the events to watch. No public incident report was used for this page. It follows what AWS documents about the agent and these two calls.

References

We respect your privacy

We use cookies to keep this site secure and working properly. With your permission, we also use optional cookies to understand usage and improve the experience. Cookie Policy

You can change your choice at any time.