CLI v2 ecs execute-command does not support SSO Session profiles

[cupakromer]

Describe the bug

ecs execute-command does not support SSO session based profiles. Attempting to use the command results in either a missing configuration error or a missing cache file error.

Missing Configuration Error

SessionId: ecs-execute-command-0123456789 : ----------ERROR------- Encountered error while initiating handshake. KMSEncryption failed on client with status 2 error: Failed to process action KMSEncryption: error while creating new KMS service, Error creating new aws sdk session profile "my-sso-profile" is configured to use SSO but is missing required configuration: sso_region, sso_start_url

Missing Cache File Error

SessionId: ecs-execute-command-0123456789 : ----------ERROR------- Encountered error while initiating handshake. KMSEncryption failed on client with status 2 error: Failed to process action KMSEncryption: Error calling KMS GenerateDataKey API: SSOProviderInvalidToken: the SSO session has expired or is invalid caused by: open /Users/me/.aws/sso/cache/a00fce5cb007c23a469c136160398db65edcb180.json: no such file or directory

Expected Behavior

Running the aws ecs execute-command should behave like other CLI commands and work with non-legacy SSO profiles.

$ aws ecs execute-command \
  --profile my-sso-profile \
  --cluster my-ecs-cluster \
  --region us-east-1 \
  --interactive \
  --task arn:aws:ecs:us-east-1:redacted:task/my-ecs-cluster/redacted \
  --command 'sh'

The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.


Starting session with SessionId: ecs-execute-command-0123456789
This session is encrypted using AWS KMS.
# 

Current Behavior

Many AWS CLI commands work just fine with the SSO session based profiles:

$ aws s3 ls --profile my-sso-profile
2023-01-27 11:27:11 myfirst-bucket
2022-11-01 11:42:57 legacy-bucket

$ aws ecs list-tasks \
  --desired-status RUNNING \
  --cluster my-ecs-cluster \
  --region us-east-1 \
  --profile my-sso-profile
{
    "taskArns": [
        "arn:aws:ecs:us-east-1: redacted:task/my-ecs-cluster/redacted",
        "arn:aws:ecs:us-east-1: redacted:task/my-ecs-cluster/redacted2",
        "arn:aws:ecs:us-east-1: redacted:task/my-ecs-cluster/redacted3"
    ]
}

Attempting to use the same profile with ecs execute-command fails:

$ aws ecs execute-command \
  --profile my-sso-profile \
  --cluster my-ecs-cluster \
  --region us-east-1 \
  --interactive \                                                                                  
  --task arn:aws:ecs:us-east-1:redacted:task/my-ecs-cluster/redacted \
  --command 'sh'

The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.


Starting session with SessionId: ecs-execute-command-0123456789


SessionId: ecs-execute-command-0123456789 : 
----------ERROR-------
Encountered error while initiating handshake. KMSEncryption failed on client with status 2 error:
Failed to process action KMSEncryption: error while creating new KMS service, Error creating new aws sdk
session profile "my-sso-profile" is configured to use SSO but is missing required configuration:
sso_region, sso_start_url

If the listed missing configuration parameters are added, then the error changes to:

Starting session with SessionId: ecs-execute-command-0123456789


SessionId: ecs-execute-command-0123456789 : 
----------ERROR-------
Encountered error while initiating handshake. KMSEncryption failed on client with status 2 error:
Failed to process action KMSEncryption: Error calling KMS GenerateDataKey API: SSOProviderInvalidToken:
the SSO session has expired or is invalid caused by:
open /Users/me/.aws/sso/cache/a00fce5cb007c23a469c136160398db65edcb180.json: no such file or directory

Switching to a previously configured legacy version of the profile allows ecs execute-command to run successfully:

$ aws ecs execute-command --profile my-legacy-profile ...

The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.


Starting session with SessionId: ecs-execute-command-0123456789
This session is encrypted using AWS KMS.
#

Reproduction Steps

Start with the following configuration:

# ~/.aws/config
[profile my-legacy-profile]
sso_start_url = https://my-sso.awsapps.com/start
sso_region = us-east-1
sso_account_id = 123456789000
sso_role_name = DevAccess
region = us-east-1
output = json
[profile my-sso-profile]
sso_session = my-sso-session
sso_account_id = 123456789000
sso_role_name = DevAccess
region = us-east-1
output = json
[sso-session my-sso-session]
sso_start_url = https://my-sso.awsapps.com/start
sso_region = us-east-1
sso_registration_scopes = sso:account:access

Login to the non-legacy SSO profile. Then using that profile, list the available task ARNs, pick one and run execute-command:

$ aws sso login --profile my-sso-profile

Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request,
open the following URL:

https://device.sso.us-east-1.amazonaws.com/

Then enter the code:

XXXX-XXXX
Successfully logged into Start URL: https://my-sso.awsapps.com/start

$ aws ecs list-tasks \
  --desired-status RUNNING \
  --cluster my-ecs-cluster \
  --region us-east-1 \
  --profile my-sso-profile
{
    "taskArns": [
        "arn:aws:ecs:us-east-1: redacted:task/my-ecs-cluster/redacted",
        "arn:aws:ecs:us-east-1: redacted:task/my-ecs-cluster/redacted2"
    ]
}

$ aws ecs execute-command \
  --profile my-sso-profile \
  --cluster my-ecs-cluster \
  --region us-east-1 \
  --interactive \                                                                                  
  --task arn:aws:ecs:us-east-1:redacted:task/my-ecs-cluster/redacted \
  --command 'sh'

The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.


Starting session with SessionId: ecs-execute-command-0123456789


SessionId: ecs-execute-command-0123456789 : 
----------ERROR-------
Encountered error while initiating handshake. KMSEncryption failed on client with status 2 error:
Failed to process action KMSEncryption: error while creating new KMS service, Error creating new aws sdk
session profile "my-sso-profile" is configured to use SSO but is missing required configuration:
sso_region, sso_start_url

Manually edit the ~/.aws/config file to add the reported missing configuration parameters:

@@ -7,6 +7,8 @@ region = us-east-1
 output = json
 [profile my-sso-profile]
 sso_session = my-sso-session
+sso_start_url = https://my-sso.awsapps.com/start
+sso_region = us-east-1
 sso_account_id = 123456789000
 sso_role_name = DevAccess
 region = us-east-1

Sign out then sign back into the profile and re-run the same ecs execute-command:

$ aws sso logout --profile my-sso-profile

$ aws sso login --profile my-sso-profile

Attempting to automatically open the SSO authorization page in your default browser.
If the browser does not open or you wish to use a different device to authorize this request,
open the following URL:

https://device.sso.us-east-1.amazonaws.com/

Then enter the code:

XXXX-XXXX
Successfully logged into Start URL: https://my-sso.awsapps.com/start

$ aws ecs execute-command \
  --profile my-sso-profile \
  --cluster my-ecs-cluster \
  --region us-east-1 \
  --interactive \                                                                                  
  --task arn:aws:ecs:us-east-1:redacted:task/my-ecs-cluster/redacted \
  --command 'sh'

The Session Manager plugin was installed successfully. Use the AWS CLI to start a session.


Starting session with SessionId: ecs-execute-command-0123456789


SessionId: ecs-execute-command-0123456789 : 
----------ERROR-------
Encountered error while initiating handshake. KMSEncryption failed on client with status 2 error:
Failed to process action KMSEncryption: Error calling KMS GenerateDataKey API: SSOProviderInvalidToken:
the SSO session has expired or is invalid caused by:
open /Users/me/.aws/sso/cache/a00fce5cb007c23a469c136160398db65edcb180.json: no such file or directory

Possible Solution

No response

Additional Information/Context

It appears the cache files generated by the SSO session profile and the legacy profile are using different values to generate the cache key.

$ rm -rf ~/.aws/sso/cache

$ aws sso login --profile my-sso-profile
# ...

$ ls -1 ~/.aws/sso/cache
b755b5ec73400c04400e978208d8559ad1f39053.json
# ...

$ aws sso logout --profile my-sso-profile

$ rm -rf ~/.aws/sso/cache

$ aws sso login --profile my-legacy-profile
# ...

$ ls -1 ~/.aws/sso/cache
a00fce5cb007c23a469c136160398db65edcb180.json
# ...

The SSO session profile generates the cache key from the session name while the legacy profile has the cache key generated from the URL:

Digest::SHA1.hexdigest("my-sso-session")
# => b755b5ec73400c04400e978208d8559ad1f39053

Digest::SHA1.hexdigest("https://my-sso.awsapps.com/start")
# => a00fce5cb007c23a469c136160398db65edcb180

The problematic ecs execute-command appears to ignore the sso_session configuration in the SSO profile and is looking for just sso_url; possibly why it complains about the missing configurations. When both sso_session and sso_url are present, the aws sso login gives higher precedence to the sso_session and uses that for the cache key which appears to be why after adding the "missing configuration" the ecs execute-command still fails, but now with a missing cache error.

CLI version used

aws-cli/2.11.20 Python/3.11.3 Darwin/22.4.0 source/arm64 prompt/off

Environment details (OS name and version, etc.)

macOS 13.3.1

[diegomosela]

I have the same problem.

[tim-finnigan]

Thanks for reaching out. @diegomosela which AWS CLI version are you using? Is this something that was working for you and @cupakromer before updating to a newer version of the AWS CLI? This may just be a limitation in the session manager plugin as reported here: aws/session-manager-plugin#58.

[diegomosela]

Thanks for the answer.

My AWS CLI version is 2.11.22 and the session manager plugin is 1.2.463.0.

I didn't get to use SSO and session-manager-plugin in a previous version, so I can't say at this point if it worked. I will try to downgrade the version to validate.

[cupakromer]

@tim-finnigan I do not see the same issue using the session manager directly. It connects just fine using the SSO profile.

$ session-manager-plugin --version
1.2.463.0


$ aws sso login --profile my-sso-profile

Attempting to automatically open the SSO authorization page...
...


$ aws ssm start-session --profile my-sso-profile --target ecs:my-ecs-cluster_redacted_redacted-123456


Starting session with SessionId: me@company.com-0123456abcde
#