KMS
56 tests across 7 clients. Of the 385 results that measure Overcast, 357 pass (92.7%).
- JS 51/55 passing in AWS SDK for JavaScript v3
- Python 51/55 passing in AWS SDK for Python (boto3)
- Go 51/55 passing in AWS SDK for Go v2
- CLI 51/55 passing in AWS CLI v2
- Java 51/55 passing in AWS SDK for Java 2.x
- .NET 51/55 passing in AWS SDK for .NET
- Rust 51/55 passing in AWS SDK for Rust
56 tests
kms-keys
| Test | JS | Python | Go | CLI | Java | .NET | Rust |
|---|---|---|---|---|---|---|---|
| CreateKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| DescribeKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| PutKeyPolicy | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| CreateKmsAlias | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ListKmsAliases | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ListKeys | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| DisableKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| EnableKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ScheduleKeyDeletion | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| CancelKeyDeletion | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| TagKMSResource TagResource | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ListKMSResourceTags ListResourceTags | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| UntagKMSResource UntagResource | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
kms-crypto
| Test | JS | Python | Go | CLI | Java | .NET | Rust |
|---|---|---|---|---|---|---|---|
| Encrypt | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| Decrypt | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| GenerateDataKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| GenerateDataKeyWithoutPlaintext | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
kms-asymmetric
| Test | JS | Python | Go | CLI | Java | .NET | Rust |
|---|---|---|---|---|---|---|---|
| Sign | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| Verify | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
kms-gen-key generated from the AWS model soaking
| Test | JS | Python | Go | CLI | Java | .NET | Rust |
|---|---|---|---|---|---|---|---|
| CreateKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| DescribeKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| UpdateKeyDescription | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| TagResource | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ListResourceTags | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| UntagResource | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| DescribeKeyMetadata DescribeKey | AWS SDK for JavaScript v3: New test, still soaking | AWS SDK for Python (boto3): New test, still soaking | AWS SDK for Go v2: New test, still soaking | AWS CLI v2: New test, still soaking | AWS SDK for Java 2.x: New test, still soaking | AWS SDK for .NET: New test, still soaking | AWS SDK for Rust: New test, still soaking |
Why 7 results do not passA test generated from the AWS model that has not yet run cleanly enough times to count.
Message ScenarioFailure: kms-gen-key/DescribeKeyMetadata: DescribeKey params={"KeyId":"4a6627e5-64ef-4a90-8ff8-806734b6bd0b"} — responseField at $.KeyMetadata.AWSAccountId: expected a string matching /^[0-9]{12}$/, actual <missing> (compat/model/scenarios/kms.json assert[0])
A test generated from the AWS model that has not yet run cleanly enough times to count.
Message kms-gen-key/DescribeKeyMetadata: op=DescribeKey params={"KeyId": "7f116185-3f28-436e-953e-af7a3409e4a1"} assertion=responseField path=$.KeyMetadata.AWSAccountId expected=matches /^[0-9]{12}$/ actual=<missing> at compat/model/scenarios/kms.json assert[0]
AWS SDK for Go v2, AWS CLI v2, AWS SDK for Java 2.x, AWS SDK for .NET, AWS SDK for Rust: New test, still soaking Go, CLI, Java, .NET, Rust
Report this
A test generated from the AWS model that has not yet run cleanly enough times to count.
Messages (5)AWS SDK for Go v2 kms-gen-key/DescribeKeyMetadata: DescribeKey params {"KeyId":"06085a3e-5d5e-447f-b933-d4c9e23eb7b3"}: responseField matches at $.KeyMetadata.AWSAccountId: expected a string matching "^[0-9]{12}$", actual <missing> (compat/model/scenarios/kms.json assert[0])
AWS CLI v2 kms-gen-key/DescribeKeyMetadata: DescribeKey params {"KeyId":"a7328a75-ae31-433f-baeb-218d31f5121c"}: responseField matches at $.KeyMetadata.AWSAccountId: expected a string matching "^[0-9]{12}$", actual <missing> (compat/model/scenarios/kms.json assert[0])
AWS SDK for Java 2.x kms-gen-key/DescribeKeyMetadata: DescribeKey params {"KeyId":"6650274a-6356-464e-b414-dbd2bb4cd69e"}: responseField matches at $.KeyMetadata.AWSAccountId: expected a string matching "^[0-9]{12}$", actual <missing> (compat/model/scenarios/kms.json assert[0])
AWS SDK for .NET kms-gen-key/DescribeKeyMetadata: DescribeKey params {"KeyId":"d1de3456-b33d-41d0-aded-dbeb31b69b89"}: responseField matches at $.KeyMetadata.AWSAccountId: expected a string matching "^[0-9]{12}$", actual <missing> (compat/model/scenarios/kms.json assert[0])
AWS SDK for Rust kms-gen-key/DescribeKeyMetadata: DescribeKey params {"KeyId":"a8d7744f-8394-4b07-8f56-088c822ba6da"}: responseField matches at $.KeyMetadata.AWSAccountId: expected a string matching "^[0-9]{12}$", actual <missing> (compat/model/scenarios/kms.json assert[0])
| |||||||
| ListKeys | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| GetKeyPolicy | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| PutKeyPolicy | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ListKeyPolicies | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| GenerateDataKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| GenerateDataKeyWithoutPlaintext | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| GenerateDataKeyPair | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| GenerateRandom | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| CreateGrant | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ListGrants | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ListRetirableGrants | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| RevokeGrant | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| CreateGrantForRetire CreateGrant | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| RetireGrant | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| CreateKeyAsymmetric CreateKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| GetPublicKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| CreateAlias | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ListAliases | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| UpdateAlias | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| DeleteAlias | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ScheduleKeyDeletionAsymmetric ScheduleKeyDeletion | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| DisableKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| EnableKey | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ScheduleKeyDeletion | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| CancelKeyDeletion | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
| ScheduleKeyDeletionFinal ScheduleKeyDeletion | AWS SDK for JavaScript v3: passes | AWS SDK for Python (boto3): passes | AWS SDK for Go v2: passes | AWS CLI v2: passes | AWS SDK for Java 2.x: passes | AWS SDK for .NET: passes | AWS SDK for Rust: passes |
kms-gen-probe generated from the AWS model
| Test | JS | Python | Go | CLI | Java | .NET | Rust |
|---|---|---|---|---|---|---|---|
| DescribeCustomKeyStores | AWS SDK for JavaScript v3: Not emulated yet | AWS SDK for Python (boto3): Not emulated yet | AWS SDK for Go v2: Not emulated yet | AWS CLI v2: Not emulated yet | AWS SDK for Java 2.x: Not emulated yet | AWS SDK for .NET: Not emulated yet | AWS SDK for Rust: Not emulated yet |
Why 7 results do not pass · 7 on Overcast AWS SDK for JavaScript v3, AWS SDK for Python (boto3), AWS SDK for Go v2, AWS CLI v2, AWS SDK for Java 2.x, AWS SDK for .NET, AWS SDK for Rust: Not emulated yet JS, Python, Go, CLI, Java, .NET, Rust
Report this
Overcast returned 501 Not Implemented: the operation has no emulation yet. Messages (6)AWS SDK for JavaScript v3 NotImplemented: kms-gen-probe/DescribeCustomKeyStores: DescribeCustomKeyStores params={} — call: expected the call to succeed, actual NotImplemented: This operation is not yet emulated. Check docs/services/ for the support matrix. (compat/model/scenarios/kms.json call)
AWS SDK for Python (boto3) An error occurred (NotImplemented) when calling the DescribeCustomKeyStores operation: This operation is not yet emulated. Check docs/services/ for the support matrix.
AWS SDK for Go v2 kms-gen-probe/DescribeCustomKeyStores: DescribeCustomKeyStores params {}: call: expected the call to succeed, actual "operation error KMS: DescribeCustomKeyStores, https response error StatusCode: 501, RequestID: d3ff7e62-68e6-448c-9e52-c12971a6bed9, api error NotImplemented: This operation is not yet emulated. Check docs/services/ for the support matrix." (compat/model/scenarios/kms.json call)
AWS SDK for Java 2.x kms-gen-probe/DescribeCustomKeyStores: DescribeCustomKeyStores params {}: call: expected the call to succeed, actual "This operation is not yet emulated. Check docs/services/ for the support matrix. (Service: Kms, Status Code: 501, Request ID: 01317319-7ed2-414b-87c4-24d92060a9a0) (SDK Attempt Count: 1)" (compat/model/scenarios/kms.json call)
AWS SDK for .NET kms-gen-probe/DescribeCustomKeyStores: DescribeCustomKeyStores params {}: call: expected the call to succeed, actual "This operation is not yet emulated. Check docs/services/ for the support matrix." (compat/model/scenarios/kms.json call)
AWS SDK for Rust kms-gen-probe/DescribeCustomKeyStores: DescribeCustomKeyStores params {}: call: expected the call to succeed, actual "service error: unhandled error (NotImplemented): Error { code: \"NotImplemented\", message: \"This operation is not yet emulated. Check docs/services/ for the support matrix.\", aws_request_id: \"a287f071-6a39-4d5f-867b-0a29f5991415\" } (ServiceError(ServiceError { source: Unhandled(Unhandled { source: ErrorMetadata { code: Some(\"NotImplemented\"), message: Some(\"This operation is not yet emulated. Check docs/services/ for the support matrix.\"), extras: Some({\"aws_request_id\": \"a287f071-6a39-4d5f-867b-0a29f5991415\"}) }, meta: ErrorMetadata { code: Some(\"NotImplemented\"), message: Some(\"This operation is not yet emulated. Check docs/services/ for the support matrix.\"), extras: Some({\"aws_request_id\": \"a287f071-6a39-4d5f-867b-0a29f5991415\"}) } }), raw: Response { status: StatusCode(501), headers: {\"access-control-allow-headers\": \"Content-Type, Authorization\", \"access-control-allow-methods\": \"GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS\", \"access-control-allow-origin\": \"*\", \"access-control-expose-headers\": \"ETag, X-Amz-*, X-*\", \"content-length\": \"120\", \"content-type\": \"application/x-amz-json-1.0\", \"x-amzn-requestid\": \"a287f071-6a39-4d5f-867b-0a29f5991415\", \"x-emulator-unsupported\": \"true\", \"date\": \"Thu, 24 Sep 2026 08:16:42 GMT\"}, body: SdkBody { inner: Once(Some(b\"{\\\"__type\\\":\\\"NotImplemented\\\",\\\"message\\\":\\\"This operation is not yet emulated. Check docs/services/ for the support matrix.\\\"}\")), retryable: true }, extensions: Extensions { extensions_02x: Extensions, extensions_1x: {hyper_util::client::legacy::connect::http::HttpInfo, aws_runtime::service_clock_skew::ResponseClockSkew} } } }))" (compat/model/scenarios/kms.json call)
| |||||||
| GetKeyRotationStatus | AWS SDK for JavaScript v3: Not emulated yet | AWS SDK for Python (boto3): Not emulated yet | AWS SDK for Go v2: Not emulated yet | AWS CLI v2: Not emulated yet | AWS SDK for Java 2.x: Not emulated yet | AWS SDK for .NET: Not emulated yet | AWS SDK for Rust: Not emulated yet |
Why 7 results do not pass · 7 on Overcast AWS SDK for JavaScript v3, AWS SDK for Python (boto3), AWS SDK for Go v2, AWS CLI v2, AWS SDK for Java 2.x, AWS SDK for .NET, AWS SDK for Rust: Not emulated yet JS, Python, Go, CLI, Java, .NET, Rust
Report this
Overcast returned 501 Not Implemented: the operation has no emulation yet. Messages (6)AWS SDK for JavaScript v3 NotImplemented: kms-gen-probe/GetKeyRotationStatus: GetKeyRotationStatus params={"KeyId":"00000000-0000-0000-0000-000000000000"} — call: expected the call to succeed, actual NotImplemented: This operation is not yet emulated. Check docs/services/ for the support matrix. (compat/model/scenarios/kms.json call)
AWS SDK for Python (boto3) An error occurred (NotImplemented) when calling the GetKeyRotationStatus operation: This operation is not yet emulated. Check docs/services/ for the support matrix.
AWS SDK for Go v2 kms-gen-probe/GetKeyRotationStatus: GetKeyRotationStatus params {"KeyId":"00000000-0000-0000-0000-000000000000"}: call: expected the call to succeed, actual "operation error KMS: GetKeyRotationStatus, https response error StatusCode: 501, RequestID: dc44f387-1894-45af-a5c4-8a2fc6fdee38, api error NotImplemented: This operation is not yet emulated. Check docs/services/ for the support matrix." (compat/model/scenarios/kms.json call)
AWS SDK for Java 2.x kms-gen-probe/GetKeyRotationStatus: GetKeyRotationStatus params {"KeyId":"00000000-0000-0000-0000-000000000000"}: call: expected the call to succeed, actual "This operation is not yet emulated. Check docs/services/ for the support matrix. (Service: Kms, Status Code: 501, Request ID: 40336ea4-9f17-423c-bf4c-ce17b8a5bca9) (SDK Attempt Count: 1)" (compat/model/scenarios/kms.json call)
AWS SDK for .NET kms-gen-probe/GetKeyRotationStatus: GetKeyRotationStatus params {"KeyId":"00000000-0000-0000-0000-000000000000"}: call: expected the call to succeed, actual "This operation is not yet emulated. Check docs/services/ for the support matrix." (compat/model/scenarios/kms.json call)
AWS SDK for Rust kms-gen-probe/GetKeyRotationStatus: GetKeyRotationStatus params {"KeyId":"00000000-0000-0000-0000-000000000000"}: call: expected the call to succeed, actual "service error: unhandled error (NotImplemented): Error { code: \"NotImplemented\", message: \"This operation is not yet emulated. Check docs/services/ for the support matrix.\", aws_request_id: \"84a28cb1-9155-4ebd-862c-1799eaab7b7a\" } (ServiceError(ServiceError { source: Unhandled(Unhandled { source: ErrorMetadata { code: Some(\"NotImplemented\"), message: Some(\"This operation is not yet emulated. Check docs/services/ for the support matrix.\"), extras: Some({\"aws_request_id\": \"84a28cb1-9155-4ebd-862c-1799eaab7b7a\"}) }, meta: ErrorMetadata { code: Some(\"NotImplemented\"), message: Some(\"This operation is not yet emulated. Check docs/services/ for the support matrix.\"), extras: Some({\"aws_request_id\": \"84a28cb1-9155-4ebd-862c-1799eaab7b7a\"}) } }), raw: Response { status: StatusCode(501), headers: {\"access-control-allow-headers\": \"Content-Type, Authorization\", \"access-control-allow-methods\": \"GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS\", \"access-control-allow-origin\": \"*\", \"access-control-expose-headers\": \"ETag, X-Amz-*, X-*\", \"content-length\": \"120\", \"content-type\": \"application/x-amz-json-1.0\", \"x-amzn-requestid\": \"84a28cb1-9155-4ebd-862c-1799eaab7b7a\", \"x-emulator-unsupported\": \"true\", \"date\": \"Thu, 24 Sep 2026 08:16:42 GMT\"}, body: SdkBody { inner: Once(Some(b\"{\\\"__type\\\":\\\"NotImplemented\\\",\\\"message\\\":\\\"This operation is not yet emulated. Check docs/services/ for the support matrix.\\\"}\")), retryable: true }, extensions: Extensions { extensions_02x: Extensions, extensions_1x: {hyper_util::client::legacy::connect::http::HttpInfo, aws_runtime::service_clock_skew::ResponseClockSkew} } } }))" (compat/model/scenarios/kms.json call)
| |||||||
| GetParametersForImport | AWS SDK for JavaScript v3: Not emulated yet | AWS SDK for Python (boto3): Not emulated yet | AWS SDK for Go v2: Not emulated yet | AWS CLI v2: Not emulated yet | AWS SDK for Java 2.x: Not emulated yet | AWS SDK for .NET: Not emulated yet | AWS SDK for Rust: Not emulated yet |
Why 7 results do not pass · 7 on Overcast AWS SDK for JavaScript v3, AWS SDK for Python (boto3), AWS SDK for Go v2, AWS CLI v2, AWS SDK for Java 2.x, AWS SDK for .NET, AWS SDK for Rust: Not emulated yet JS, Python, Go, CLI, Java, .NET, Rust
Report this
Overcast returned 501 Not Implemented: the operation has no emulation yet. Messages (6)AWS SDK for JavaScript v3 NotImplemented: kms-gen-probe/GetParametersForImport: GetParametersForImport params={"KeyId":"00000000-0000-0000-0000-000000000000","WrappingAlgorithm":"RSAES_OAEP_SHA_1","WrappingKeySpec":"RSA_2048"} — call: expected the call to succeed, actual NotImplemented: This operation is not yet emulated. Check docs/services/ for the support matrix. (compat/model/scenarios/kms.json call)
AWS SDK for Python (boto3) An error occurred (NotImplemented) when calling the GetParametersForImport operation: This operation is not yet emulated. Check docs/services/ for the support matrix.
AWS SDK for Go v2 kms-gen-probe/GetParametersForImport: GetParametersForImport params {"KeyId":"00000000-0000-0000-0000-000000000000","WrappingAlgorithm":"RSAES_OAEP_SHA_1","WrappingKeySpec":"RSA_2048"}: call: expected the call to succeed, actual "operation error KMS: GetParametersForImport, https response error StatusCode: 501, RequestID: f4f2ec34-4c6f-4152-ab90-05b6fb4d1d77, api error NotImplemented: This operation is not yet emulated. Check docs/services/ for the support matrix." (compat/model/scenarios/kms.json call)
AWS SDK for Java 2.x kms-gen-probe/GetParametersForImport: GetParametersForImport params {"KeyId":"00000000-0000-0000-0000-000000000000","WrappingAlgorithm":"RSAES_OAEP_SHA_1","WrappingKeySpec":"RSA_2048"}: call: expected the call to succeed, actual "This operation is not yet emulated. Check docs/services/ for the support matrix. (Service: Kms, Status Code: 501, Request ID: 7ba0f3e4-d35b-4db2-a66e-133af630ca31) (SDK Attempt Count: 1)" (compat/model/scenarios/kms.json call)
AWS SDK for .NET kms-gen-probe/GetParametersForImport: GetParametersForImport params {"KeyId":"00000000-0000-0000-0000-000000000000","WrappingAlgorithm":"RSAES_OAEP_SHA_1","WrappingKeySpec":"RSA_2048"}: call: expected the call to succeed, actual "This operation is not yet emulated. Check docs/services/ for the support matrix." (compat/model/scenarios/kms.json call)
AWS SDK for Rust kms-gen-probe/GetParametersForImport: GetParametersForImport params {"KeyId":"00000000-0000-0000-0000-000000000000","WrappingAlgorithm":"RSAES_OAEP_SHA_1","WrappingKeySpec":"RSA_2048"}: call: expected the call to succeed, actual "service error: unhandled error (NotImplemented): Error { code: \"NotImplemented\", message: \"This operation is not yet emulated. Check docs/services/ for the support matrix.\", aws_request_id: \"6179e296-cdb9-483b-822c-8e398e4ea0fc\" } (ServiceError(ServiceError { source: Unhandled(Unhandled { source: ErrorMetadata { code: Some(\"NotImplemented\"), message: Some(\"This operation is not yet emulated. Check docs/services/ for the support matrix.\"), extras: Some({\"aws_request_id\": \"6179e296-cdb9-483b-822c-8e398e4ea0fc\"}) }, meta: ErrorMetadata { code: Some(\"NotImplemented\"), message: Some(\"This operation is not yet emulated. Check docs/services/ for the support matrix.\"), extras: Some({\"aws_request_id\": \"6179e296-cdb9-483b-822c-8e398e4ea0fc\"}) } }), raw: Response { status: StatusCode(501), headers: {\"access-control-allow-headers\": \"Content-Type, Authorization\", \"access-control-allow-methods\": \"GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS\", \"access-control-allow-origin\": \"*\", \"access-control-expose-headers\": \"ETag, X-Amz-*, X-*\", \"content-length\": \"120\", \"content-type\": \"application/x-amz-json-1.0\", \"x-amzn-requestid\": \"6179e296-cdb9-483b-822c-8e398e4ea0fc\", \"x-emulator-unsupported\": \"true\", \"date\": \"Thu, 24 Sep 2026 08:16:42 GMT\"}, body: SdkBody { inner: Once(Some(b\"{\\\"__type\\\":\\\"NotImplemented\\\",\\\"message\\\":\\\"This operation is not yet emulated. Check docs/services/ for the support matrix.\\\"}\")), retryable: true }, extensions: Extensions { extensions_02x: Extensions, extensions_1x: {hyper_util::client::legacy::connect::http::HttpInfo, aws_runtime::service_clock_skew::ResponseClockSkew} } } }))" (compat/model/scenarios/kms.json call)
| |||||||
| ListKeyRotations | AWS SDK for JavaScript v3: Not emulated yet | AWS SDK for Python (boto3): Not emulated yet | AWS SDK for Go v2: Not emulated yet | AWS CLI v2: Not emulated yet | AWS SDK for Java 2.x: Not emulated yet | AWS SDK for .NET: Not emulated yet | AWS SDK for Rust: Not emulated yet |
Why 7 results do not pass · 7 on Overcast AWS SDK for JavaScript v3, AWS SDK for Python (boto3), AWS SDK for Go v2, AWS CLI v2, AWS SDK for Java 2.x, AWS SDK for .NET, AWS SDK for Rust: Not emulated yet JS, Python, Go, CLI, Java, .NET, Rust
Report this
Overcast returned 501 Not Implemented: the operation has no emulation yet. Messages (6)AWS SDK for JavaScript v3 NotImplemented: kms-gen-probe/ListKeyRotations: ListKeyRotations params={"KeyId":"00000000-0000-0000-0000-000000000000"} — call: expected the call to succeed, actual NotImplemented: This operation is not yet emulated. Check docs/services/ for the support matrix. (compat/model/scenarios/kms.json call)
AWS SDK for Python (boto3) An error occurred (NotImplemented) when calling the ListKeyRotations operation: This operation is not yet emulated. Check docs/services/ for the support matrix.
AWS SDK for Go v2 kms-gen-probe/ListKeyRotations: ListKeyRotations params {"IncludeKeyMaterial":"","KeyId":"00000000-0000-0000-0000-000000000000"}: call: expected the call to succeed, actual "operation error KMS: ListKeyRotations, https response error StatusCode: 501, RequestID: 7d505c06-2a52-4791-83ce-50370cf992b5, api error NotImplemented: This operation is not yet emulated. Check docs/services/ for the support matrix." (compat/model/scenarios/kms.json call)
AWS SDK for Java 2.x kms-gen-probe/ListKeyRotations: ListKeyRotations params {"KeyId":"00000000-0000-0000-0000-000000000000"}: call: expected the call to succeed, actual "This operation is not yet emulated. Check docs/services/ for the support matrix. (Service: Kms, Status Code: 501, Request ID: 62525ac9-f189-46d1-8896-e0698e2caecb) (SDK Attempt Count: 1)" (compat/model/scenarios/kms.json call)
AWS SDK for .NET kms-gen-probe/ListKeyRotations: ListKeyRotations params {"KeyId":"00000000-0000-0000-0000-000000000000"}: call: expected the call to succeed, actual "This operation is not yet emulated. Check docs/services/ for the support matrix." (compat/model/scenarios/kms.json call)
AWS SDK for Rust kms-gen-probe/ListKeyRotations: ListKeyRotations params {"KeyId":"00000000-0000-0000-0000-000000000000"}: call: expected the call to succeed, actual "service error: unhandled error (NotImplemented): Error { code: \"NotImplemented\", message: \"This operation is not yet emulated. Check docs/services/ for the support matrix.\", aws_request_id: \"bac1ca57-84f0-411d-9eea-805e46497323\" } (ServiceError(ServiceError { source: Unhandled(Unhandled { source: ErrorMetadata { code: Some(\"NotImplemented\"), message: Some(\"This operation is not yet emulated. Check docs/services/ for the support matrix.\"), extras: Some({\"aws_request_id\": \"bac1ca57-84f0-411d-9eea-805e46497323\"}) }, meta: ErrorMetadata { code: Some(\"NotImplemented\"), message: Some(\"This operation is not yet emulated. Check docs/services/ for the support matrix.\"), extras: Some({\"aws_request_id\": \"bac1ca57-84f0-411d-9eea-805e46497323\"}) } }), raw: Response { status: StatusCode(501), headers: {\"access-control-allow-headers\": \"Content-Type, Authorization\", \"access-control-allow-methods\": \"GET, HEAD, POST, PUT, PATCH, DELETE, OPTIONS\", \"access-control-allow-origin\": \"*\", \"access-control-expose-headers\": \"ETag, X-Amz-*, X-*\", \"content-length\": \"120\", \"content-type\": \"application/x-amz-json-1.0\", \"x-amzn-requestid\": \"bac1ca57-84f0-411d-9eea-805e46497323\", \"x-emulator-unsupported\": \"true\", \"date\": \"Thu, 24 Sep 2026 08:16:42 GMT\"}, body: SdkBody { inner: Once(Some(b\"{\\\"__type\\\":\\\"NotImplemented\\\",\\\"message\\\":\\\"This operation is not yet emulated. Check docs/services/ for the support matrix.\\\"}\")), retryable: true }, extensions: Extensions { extensions_02x: Extensions, extensions_1x: {hyper_util::client::legacy::connect::http::HttpInfo, aws_runtime::service_clock_skew::ResponseClockSkew} } } }))" (compat/model/scenarios/kms.json call)
| |||||||
No test matches these filters.
Not tested (22)
Operations with no test at all. The test generator declined to write one, for the reason given.
| Operation | Reason | Detail |
|---|---|---|
| ConnectCustomKeyStore | never-probe | Connects a custom key store to its AWS CloudHSM cluster or external key store proxy, which changes whether every KMS key in that store can be used at all. |
| CreateCustomKeyStore | never-probe | Creates a custom key store bound to a CloudHSM cluster or an external key manager. Nothing in a probe group removes it, and an orphaned key store keeps its backing cluster associated. |
| Decrypt | probe-of-implemented-op | Decrypt is implemented (Supported), so it may not be probed, and no recipe resource gives it a role |
| DeleteCustomKeyStore | never-probe | Deletes a custom key store. AWS refuses only while the store still holds KMS keys, so a store emptied by an earlier deletion is destroyed by this call. |
| DeleteImportedKeyMaterial | never-probe | Deletes the imported key material of a KMS key. Every ciphertext encrypted under that key becomes undecryptable until the same material is imported again, and AWS keeps no copy of it. |
| DeriveSharedSecret | never-probe | Performs an ECDH key agreement with the caller's public key and bills for it. It also takes a raw public key blob, which the IR cannot express. |
| DisableKeyRotation | never-probe | Turns off automatic annual rotation for a live KMS key, which is a control an account's compliance posture is usually built on. |
| DisconnectCustomKeyStore | never-probe | Disconnects a custom key store from its backing cluster. Every KMS key in the store stops working for cryptographic operations until it is reconnected. |
| EnableKeyRotation | never-probe | Turns on automatic annual rotation for a live KMS key. AWS bills for each key version the rotation creates, and rotation cannot be undone for versions already generated. |
| Encrypt | probe-of-implemented-op | Encrypt is implemented (Supported), so it may not be probed, and no recipe resource gives it a role |
| GenerateDataKeyPairWithoutPlaintext | never-probe | GenerateDataKeyPairWithoutPlaintext is not a read operation; probes call unimplemented operations against a live account and must be side-effect free |
| GenerateMac | never-probe | GenerateMac is not a read operation; probes call unimplemented operations against a live account and must be side-effect free |
| GetKeyLastUsage | probe-binds-live-resource | GetKeyLastUsage.KeyId would bind to key.id, a value exported from a resource the run owns; a probe calls an operation the emulator does not implement, so against real AWS it would act on that resource for real. Add a curated literal for KeyId to values.json, or leave the operation refused |
| ImportKeyMaterial | never-probe | Replaces the key material of a KMS key with material the caller supplies, which changes what every existing ciphertext under that key decrypts with. |
| ReEncrypt | probe-of-implemented-op | ReEncrypt is implemented (Supported), so it may not be probed, and no recipe resource gives it a role |
| ReplicateKey | never-probe | Creates a replica of a multi-Region key in another Region. The replica is a KMS key of its own, in a Region nothing in this run looks at, and it can only be removed by scheduling its deletion there. |
| RotateKeyOnDemand | never-probe | Creates a new key version of a live KMS key immediately. AWS bills for each version and a version cannot be removed. |
| Sign | probe-of-implemented-op | Sign is implemented (Supported), so it may not be probed, and no recipe resource gives it a role |
| UpdateCustomKeyStore | never-probe | Changes the cluster, credentials or proxy configuration of a live custom key store, which can leave every KMS key in it unusable. |
| UpdatePrimaryRegion | never-probe | Moves the primary of a multi-Region key to another Region, demoting the current primary to a replica. It changes which Region may replicate and manage the key. |
| Verify | probe-of-implemented-op | Verify is implemented (Supported), so it may not be probed, and no recipe resource gives it a role |
| VerifyMac | probe-of-implemented-op | VerifyMac is implemented (Supported), so it may not be probed, and no recipe resource gives it a role |