Why does '_cat/indices?v' return 403 Forbidden for Amazon OpenSearch Serverless?

Question

I am running a simple curl request using Curl 8.3.0 to get info about indices in an Amazon OpenSearch cluster.

curl -XGET https://my_domain.us-east-1.aoss.amazonaws.com/_cat/indices?v \
  --aws-sigv4 aws:amz:us-east-1:aoss \
  --user $AWS_ACCESS_KEY:$AWS_SECRET_KEY \
  --header "Content-Type: application/json; charset=utf-8" \
  --header "x-amz-security-token:${AWS_SESSION_TOKEN}"

However, I get a 403 Forbidden error:

{"status":403,"request-id":"a6603a35-6757-9a13-86c9-xxx","error":{"reason":"403 Forbidden","type":"Forbidden"}}

Output of curl -v:

Note: Unnecessary use of -X or --request, GET is already inferred.
*   Trying x.x.x.x:443...
* Connected to my_domain.us-east-1.aoss.amazonaws.com (x.x.x.x) port 443
* ALPN: curl offers http/1.1
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: none
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=*.us-east-1.aoss.amazonaws.com
*  start date: Dec 18 00:00:00 2022 GMT
*  expire date: Jan 16 23:59:59 2024 GMT
*  subjectAltName: host "my_domain.us-east-1.aoss.amazonaws.com" matched cert's "*.us-east-1.aoss.amazonaws.com"
*  issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
*  SSL certificate verify ok.
* using HTTP/1.x
* Server auth using AWS_SIGV4 with user ''
> GET /_cat/indices?v HTTP/1.1
> Host: my_domain.us-east-1.aoss.amazonaws.com
> Authorization: AWS4-HMAC-SHA256 Credential=/20230927/us-east-1/aoss/aws4_request, SignedHeaders=content-type;host;x-amz-date;x-amz-security-token, Signature=ab....
> X-Amz-Date: 20230927T065937Z
> User-Agent: curl/8.3.0
> Accept: */*
> Content-Type: application/json; charset=utf-8
> x-amz-security-token:abc
> 
< HTTP/1.1 403 Forbidden
< x-request-id: 642feace-d1d2-9705-a97b-xxx
< x-aoss-response-hint: X01:gw-helper-deny
< content-type: application/json
< date: Wed, 27 Sep 2023 06:59:37 GMT
< content-length: 121
< server: aoss-amazon
< 
{"status":403,"request-id":"642feace-d1d2-9705-a97b-xxx","error":{"reason":"403 Forbidden","type":"Forbidden"}}
* Connection #0 to host my_domain.us-east-1.aoss.amazonaws.com left intact

Here is my data access policy:

{
  "Rules": [
    {
      "ResourceType": "index",
      "Resource": [
        "index/my_collection/*"
      ],
      "Permission": [
        "aoss:CreateIndex",
        "aoss:DeleteIndex",
        "aoss:DescribeIndex",
        "aoss:ReadDocument",
        "aoss:UpdateIndex",
        "aoss:WriteDocument"
      ]
    },
    {
      "ResourceType": "collection",
      "Resource": [
        "collection/my_collection"
      ],
      "Permission": [
        "aoss:CreateCollectionItems",
        "aoss:DeleteCollectionItems",
        "aoss:DescribeCollectionItems",
        "aoss:UpdateCollectionItems"
      ]
    }
  ],
  "Principal": [
    my_role_arn,
  ]
}

Answer 1

class MyModal {
  int myField1;
  String myField2;
  List<MyModal> adjacentNodes;
  MyModal(this.myField1,this.myField2);

  MyModal.clone(MyModal source) : 
      this.myField1 = source.myField1, 
      this.myField2 = source.myField2,
      this.adjacentNodes = source.adjacentNodes.map((item) => new MyModal.clone(item)).toList();
}

var secondList = originalList.map((item) => new MyModal.clone(item)).toList();

If a member of MyModal is of a non-primitive type like String, int, double, num, bool, then the clone() method needs to clone the instances references point to as well.

I think for your use case using immutable values is a better approach, for example with https://pub.dartlang.org/packages/built_value

Answer 2

you can use List.from() function. try this code:

//modal for list
class MyModal {
  int myField1;
  String name;
  List<MyModal> adjacentNodes;
  MyModal(this.myField1, this.name) {
    adjacentNodes = new List<MyModal>();
  }
}

void runCopy() {
//pre code
  List<MyModal> originalList = new List<MyModal>();
  originalList.add(new MyModal(1, "firstBuddy"));

//copying list
  List<MyModal> secondList = List.from(originalList);
  secondList.addAll(originalList);
  print(originalList);
  print(secondList);
}

Answer 3

Call _cat/indices?v=true instead.

This is a bug within OpenSearch Serverless & the _cat/indices endpoint. It incorrectly returns a 403 Forbidden, if you do not provide an actual boolean value for v.

Endpoint	Amazon OpenSearch Serverless	Local OpenSearch Instance
_cat/indices	✅ 200 OK	✅ 200 OK
_cat/indices?v	❌ 403 Forbidden	✅ 200 OK
_cat/indices?v=false	✅ 200 OK	✅ 200 OK
_cat/indices?v=true	✅ 200 OK	✅ 200 OK

Amazon OpenSearch Serverless:

➜  ~ curl -XGET "https://xxx.eu-west-1.aoss.amazonaws.com/_cat/indices" \
          --aws-sigv4 "aws:amz:eu-west-1:aoss" \
          --user "$AWS_ACCESS_KEY":"$AWS_SECRET_KEY" \
          --header "Content-Type: application/json; charset=utf-8" \
          --header "x-amz-security-token:${AWS_SESSION_TOKEN}" -i
HTTP/1.1 200 OK
content-type: text/plain; charset=UTF-8
content-length: 0
x-envoy-upstream-service-time: 14
date: Wed, 18 Oct 2023 12:21:25 GMT
server: aoss-amazon-m
x-request-id: f8785ab2-e971-96f5-b7ce-0a613acf38c5

➜  ~ curl -XGET "https://xxx.eu-west-1.aoss.amazonaws.com/_cat/indices?v" \
          --aws-sigv4 "aws:amz:eu-west-1:aoss" \
          --user "$AWS_ACCESS_KEY":"$AWS_SECRET_KEY" \
          --header "Content-Type: application/json; charset=utf-8" \
          --header "x-amz-security-token:${AWS_SESSION_TOKEN}" -i
HTTP/1.1 403 Forbidden
x-request-id: ef2b256b-3bb2-9583-be96-86eb88f17660
x-aoss-response-hint: X01:gw-helper-deny
content-type: application/json
date: Wed, 18 Oct 2023 12:21:28 GMT
content-length: 121
server: aoss-amazon

{"status":403,"request-id":"ef2b256b-3bb2-9583-be96-86eb88f17660","error":{"reason":"403 Forbidden","type":"Forbidden"}}

➜  ~ curl -XGET "https://xxx.eu-west-1.aoss.amazonaws.com/_cat/indices?v=false" \
          --aws-sigv4 "aws:amz:eu-west-1:aoss" \
          --user "$AWS_ACCESS_KEY":"$AWS_SECRET_KEY" \
          --header "Content-Type: application/json; charset=utf-8" \
          --header "x-amz-security-token:${AWS_SESSION_TOKEN}" -i
HTTP/1.1 200 OK
content-type: text/plain; charset=UTF-8
content-length: 0
x-envoy-upstream-service-time: 23
date: Wed, 18 Oct 2023 12:21:33 GMT
server: aoss-amazon-m
x-request-id: 9007beb6-17b0-9eb9-942c-22a2794a687f

➜  ~ curl -XGET "https://xxx.eu-west-1.aoss.amazonaws.com/_cat/indices?v=true" \
          --aws-sigv4 "aws:amz:eu-west-1:aoss" \
          --user "$AWS_ACCESS_KEY":"$AWS_SECRET_KEY" \
          --header "Content-Type: application/json; charset=utf-8" \
          --header "x-amz-security-token:${AWS_SESSION_TOKEN}" -i
HTTP/1.1 200 OK
content-type: text/plain; charset=UTF-8
content-length: 83
x-envoy-upstream-service-time: 35
date: Wed, 18 Oct 2023 12:21:37 GMT
server: aoss-amazon-m
x-request-id: da7599ca-c624-91fc-b964-a89c78463265

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

Local Amazon OpenSearch:

➜  ~ curl -XGET "127.0.0.1:9200/_cat/indices" -i
HTTP/1.1 200 OK
content-type: text/plain; charset=UTF-8
content-length: 0

➜  ~ curl -XGET "127.0.0.1:9200/_cat/indices?v" -i
HTTP/1.1 200 OK
content-type: text/plain; charset=UTF-8
content-length: 83

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size

➜  ~ curl -XGET "127.0.0.1:9200/_cat/indices?v=false" -i
HTTP/1.1 200 OK
content-type: text/plain; charset=UTF-8
content-length: 0

➜  ~ curl -XGET "127.0.0.1:9200/_cat/indices?v=true" -i
HTTP/1.1 200 OK
content-type: text/plain; charset=UTF-8
content-length: 83

health status index uuid pri rep docs.count docs.deleted store.size pri.store.size