List all directories and secrets (recursively) in Vault

Question

I'm writing a method in Python that takes in an engine name, and lists all of the sub directories and secrets in the directory. I've been playing around with hvac and I've been able to list all of the secrets within a specific directory using the following:

client = hvac.Client()
client = hvac.Client(
  url=os.environ['VAULT_URL'],
  token=os.environ['VAULT_TOKEN']
)

path='myDirectory'
mount_point='myEngine'
response=client.secrets.kv.read_secret_version(
  path=path,
  mount_point=mount_point
)

print(response['data']['data'])

As said, this does successfully work, and it does output whats inside the path specified, but if I want to list everything inside of the mount_point, I've found answers that say I should use list_secrets, but I can't seem to get list_secrets to work without specifying a path as well.

I've tried the following with very little success.

mount_point = 'myEngine'
list_response = client.secrets.kv.v2.list_secrets(
  path=mount_point
)

list_folders = list_response['data']['keys']
print(list_folders)

This obviously doesn't work, but it does work if I give it both the path and the mount_point, but that gives me the contents of the path, and I need everything in the engine.

I know giving it the mount_point for the path seems odd, but I couldn't really think of what else to do to list out the entire mount_point and I'd seen old examples that looked similar to that.

Is there a way to get the output in JSON like I'm wanting, or even a way to just list everything inside of the engine itself (recursively) and then build the json myself would be great.

Answer 1

You can use the client.adapter.request() function to achieve what your looking for. Pass the mount name into the following call:

client.adapter.request("GET", "v1/<mount name>/metadata/?list=1")

That will return JSON that you can parse to get the keys your looking for. Sample JSON below:

{'request_id': '16fedd1b-50b3-f46a-44c7-22d0a0c8edef', 'lease_id': '', 'renewable': False, 'lease_duration': 0, 'data': {'keys': ['test', 'test2']}, 'wrap_info': None, 'warnings': None, 'auth': None}

The output above corresponds to the following view in the Vault UI:

Vault UI view of keys

Answer 2

You can use the following to recursively list all the secrets using hvac.

def list_secrets(client, path=None, mount_path=None):
    list_response = client.secrets.kv.v2.list_secrets(
        path=path,
        mount_point=mount_path
    )

    return list_response


def list_all_secrets(client, global_path_list, path=None, mount_path=None):
    if mount_path is None:
        pass

    if path is None:
        try:
            response = list_secrets(client, mount_path=mount_path)
        except VaultError:
            return
    else:
        response = list_secrets(client, mount_path=mount_path, path=path)

    keys = response['data']['keys']

    for key in keys:
        if key[-1] == "/":
            if path is not None:
                _path = path + key
            else:
                _path = key
            list_all_secrets(client, global_path_list, mount_path=mount_path, path=_path)
        else:
            _temp = path + key

            if _temp is None:
                pass
            else:
                global_path_list.append(_temp)

    return global_path_list

global_path_list = []

secrets_list = list_all_secrets(client, global_path_list, 'myDirectory/', 'myEngine/')

print(secrets_list)