How to list Virtual Machines Classic in Azure

Question

I would like to programmatically list and control virtual machines classic (old one) in Azure. For managed it is not problem, there are libraries and the rest API is working, but once I am calling the old API for listing classic, I got 403 (Forbidden).

Is the code fine? Do I need to manage credentials for old API on another place?

My code is here:

static void Main(string[] args)
{
    string apiNew = "https://management.azure.com/subscriptions/xxxxxxxxxxxxxxxxxxxxxxxx/providers/Microsoft.Compute/virtualMachines?api-version=2018-06-01";
    string apiOld = "https://management.core.windows.net/xxxxxxxxxxxxxxxxxxxxxxxx/services/vmimages"

    AzureRestClient client = new AzureRestClient(credentials.TenantId, credentials.ClientId, credentials.ClientSecret);

    //OK - I can list the managed VMs.         
    string resultNew = client.GetRequestAsync(apiNew).Result;

    // 403 forbidden
    string resultOld = client.GetRequestAsync(apiOld).Result;        
}

public class AzureRestClient : IDisposable
{
    private readonly HttpClient _client;

    public AzureRestClient(string tenantName, string clientId, string clientSecret)
    {
        _client = CreateClient(tenantName, clientId, clientSecret).Result;
    }

    private async Task<string> GetAccessToken(string tenantName, string clientId, string clientSecret)
    {
        string authString = "https://login.microsoftonline.com/" + tenantName;
        string resourceUrl = "https://management.core.windows.net/";

        var authenticationContext = new AuthenticationContext(authString, false);
        var clientCred = new ClientCredential(clientId, clientSecret);
        var authenticationResult = await authenticationContext.AcquireTokenAsync(resourceUrl, clientCred);
        var token = authenticationResult.AccessToken;

        return token;
    }

    async Task<HttpClient> CreateClient(string tenantName, string clientId, string clientSecret)
    {
        string token = await GetAccessToken(tenantName, clientId, clientSecret);
        HttpClient client = new HttpClient();
        client.DefaultRequestHeaders.Accept.Clear();
        client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);        
        return client;
    }

    public async Task<string> GetRequestAsync(string url)
    {           
        return await _client.GetStringAsync(url);            
    }
}

UPDATE 1:

Response details:

HTTP/1.1 403 Forbidden
Content-Length: 288
Content-Type: application/xml; charset=utf-8
Server: Microsoft-HTTPAPI/2.0
Date: Mon, 22 Oct 2018 11:03:40 GMT

HTTP/1.1 403 Forbidden
Content-Length: 288
Content-Type: application/xml; charset=utf-8
Server: Microsoft-HTTPAPI/2.0
Date: Mon, 22 Oct 2018 11:03:40 GMT

<Error xmlns="http://schemas.microsoft.com/windowsazure" xmlns:i="http://www.w3.org/2001/XMLSchema-instance">
    <Code>ForbiddenError</Code>
    <Message>The server failed to authenticate the request.
      Verify that the certificate is valid and is associated with this subscription.</Message>
</Error>

Update 2:

I found that same API is used by powershell command Get-AzureVMImage and it is working from powershell. Powershell ask me first to login to Azure with interactive login windows by email and password and the the request use Bearer header to authenticate like mine code.

If I sniff the access token (Bearer header) from communication created by Powershell, I can communicate with that API with success.

Update 3: SOLVED, answer bellow.

Answer 1

1. Reason for 403 when you're calling List VM Images API

It's because your Azure AD registered application is not using the "Windows Azure Service Management API" delegated permissions correctly. I say this because I see your code is acquiring the token directly using application identity (ClientCredential) and not as a user.

Please see the screenshots below. Window Azure Service Management API clearly does not provide any application permissions, only thing that can be used is a delegated permission. If you want to understand more about the difference between the two kinds of permissions, read Permissions in Azure AD. To put it very briefly, when using delegated permissions, the app is delegated permission to act as the signed-in user when making calls to an API. So there has to be a signed-in user.

I was able to reproduce the 403 error using your code and then able to make it work and return a list of classic VM's with some changes. I'll explain the required changes next.

Go to your Azure AD > App registrations > your app > Settings > Required permissions :

2. Changes required to make it work

Change will be to acquire token as a signed in user and not directly using application's clientId and secret. Since your application is a console app, it would make sense to do something like this, which will prompt the user to enter credentials:

var authenticationResult = await authenticationContext.AcquireTokenAsync(resourceUrl, clientId, new Uri(redirectUri), new PlatformParameters(PromptBehavior.Auto));

Also, since your application is a console application, it would be better to register it as a "Native" application instead of a web application like you have it right now. I say this because console applications or desktop client based applications which can run on user systems are not secure to handle application secrets, so you should not register them as "Web app / API" and not use any secrets in them as it's a security risk.

So overall, 2 changes and you should be good to go. As I said earlier, I have tried these and can see the code working fine and getting a list of classic VMs.

a. Register your application in Azure AD as a native app (i.e. Application Type should be native and not Web app / API), then in required permissions add the "Window Azure Service Management API" and check the delegated permissions as per earlier screenshots in point 1

b. Change the way to acquire token, so that delegated permissions can be used as per the signed in user. Of course, signed in user should have permissions to the VM's you're trying to list or if you have multiple users, the list will reflect those VM's which currently signed in user has access to.

Here is the entire working code after I modified it.

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.IdentityModel.Clients.ActiveDirectory;
using System.Net.Http;
using System.Net.Http.Headers;

namespace ListVMsConsoleApp
{
    class Program
    {
        static void Main(string[] args)
        {
            string tenantId = "xxxxxx";
            string clientId = "xxxxxx";
            string redirectUri = "https://ListClassicVMsApp";

            string apiNew = "https://management.azure.com/subscriptions/xxxxxxxx/providers/Microsoft.Compute/virtualMachines?api-version=2018-06-01";
            string apiOld = "https://management.core.windows.net/xxxxxxxx/services/vmimages";

            AzureRestClient client = new AzureRestClient(tenantId, clientId, redirectUri);

            //OK - I can list the managed VMs.         
            //string resultNew = client.GetRequestAsync(apiNew).Result;

            // 403 forbidden - should work now
            string resultOld = client.GetRequestAsync(apiOld).Result;
        }

    }

    public class AzureRestClient
    {
        private readonly HttpClient _client;

        public AzureRestClient(string tenantName, string clientId, string redirectUri)
        {
            _client = CreateClient(tenantName, clientId, redirectUri).Result;
        }

        private async Task<string> GetAccessToken(string tenantName, string clientId, string redirectUri)
        {
            string authString = "https://login.microsoftonline.com/" + tenantName;
            string resourceUrl = "https://management.core.windows.net/";

            var authenticationContext = new AuthenticationContext(authString, false);
            var authenticationResult = await authenticationContext.AcquireTokenAsync(resourceUrl, clientId, new Uri(redirectUri), new PlatformParameters(PromptBehavior.Auto));

            return authenticationResult.AccessToken;
        }

        async Task<HttpClient> CreateClient(string tenantName, string clientId, string redirectUri)
        {
            string token = await GetAccessToken(tenantName, clientId, redirectUri);
            HttpClient client = new HttpClient();
            client.DefaultRequestHeaders.Accept.Clear();
            client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", token);
            client.DefaultRequestHeaders.Add("x-ms-version", "2014-02-01");
            return client;
        }

        public async Task<string> GetRequestAsync(string url)
        {
            return await _client.GetStringAsync(url);
        }
    }
}

Answer 2

According to the linked documentation you appear to be missing a required request header when requesting the classic REST API

x-ms-version - Required. Specifies the version of the operation to use for this request. This header should be set to 2014-02-01 or higher.

Reference List VM Images: Request Headers

To allow for the inclusion of the header, create an overload for GET requests in the AzureRestClient

public async Task<string> GetRequestAsync(string url, Dictionary<string, string> headers) {
    var request = new HttpRequestMessage(HttpMethod.Get, url);
    if (headers != null)
        foreach (var header in headers) {
            request.Headers.TryAddWithoutValidation(header.Key, header.Value);
        }
    var response = await _client.SendAsync(request);
    return await response.Content.ReadAsStringAsync();
}

and include the required header when calling apiOld

var headers = new Dictionary<string, string>();
headers["x-ms-version"] = "2014-02-01";

string resultOld = client.GetRequestAsync(apiOld, headers).GetAwaiter().GetResult();

Answer 3

Finnaly I got it to work:

First Open Powershell:

Get-AzurePublishSettingsFile

and save that file.

then type in Powershell

Import-AzurePublishSettingsFile [mypublishsettingsfile]

Open certificate store and find imported certificate. And use that certificate at the same time with credentials within the HttpClient.