Azure REST API Scopes

When working with the Azure REST API you need to provide the scope in all API requests, so Azure knows where you are looking. However, throughout their documentation that although they ask for the scope they do not explain or link to an explanation of what a scope is and what the formats are. Therefore, I have collected them and got a simple explanation for each of them.

As mentioned above, the scope is like a search filter and also part of the permissions. For example, if you was getting a list of Resources you might use the Resource Group scope to get only them Resources, or you might go for the Subscription scope to get all Resources in the requested Subscription. This might also be due to permissions, if the Service Principle account you are using doesn’t have access to the whole Tenant, but does to specific Subscriptions.

Scopes

Subscription scope

subscriptions/{subscriptionId}

Example:

subscriptions/d7f90b53-af20-4061-8206-f05e31852a44

Resource Group scope

subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}

Example:

subscriptions/d7f90b53-af20-4061-8206-f05e31852a44/resourceGroups/my-rg-2020

Providers scope

These scopes can vary depending on what the scope is for. For example this is the scope for the Billing Account:

providers/Microsoft.Billing/billingAccounts/{billingAccountId}

You can find all of the scopes by following https://docs.microsoft.com/en-us/azure/role-based-access-control/resource-provider-operations

Tenant scope

The Tenant scope is easist, as you just don’t put anything. For example getting a list of Role Definitions

The official URL is:

GET https://management.azure.com/{scope}/providers/Microsoft.Authorization/roleDefinitions?api-version=2015-07-01

But to get the Role Definitions from the Tenant Level and below, you just remove the scope segment:

GET https://management.azure.com/providers/Microsoft.Authorization/roleDefinitions?api-version=2015-07-01

Deleting Recovery Service Vault Backup Server (MABS)

This is one of them quick shares to try help the world of Azure developers just a little bit.

While doing some Azure Development using the Microsoft Azure REST API, I got to a point to working in the Azure Recovery Service Vault. I was trying to delete a Back Up Server Agent using the API, which after a little bit of a hunt I found it is actually called ‘Registered Identities

This of course only has one endpoint to call, so it must just be that easy…
However when calling this API I would just get the following:

{
"error":
{
"code":"ServiceContainerNotEmptyWithBackendMessage",
"message":" There are backup items associated with the server. Stop protection and delete backups for each of the backup items to delete the server's registration. If the issue persists, contact support. Refer to https://aka.ms/DeleteBackupItems for more info. ",
"target":null,
"details":null,
"innerError":null
}
}

This was odd as I couldn’t find anything to imply a reason for this error. After then investigating the network traffic while deleting the resource in the Azure Portal, I found a slight difference which also fixed the issue. This was the version of the API.

In the documentation it is stated as ‘2016-06-01’ , but in the network traffic it had version ‘2019-05-13’. If you now change:

DELETE https://management.azure.com/Subscriptions/77777777-d41f-4550-9f70-7708a3a2283b/resourceGroups/BCDRIbzRG/providers/Microsoft.RecoveryServices/vaults/BCDRIbzVault/registeredIdentities/dpmcontainer01?api-version=2016-06-01

To

DELETE https://management.azure.com/Subscriptions/77777777-d41f-4550-9f70-7708a3a2283b/resourceGroups/BCDRIbzRG/providers/Microsoft.RecoveryServices/vaults/BCDRIbzVault/registeredIdentities/dpmcontainer01?api-version=2019-05-13

<p value="<amp-fit-text layout="fixed-height" min-font-size="6" max-font-size="72" height="80">It will now all work.It will now all work.

How to get Azure Backup Protection Containers?

When working with the Azure REST API, I had the requirement to get the Recovery Service Vaults Backup Protection Containers. Naturally, like many, I went to the Microsoft Azure REST API documentation website to learn how to call this API, but when following it I hit a few issues that was not clear and not part of the documentation. Here below are some of the issues I had, that I hope can solve yours… If your having any.

This is the link to the documentation if you don’t have it already to follow.
https://docs.microsoft.com/en-gb/rest/api/backup/backupprotectioncontainers/list

Get All Doesn’t Work

As you can see from the documentation you should be able to call the GET request for:

GET https://management.azure.com/Subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.RecoveryServices/vaults/{vaultName}/backupProtectionContainers?api-version=2016-12-01

This should be a non-filtered list of all the containers not matter their type. However, when calling this you will get the error saying you are missing required properties. With such a helpful message response, how could you not understand what is going wrong.

What I have found is you need to use the filter request, but it is still not that simple. From the example in the documentation you can use the Backup Management Type. However, if like me you want all the containers no matter the type then this isn’t the most helpful, so looking into the OData filters documentation you can use many other requests.

Instead of doing backupManagementType eq ‘AzureWorkload’ which is Backup Management Type equals Azure Workload, you can use ‘ne’ that is not equal to. With this you can put some random text in the string to say Get All Backup Protection Containers Where Backup Management Type Does Not Equal RandomWords. Makes sense yes? Well not for this API. This still results in the same error. I even tried to change the parameter it is searching on to something all valid containers should have like ‘registrationStatus’, but that didn’t work.

The only method I got this API call to work was to call it with the filter of Backup Management Type.

Trying to reduce code I added the Backup Models from the Nuget Package Microsoft.Azure.Management.RecoveryServices.Backup

Azure C# SDK missing options

As stated above you need to use the filter with Backup Management Type to get the Protected Containers, so to get all of them we need to call each type. As I don’t know all the types,  I found the Enum BackupManagementTypes in the above Nuget package that I can then loop through.

However, I am not sure if it is just out of date or that the Nuget package is in preview, but no all the types are in there. Therefore, I created my own using the values in the API Documentation.

/// <summary>
/// Azure Backup Management Types 
/// https://docs.microsoft.com/en-gb/rest/api/backup/backupprotectioncontainers/list#backupmanagementtype
/// </summary>
        public enum BackupManagmentTypes
        {
            AzureBackupServer,
            AzureIaasVM,
            AzureSql,
            AzureStorage,
            AzureWorkload,
            DPM,
            DefaultBackup,
            Invalid,
            MAB
        }

Then it can be  used like this:

foreach (var backupManagementTypeStr in Enum.GetValues(typeof(BackupManagmentTypes)))
{
	var urlWithFilter = $"{containerUrl}&$filter=backupManagementType eq '{backupManagementTypeStr}'";
}

Azure C# SDK incorrect format

With the Nuget package I thought I could the Desterilise the response into the class ProtectionContainerListResponse as this is what the documentation says the type is, but it failed as the model of this class follows ‘response.ItemList.ProtectionContainers’, whereas the expected response follows ‘response.values’

When looking at the response in the API Documentation I have also seen a pattern to the response types for the Recovery Service API. They all have a list with ‘nextlink’ and ‘values’, which values is a list of the response type. However, the types in the list also follow the same pattern with all the properties the same except what type the property ‘properties’ is. Therefore, you can make a generic response as below:

  /// <summary>
        /// API Response
        /// </summary>
        public class ResourceResponse<T> where T : class
        {
            /// <summary>
            /// The uri to fetch the next page of resources. Call ListNext() fetches next page of resources.
            /// </summary>
            public string nextLink { get; set; }
            /// <summary>
            /// List of resources.
            /// </summary>
            public List<ResourceDetails<T>> value { get; set; }
        }

        /// <summary>
        /// Resource Details
        /// </summary>
        public class ResourceDetails<T> where T : class
        {
            /// <summary>
            /// Optional ETag.
            /// </summary>
            public string eTag { get; set; }
            /// <summary>
            /// Resource Id represents the complete path to the resource.
            /// </summary>
            public string id { get; set; }
            /// <summary>
            /// Resource location.
            /// </summary>
            public string location { get; set; }
            /// <summary>
            /// Resource name associated with the resource.
            /// </summary>
            public string name { get; set; }
            /// <summary>
            /// resource properties
            /// </summary>
            public T properties { get; set; }
            /// <summary>
            /// Resource tags.
            /// </summary>
            public object tags { get; set; }
            /// <summary>
            /// Resource type represents the complete path of the form Namespace/ResourceType/ResourceType/...
            /// </summary>
            public string type { get; set; }
        }

How to authenticate with Fortify Security with PowerShell

Fortify, a security scanning tool for code, has some great features but also some limiting features. Therefore I sought to use their open REST API to expand on its functionality to enhance how we are using it within the DevOps pipeline. This was of course step one to find how to authenticate with Fortify to start doing the requests to its services.

Fortify does have the Swagger page of the URL’s to show what endpoints it offers, but doesn’t detail the authentication endpoint. It then does have the documentation on how to authenticate, but it is not detailed out for easy use.

Therefore this is why I thought I would expand on the details to show other how to authenticate easily, while using PowerShell as the chosen language.

Fortify Swagger

The API Layer from Fortify provides the Swagger definitions. If you chose you provided Data Centre from the link below, you can then simply add ‘/swagger’ to the end to see the definitions, for example https://api.emea.fortify.com/swagger/ui/index

Data Centre URL: https://emea.fortify.com/Docs/en/Content/Additional_Services/API/API_About.htm

Authentication

As mentioned before Fortify does document how to authenticate with the API here https://emea.fortify.com/Docs/en/index.htm#Additional_Services/API/API_Auth.htm%3FTocPath%3DAPI%7C_____3

First thing is to find out what details you require for the request like it has mentioned in the documentation. We require the calling Data Centre URL, which you used above for the Swagger definitions, that is then suffixed with ‘/oauth/token’ e.g. ‘https://api.emea.fortify.com/oauth/token&#8217;

We then need scope of what you would like to request, which are both detailed out in this link in the documentation plus also on each of the Swagger definition under the ‘Implementation Notes’, it specifies what scope is required for each request. This value needs to be entered as lowercase to be accepted.

This is the same as the Grant Type, which is a fixed value of ‘client_credentials’ all in lowercase.

Final detail we need is the ‘client_id’ and the ‘client_secret’, but what I found is what we really need is the API Key and the API Secret that is managed in your Fortify portal. If you sign in to your portal, for the Data Centre and product I have access to, you can navigate to the ‘Administration’ then ‘Settings’ and finally ‘API’. From this section you can create the API details with the required set of permissions. Note that the permission are changeable post setting this up so you do not need to commit yet. You should then get all the details required for these two parameters where client_id = API Key and client_secret = API Secret.

Your details in PowerShell should look like this:

$body = @{
scope = "api-tenant"
grant_type = "client_credentials"
client_id = "a1aa1111-11a1-1111-aaa1-aa1a11a1aaaa"
client_secret = "AAxAbAA1AAdrAA1AAAkyAAAwAAArA11uAzArA1A11"
}

From there we can do a simple ‘Invoke-RestMethod’ using PowerShell, with a key things to note. It is that the content type is ‘application/x-www-form-urlencoded’, without this you will keep getting an error saying the ‘Grant Type’ is not valid. With this as well you will notice as above the body is not in JSON, but are formatted as Parameters in the body of the request.

Below is the full example of the request using PowerShell, which I have also included the requests to set the default proxy so if you are requesting behind a proxy, this should still work.

## Set Proxy

[System.Net.WebRequest]::DefaultWebProxy = [System.Net.WebRequest]::GetSystemWebProxy()

[System.Net.WebRequest]::DefaultWebProxy.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials

## Create Details

$uri = “https://api.emea.fortify.com/oauth/token”

$body = @{

scope = “api-tenant”

grant_type = “client_credentials”

client_id = “a1aa1111-11a1-1111-aaa1-aa1a11a1aaaa”

client_secret = “AAxAbAA1AAdrAA1AAAkyAAAwAAArA11uAzArA1A11”

}

## Request

$response = Invoke-RestMethod -ContentType “application/x-www-form-urlencoded” -Uri $uri -Method POST -Body $body -UseBasicParsing

## Response

Write-Host $response

Setting Bearer tokens in PowerShell

This is a quick post to put out how to set a Bearer Token using PowerShell for Invoke-RestMethod and Invoke-WebRequest as it was something I could not find a clear explaining post.

This is simply set in the headers of the request as below, where ‘$bearer_token’ is the token’s variable. I have put it in the ‘$headers’ variable, which is then used in the Invoke-RestMethod.

$headers = @{Authorization = "Bearer $bearer_token"}
$response = Invoke-RestMethod -ContentType "$contentType" -Uri $url -Method $method -Headers $headers -UseBasicParsing