Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Azure ARM template for Service bus with Topics with many Subscriptions

Hi I have an ARM template for create a ServiceBus with its topics and subscriptions. But I can only accomplish 1 topic - 1 subscription because I cannot make a nested loop to create many subscriptions per topic.

I wish I could execute a template like this:

params:

{
   "serviceBusName": "mybus",
   "topics": 
    [ 
      { 
         "topicName": "mytopic1",
         "subscriptions": [ "mysubscription1", "mysubscription2"]
      },
      { 
         "topicName": "mytopic2",
         "subscriptions": [ "mysubscription1"]
      }  
    ]
}

This is my actual template:

{
  "$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "ServiceBusNamespaceName": {
      "type": "string"
    },
    "ServiceBusSku": {
      "type": "string",
      "allowedValues": [
        "Basic",
        "Standard"
      ],
      "defaultValue": "Standard"
    },
    "ServiceBusSmallSizeTopicInMb": {
      "type": "int",
      "defaultValue": 1024
    },
    "ServiceBusMaxSizeTopicInMb": {
      "type": "int",
      "defaultValue": 1024
    },
    "Topics": {
      "type": "array"
    }
  },
  "variables": {
    "DefaultSASKeyName": "RootManageSharedAccessKey",
    "DefaultAuthRuleResourceId": "[resourceId('Microsoft.ServiceBus/namespaces/authorizationRules', parameters('ServiceBusNamespaceName'), variables('DefaultSASKeyName'))]",
    "SbVersion": "2017-04-01"
  },
  "resources": [
    {
      "apiVersion": "2017-04-01",
      "name": "[parameters('ServiceBusNamespaceName')]",
      "type": "Microsoft.ServiceBus/namespaces",
      "location": "[resourceGroup().location]",
      "sku": {
        "name": "[parameters('ServiceBusSku')]"
      },
      "tags": {
        "displayName": "ServiceBus"
      }
    },
    {
      "copy": {
        "name": "topics",
        "count": "[length(parameters('Topics'))]"
      },
      "type": "Microsoft.ServiceBus/namespaces/topics",
      "name": "[concat(parameters('ServiceBusNamespaceName'), '/', parameters('Topics')[copyIndex()].topic)]",
      "apiVersion": "2017-04-01",
      "location": "[resourceGroup().location]",
      "scale": null,
      "properties": {
        "defaultMessageTimeToLive": "P1D",
        "maxSizeInMegabytes": "[parameters('ServiceBusMaxSizeTopicInMb')]",
        "requiresDuplicateDetection": false,
        "duplicateDetectionHistoryTimeWindow": "PT10M",
        "enableBatchedOperations": true,
        "status": "Active",
        "supportOrdering": true,
        "autoDeleteOnIdle": "P10675199DT2H48M5.4775807S",
        "enablePartitioning": false,
        "enableExpress": false
      },
      "dependsOn": [
        "[resourceId('Microsoft.ServiceBus/namespaces', parameters('ServiceBusNamespaceName'))]"
      ],
      "resources": [
        {
          "apiVersion": "[variables('sbVersion')]",
          "name": "[parameters('Topics')[copyIndex()].subscription]",
          "type": "Subscriptions",
          "dependsOn": [
            "[parameters('Topics')[copyIndex()].topic]"
          ],
          "properties": {
            "lockDuration": "PT1M",
            "requiresSession": "false",
            "defaultMessageTimeToLive": "P7D",
            "deadLetteringOnMessageExpiration": "false",
            "maxDeliveryCount": "2",
            "enableBatchedOperations": "true",
            "autoDeleteOnIdle": "P7D"
          }
        }
      ]
    }
  ],
  "outputs": {
    "NamespaceDefaultConnectionString": {
      "type": "string",
      "value": "[listkeys(variables('DefaultAuthRuleResourceId'), variables('SbVersion')).primaryConnectionString]"
    },
    "DefaultSharedAccessPolicyPrimaryKey": {
      "type": "string",
      "value": "[listkeys(variables('DefaultAuthRuleResourceId'), variables('SbVersion')).primaryKey]"
    }
  }
}

And an example of the params json for the actual template:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "ServiceBusNamespaceName": {
            "value": "mybus"
        },
        "ServiceBusSku": {
            "value": "Standard"
        },
        "Topics ": {
            "value": [
                {
                    "topic": "mytopic1",
                    "subscription": "mysubscription1"
                },
                {
                    "topic": "mytopic2",
                    "subscription": "mysubscription1"
                }
            ]
        }
    }
}
like image 430
Omar Amalfi Avatar asked Aug 28 '19 10:08

Omar Amalfi


People also ask

Which component of Azure allows the user to subscribe to a certain topic?

The Service Bus Explorer allows users to connect to a Service Bus namespace and administer messaging entities in an easy manner. The tool provides advanced features like import/export functionality or the ability to test topic, queues, subscriptions, relay services, notification hubs and events hubs.


3 Answers

I have managed to get it working while using the most simple topics/subscription structure and using a nested loop with sub-deployments. An important factor in this is that you use an internal scope for the sub deployment, and pass the topic you want to create the subscriptions for as parameter to the sub-deployment.

{
    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {

        // Since we don't want to create complex ARM structure, we use the most basic variable structure
        // to define the topics and subscriptions.
        // The complexity is in the ARM template.
        "topics": {
            "type": "array",
            "defaultValue":
            [
                {
                    "name": "articles",
                    "subscriptions": [
                        "diagnostics",
                        "orders"
                    ]
                },
                {
                    "name": "customers",
                    "subscriptions": [
                        "diagnostics",
                        "orders"
                    ]
                },
                {
                    "name": "orders",
                    "subscriptions": [
                        "diagnostics",
                        "orders"
                    ]
                }
            ]
        }
    },
    "variables": {
        "baseName": "[resourceGroup().name]",
        "serviceBusName": "[variables('baseName')]",
        "topics": "[parameters('topics')]"
    },
    "resources": [
        {
            //
            // Create the Service-bus itself.
            //
            "name": "[variables('serviceBusName')]",
            "type": "Microsoft.ServiceBus/namespaces",
            "apiVersion": "2018-01-01-preview",
            "location": "[resourceGroup().location]",
            "sku": { "name": "Standard" },
            "kind": "Messaging",
            "properties": {},
            "resources": [
            ]
        },
        {
            //
            // Now we are going to create the topics. This is a loop throught the topics variable.
            //
            "copy": {
                "name": "topics-loop",
                "count": "[length(variables('topics'))]"
            },
            "name": "[concat(variables('serviceBusName'),'/', variables('topics')[copyIndex()].name)]",
            "type": "Microsoft.ServiceBus/namespaces/topics",
            "apiVersion": "2017-04-01",
            "dependsOn": [
                "[concat('Microsoft.ServiceBus/Namespaces/', variables('serviceBusName'))]"
            ],
            "properties": {}
        },
        {
            //
            // The following structure looks rather complex. Since nested loops are not supported for resources (it is for properties),
            // we create a deployment that is looped and passes the required variables as parameters to the inner deployment.
            // In the inner deployment another loop is created, and this creates the subscriptions.
            //

            "copy": {
                "name": "subscriptions-outer-loop",
                "count": "[length(variables('topics'))]"
            },

            // The name of the deployment.
            "name": "[concat(variables('serviceBusName'),'-',variables('topics')[copyIndex()].name,'-subscriptions')]",
            "type": "Microsoft.Resources/deployments",
            "apiVersion": "2019-10-01",
            "properties": {
                "mode": "Incremental",

                // Set the scope to Inner. Everything from the outer deployment that is needed in the inner template,
                // can be moved via parameters. You cannot do nested loops if you let te scope be outer.
                "expressionEvaluationOptions": {
                    "scope": "inner"
                },

                // The following parameters are defined by the inner template.
                "parameters": {
                    "serviceBusName": {
                        "value": "[variables('serviceBusName')]"
                    },

                    // The current topic, this is an important one, 
                    // it communicates the current topic to the inner template
                    "currentTopic": {
                        "value": "[variables('topics')[copyIndex()]]"
                    }
                },
                "template": {
                    "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
                    "contentVersion": "1.0.0.0",
                    "parameters": {
                        // Define the parameters that are set by the outer template.
                        "serviceBusName": { "type": "string" },
                        "currentTopic": { "type": "object" }
                    },
                    "resources": [
                        {
                            // The inner loop. This will create the subscriptions.
                            "copy": {
                                "name": "subscriptions-inner-loop",
                                "count": "[length(parameters('currentTopic').subscriptions)]"
                            },
                            "name": "[concat(
                                    parameters('serviceBusName'),'/', 
                                    parameters('currentTopic').name, '/',
                                    parameters('currentTopic').subscriptions[copyIndex()])]",
                            "type": "Microsoft.ServiceBus/namespaces/topics/subscriptions",
                            "apiVersion": "2017-04-01",
                            "properties": {}
                        }
                    ]
                }
            },

            //This depends on the outer loop.
            "dependsOn": [ "topics-loop" ]
        }
    ]
}
like image 133
Robin557 Avatar answered Oct 19 '22 08:10

Robin557


In general, there are two ways to do something like this. You can restructure your code so that the subscriptions are top level resources. Or you use the named variant of copyIndex to achieve nested loops. Both variants can be seen in this blog post and the comment for it.

Azure ARM Templates – Are nested loops possible?

However, for your case, the only option is to make the subscriptions a top level resource. See aka.ms/arm-copy/#looping-on-a-nested-resource for more details.

This is the full example taken from the blog post mentioned above.

{
    "$schema": "http://schema.management.azure.com/schemas/2014-04-01-preview/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "serviceBusNamespaceName": {
            "type": "string",
            "metadata": {
                "description": "Name of the Service Bus namespace"
            }
        },
        "topics":{
            "type": "array",
            "metadata": {
                "description": "List of topics"
            }
        },
        "subscriptions":{
            "type": "array",
            "metadata": {
                "description": "List of subscriptions"
            }
        }

    },
    "variables": {},
    "resources": [
        {
            "type": "Microsoft.ServiceBus/namespaces",
            "sku": {
                "name": "Standard"
            },
            "name": "[parameters('serviceBusNamespaceName')]",
            "apiVersion": "2017-04-01",
            "location": "[resourceGroup().location]",
            "properties": {}
        },
        {
            "type": "Microsoft.ServiceBus/namespaces/topics",
            "name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('topics')[copyIndex()])]",
            "apiVersion": "2017-04-01",
            "location": "[resourceGroup().location]",
            "copy": {
                "name": "topicLoop",
                "count": "[length(parameters('topics'))]"
            },
            "properties": {},
            "dependsOn": [
                "[concat('Microsoft.ServiceBus/namespaces/', parameters('serviceBusNamespaceName'))]"
            ]
        },
        {
            "type": "Microsoft.ServiceBus/namespaces/topics/subscriptions",
            "name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('subscriptions')[copyIndex()].topic, '/', parameters('subscriptions')[copyIndex()].subscription)]",
            "apiVersion": "2017-04-01",
            "location": "[resourceGroup().location]",
            "copy": {
                "name": "subscriptionLoop",
                "count": "[length(parameters('subscriptions'))]"
            },
            "properties": {},
            "dependsOn": [
                "topicLoop"
            ]
        }
    ]
}
{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
      "serviceBusNamespaceName": {
        "value": "rjtestsbnmspace"
      },
      "topics": {
        "value": ["topic1", "topic2"]
      },
      "subscriptions": {
        "value": [{
          "topic": "topic1",
          "subscription": "subscription1"
          },
          {
            "topic": "topic1",
            "subscription": "subscription2"
          },
          {
            "topic": "topic2",
            "subscription": "subscription3"
          }
        ]
      }
    }
  }

This example for VMs uses named copyIndex - it works when the nested loop is not for a resource itself.

{
  "name": "[concat(parameters('vmName'), padLeft(copyIndex(1), 2, '0'))]",
  "type": "Microsoft.Compute/virtualMachines",
  "copy": {
    "name": "vmLoop",
    "count": "[parameters('vmCount')]"
  },
  "properties": {
    "osProfile": {
      "computerName": "[concat(parameters('vmName'), padLeft(copyIndex(1), 2, '0'))]"
    },
    "hardwareProfile": {
      "vmSize": "[parameters('vmSize')]"
    },
    "storageProfile": {
      "osDisk": {
        "name": "[concat(parameters('vmName'), padLeft(copyIndex(1), 2, '0'),'_OSDisk')]",
        "createOption": "FromImage",
        "managedDisk": {
          "storageAccountType": "[parameters('vmDiskType')]"
        }
      },
      "copy": [
        {
          "name": "dataDisks",
          "count": "[parameters('dataDiskCount')]",
          "input": {
            "caching": "[parameters('dataDiskCaching')]",
            "name": "[concat(parameters('vmName'), padLeft(copyIndex('vmLoop', 1), 2, '0'), '-dataDisk', padLeft(copyIndex('dataDisks'), 2, '0'))]",
            "lun": "[copyIndex('dataDisks')]",
            "createOption": "Empty",
            "diskSizeGB": "[parameters('dataDiskSize')]",
            "managedDisk": {
              "storageAccountType": "[parameters('vmDiskType')]"
            }
          }
        }
      ]
    }
  }
}
like image 24
Alex AIT Avatar answered Oct 19 '22 07:10

Alex AIT


you can achieve that using nested deployment, high level view would be like this:

{
    "apiVersion": "2017-05-10",
    "name": "[concat('topicLoop-', copyIndex())]",
    "type": "Microsoft.Resources/deployments",
    "copy": {
        "name": "topicLoop",
        "count": "[length(parameters('topics'))]"
    },
    "dependsOn": [
       // this should on the service bus resource
    ]
    "properties": {
        "mode": "Incremental",
        "templateLink": {
            "uri": "nested_template_link"
        },
        "parameters": {
            "iteration": {
                "value": "[parameters('topics')[copyIndex()]]"
            }
        }
    }
},

and then inside your nested template you would have something like this:

{
    "type": "Microsoft.ServiceBus/namespaces/topics",
    "name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('iteration').topicName)]",
    "apiVersion": "2017-04-01",
    "location": "[resourceGroup().location]"
},
{
    "type": "Microsoft.ServiceBus/namespaces/topics/subscriptions",
    "name": "[concat(parameters('serviceBusNamespaceName'), '/', parameters('iteration').topicName, '/', parameters('iteration').subscriptions[copyIndex()])]",
    "apiVersion": "2017-04-01",
    "location": "[resourceGroup().location]",
    "copy": {
        "name": "subscriptionLoop",
        "count": "[length(parameters('iteration').subscriptions)]"
    },
    "properties": {},
    "dependsOn": [
        "[concat(parameters('serviceBusNamespaceName'), '/', parameters('iteration').topicName)"]
    ]
}

this would allow you to keep existing parameter structure and make the template more flexible, the other answers solution is not really maintainable (you'd have to edit 2 parameters, instead of just one, which is a big deal, tbh)

like image 3
4c74356b41 Avatar answered Oct 19 '22 07:10

4c74356b41