When developing some resource in Terraform you develop a large complex map or list of entries. This can become hard to manage, difficult to read and worst to maintain. An easier method is to convert these items into a Comma Separated Values (CSV) file. This will condense something that could be 100’s of lines down to 10’s of lines. In this example I will create an Azure Network Security Group with multiple rules to compare the difference. This of course can be utilized in various forms on any Terraform resource.
To show an example of the rules if we was to use Terraform variables, below I have 4 rules that we would configure. This is just a small subset of an example, as there could be many more rules, but you can see with just these it is 46 lines long.
nsg_rules = [
{
name = "ssh"
priority = 100
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "22"
source_address_prefix = "VirtualNetwork"
destination_address_prefix = "*"
},
{
name = "rdp"
priority = 101
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "3389"
source_address_prefix = "VirtualNetwork"
destination_address_prefix = "*"
},
{
name = "sql"
priority = 102
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_range = "1433"
source_address_prefix = "SqlManagement"
destination_address_prefix = "192.168.2.0/24"
},
{
name = "http-https"
priority = 201
direction = "Inbound"
access = "Allow"
protocol = "Tcp"
source_port_range = "*"
destination_port_ranges = [80, 443]
source_address_prefixes = ["1.2.3.4/32", "5.6.7.8/32"]
destination_address_prefix = "AzureLoadBalancer"
}
]
Now instead this can be converted down to a CSV, that is only 5 lines long and managed in a dedicated file. If using VS Code, I would also recommend Rainbow CSV that will highlight each column to make it even easier to maintain.
As you can see, we have just put the property names as the column and then the values associated below that. Any array of items need to be comma delimited with double quotes surrounding the value.
name,priority,direction,access,protocol,source_port_range,destination_port_range,source_address_prefix,destination_address_prefix
ssh,100,Inbound,Allow,Tcp,*,22,VirtualNetwork,*
rdp,101,Inbound,Allow,Tcp,*,3389,VirtualNetwork,*
sql,102,Inbound,Allow,Tcp,*,1433, SqlManagement,192.168.2.0/24
http-https,201,Inbound,Allow,Tcp,*,"80,443","1.2.3.4/32, 5.6.7.8/32",AzureLoadBalancer
We can then consume the file data into a local Terraform variable, can convert it from CSV to a list of items using the Terraform Function csvdecode. The data is converted from the CSV to the same format as the Terraform Variable example above.
The example below creates the Azure Resource Group, Network Security Groups and then the code for importing the CSV data. This data is used to loop through the list in a map format.
resource "azurerm_resource_group" "this" {
name = "example-rg"
location = "uksouth"
}
resource "azurerm_network_security_group" "this" {
name = "example-nsg"
location = "uksouth"
resource_group_name = azurerm_resource_group.this.name
}
locals {
csv_data = file("${path.module}/nsg_rules.csv")
nsg_rules = csvdecode(local.csv_data)
}
resource "azurerm_network_security_rule" "this" {
for_each = { for nsg_rule in local.nsg_rules : nsg_rule.name => nsg_rule }
resource_group_name = azurerm_resource_group.this.name
network_security_group_name = azurerm_network_security_group.this.name
name = each.key
direction = each.value.direction
access = each.value.access
priority = each.value.priority
protocol = each.value.protocol
source_port_range = each.value.source_port_range
destination_port_range = each.value.destination_port_range
source_address_prefix = each.value.source_address_prefix
destination_address_prefix = each.value.destination_address_prefix
}
You can then manage each NSG or other collection in dedicated files, but what I have found in some cases is this can get complex as well. If you for instance have multiple NSG’s then you will be multiple files, plus if you split them by environment you could have even more. Further to that they might even share some of the rules per environment and NSG.
For this we can extend the CSV for a filter, by adding the environment and NSG as a new column. In the environment you can have the environment name or * for all environments. You can then do the same for the NSG name, which for this example I am using hub and spoke.
name,priority,direction,access,protocol,source_port_range,destination_port_range,source_address_prefix,destination_address_prefix,environment,nsg_name
ssh,100,Inbound,Allow,Tcp,*,22,VirtualNetwork,*,*,hub
rdp,101,Inbound,Allow,Tcp,*,3389,VirtualNetwork,*,*,hub
sql,102,Inbound,Allow,Tcp,*,1433, SqlManagement,192.168.2.0/24,dev,spoke
http-https,201,Inbound,Allow,Tcp,*,"80,443","1.2.3.4/32, 5.6.7.8/32",AzureLoadBalancer,*,spoke
sql,102,Inbound,Allow,Tcp,*,1433, SqlManagement,192.168.3.0/24,test,spoke
Now with these properties added to the object of each item, we can filter which we want to apply on which NSG. I have created a local variable for the NSG names and the current environment, but these can easily be added as variables. I then loop each NSG name to create the NSG.
resource "azurerm_network_security_group" "this" {
for_each = { for nsg in local.nsg_names : nsg => nsg }
name = "example-${each.key}-nsg"
location = "uksouth"
resource_group_name = azurerm_resource_group.this.name
}
locals {
nsg_names = ["hub", "spoke"]
current_environment = "dev"
csv_data = file("${path.module}/nsg_rules.csv")
nsg_rules = csvdecode(local.csv_data)
}
We can then use a complex loop, to go through each NSG and for each NSG we go through each of the rules. There is then a filter on the rules loop, that will only add the item if the environment name is the same as the local or ‘*’ and the NSG key is the same as the one in the loop or ‘*’.
resource "azurerm_network_security_rule" "this" {
for_each = {
for item in flatten([
for nsg in local.nsg_names : [
for nsg_rule in local.nsg_rules : {
nsg_rule = nsg_rule
nsg = azurerm_network_security_group.this[nsg]
}
if((nsg_rule.environment == "*" || nsg_rule.environment == local.current_environment) && (nsg_name == "*" || nsg_name == "spoke"))
]
]) : "${item.nsg.name}_${item.nsg_rule.name}" => item
}
resource_group_name = azurerm_resource_group.this.name
network_security_group_name = each.value.nsg.name
name = each.value.nsg_rule
direction = each.value.nsg_rule.direction
access = each.value.nsg_rule.access
priority = each.value.nsg_rule.priority
protocol = each.value.nsg_rule.protocol
source_port_range = each.value.nsg_rule.source_port_range
destination_port_range = each.value.nsg_rule.destination_port_range
source_address_prefix = each.value.nsg_rule.source_address_prefix
destination_address_prefix = each.value.nsg_rule.destination_address_prefix
}
This can be expanded for more NSG’s or for other resources. I think this can make managing complex rules or list of items for resources much easier but keeps it all with source control and Terraform.
PR Code 