Returning Complex Objects from Terraform External Data Sources

Terraform’s external data source is a useful escape hatch when you need to call out to a script to fetch data that isn’t available via a provider. However, it comes with a sharp edge that often catches people out:

Terraform expects the result to be a flat map of string → string values.

If your script returns a nested object, Terraform will fail with a fairly unhelpful error message.

In this post, we’ll look at why this happens and how to safely return complex data by flattening it into a Terraform‑friendly JSON object using PowerShell, but this can be used with any of the languages.

The Problem: Nested Objects Don’t Work

The Terraform external data source expects your script to:

• Read JSON from stdin
• Output JSON to stdout
• Return a flat object where all values are strings

This means nested objects or arrays will break the provider.

Example: What Doesn’t Work

Imagine a PowerShell script that returns a structured object:

result = @{
    subscription = @{
        id   = "12345"
        name = "prod-subscription"
    }
    environment = "prod"
}

$result | ConvertTo-Json

Terraform will reject this because subscription is an object, not a string.

Terraform-Safe Output Shape

Terraform expects something closer to this:

{
  "subscription_id": "12345",
  "subscription_name": "prod-subscription",
  "environment": "prod"
}

The key rule is simple:

Flatten your object before emitting JSON.

Flattening Objects in PowerShell

Here’s a simple pattern for flattening a nested PowerShell object into a Terraform‑compatible structure.

Example PowerShell Script

# Read Terraform query input
$query = [Console]::In.ReadLine() | ConvertFrom-Json

# Simulate a complex object
$subscription = @{
    id   = "12345"
    name = "prod-subscription"
    tags = @{
        owner = "platform"
        cost  = "infra"
    }
}

# Flatten the object
$result = @{
    subscription_id   = $subscription.id
    subscription_name = $subscription.name
    environment       = $query.environment
    tag_owner         = $subscription.tags.owner
    tag_cost          = $subscription.tags.cost
}

# Output flat JSON
$result | ConvertTo-Json -Compress

A few important details:

• Use :In.ReadLine() instead of Read-Host to avoid polluting stdout
• Ensure every value is a string
• Use -Compress to avoid formatting noise

Calling This from Terraform

Your Terraform data source would look something like this:

data "external" "subscription" {
  program = [
    "pwsh",
    "${path.module}/get-subscription.ps1"
  ]

  query = {
    environment = "prod"
  }
}

You can then safely consume the results:

output "subscription_id" {
  value = data.external.subscription.result.subscription_id
}

When to Use This Pattern

This approach works well when:

• You need to integrate with REST APIs or internal tooling
• A Terraform provider doesn’t exist
• The data is read‑only and deterministic

If you find yourself returning increasingly complex data structures, it’s usually a sign that a proper Terraform provider would be a better long‑term solution.

Final Thoughts

Terraform’s external data source is powerful, but strict. By flattening your PowerShell output into a simple string‑based map, you can avoid runtime errors and keep your configurations clean and predictable.

This pattern has saved me more than once when integrating Terraform with bespoke systems—and hopefully it saves you a bit of debugging time too.