Google deployment manager fails when the MIG (managed instance group) and the load balancer are in the same config file

Question

I'm working on IaC (Goole Deployment Manager - Python and YAML) and tinkering with Google's external load balancer. As part of the PoC I created:

1. An instance template based on ubuntu with startup script installing apache2 and an index.html file.
2. Create an instance group based on the above template.
3. Place an external HTTP load balancer in front of the instance group.
4. A network and a firewall rule to make debugging easier.

I heavily borrowed from Cloud Foundation Toolkit, and I'm showing only the config part of the code below. I used Python for templating.

imports:
- path: ../network/vpc/vpc_network.py
  name: vpc_network.py
- path: ../network/firewall/firewall_rule.py
  name: firewall_rule.py
- path: ../compute/instance_template/instance_template.py
  name: instance_template.py
- path: ../compute/health_checks/health_check.py
  name: health_check.py
- path: ../compute/instance_group/mananged/instance_group.py
  name: instance_group.py
- path: ../network_services/load_balancing/external_loadBalancers/external_load_balancer.py
  name: external_load_balancer.py

resources:
- name: demo-firewall-rules-1
  type: firewall_rule.py
  properties:
    rules:
    - name: "allow-ssh-for-all"
      description: "tcp firewall enable from all" 
      network: $(ref.net-10-69-16.network)
      priority: 1000
      action: "allow"
      direction: "INGRESS"
      sourceRanges: ['0.0.0.0/0']
      ipProtocol: "tcp"
      ipPorts: ["22"]
    - name: "test-health-check-for-ig"
      description: "enable health check to work on the project" 
      network: $(ref.net-10-69-16.network)
      priority: 1000
      action: "allow"
      direction: "INGRESS"
      sourceRanges: ['130.211.0.0/22', '35.191.0.0/16']
      ipProtocol: "tcp"
      ipPorts: ["80", "443"]
    - name: "allow-http-https-from-anywhere"
      description: "allow http and https from anywhere" 
      network: $(ref.net-10-69-16.network)
      priority: 1000
      action: "allow"
      direction: "INGRESS"
      sourceRanges: ['0.0.0.0/0']
      ipProtocol: "tcp"
      ipPorts: ["80", "443"]

- name: net-10-69-16
  type: vpc_network.py
  properties:
    subnetworks:
      - region: australia-southeast1
        cidr: 10.69.10.0/24
      - region: australia-southeast1
        cidr: 10.69.20.0/24
      - region: australia-southeast1
        cidr: 10.69.30.0/24

- name: mig-regional-1
  type: instance_group.py
  properties:
    region: australia-southeast1
    instanceTemplate: $(ref.it-demo-1.selfLink)
    targetSize: 2
    autoHealingPolicies:
    - healthCheck: $(ref.demo-http-healthcheck-MIG-1.selfLink)
      initialDelaySec: 400

- name: it-demo-1
  type: instance_template.py
  properties:
    machineType: n1-standard-1
    tags:
      items:
        - http
    disks:
    - deviceName: boot-disk-v1
      initializeParams:
        sourceImage: projects/ubuntu-os-cloud/global/images/ubuntu-2004-focal-v20201211
        diskType: pd-ssd
    networkInterfaces:
    - network: $(ref.net-10-69-16.network)
      subnetwork: $(ref.net-10-69-16.subnetworks[0])
      accessConfigs:
      - type: ONE_TO_ONE_NAT 
    metadata:
      items:
      - key: startup-script
        value: |
          sudo apt-get install -y apache2
          sudo apt-get install -y php7.0
          sudo service apache2 restart
          sudo echo "Ho Ho Ho from $HOSTNAME" > /var/www/html/index.html  

- name: demo-http-healthcheck-MIG-1
  type: health_check.py
  properties:
    type: HTTP
    checkIntervalSec: 5
    timeoutSec: 5
    unhealthyThreshold: 2
    healthyThreshold: 2
    httpHealthCheck: 
      port: 80
      requestPath: /

- name: http-elb-1
  type: external_load_balancer.py
  properties:
    portRange: 80
    backendServices:
      - resourceName: backend-service-for-http-1
        sessionAffinity: NONE
        affinityCookieTtlSec: 1000
        portName: http
        healthCheck: $(ref.demo-http-healthcheck-MIG-1.selfLink)
        backends:
          - group: $(ref.mig-regional-1.selfLink)
            balancingMode: UTILIZATION
            maxUtilization: 0.8
    urlMap:
      defaultService: backend-service-for-http-1

The problem is with the load balancer. When it is present in the same file, the deployment fails saying the managed instance group mig-regional-1 does not exist. But when I move the load balancer part out of this file and deploy the load balancer separately, (after a bit of delay) it all goes through well.

The most probable explanation is, the instance group is not ready when the load balancer is trying to reference it which is the exact situation ref.*.selfLink is supposed to handle.

This is not exactly a blocker, but it is nice to have the entire config in 1 file.

So, my questions:

1. Have any of you faced this before or am I missing something here?
2. Do you have any solution for this?

Answer 1

Your template need to explicit its dependencies.

You can have dependencies between your resources, such as when you need certain parts of your environment to exist before you can deploy other parts of the environment.

You can specify dependencies using the dependsOn option in your templates.

Seems that your External LB needs dependsOn attribute that has MIG value.

refer here to get more information about expliciting dependencies.