IP not available

[Foxite]

We are deploying a new cluster using our Terrakube server. Here is my module configuration, truncated of line breaks and comments:

terraform {
  required_version = ">= 1.5.0"
  required_providers {
    hcloud = {
      source  = "hetznercloud/hcloud"
      version = ">= 1.51.0"
    }
  }
}

provider "hcloud" {
  token = var.hcloud_token
}

output "kubeconfig" {
  value     = module.kube-hetzner.kubeconfig
  sensitive = true
}

variable "hcloud_token" {
  sensitive = true
  default   = ""
}

variable "hcloud_ssh_public_key" {
  sensitive = false
  default   = ""
}

variable "hcloud_ssh_private_key" {
  sensitive = true
  default   = ""
}

module "kube-hetzner" {
  providers = {
    hcloud = hcloud
  }
  hcloud_token = var.hcloud_token
  source = "kube-hetzner/kube-hetzner/hcloud"
  cluster_name = "REDACTED"
  use_cluster_name_in_node_name = true
  ssh_port = 2222
  ssh_public_key = var.hcloud_ssh_public_key
  ssh_private_key = var.hcloud_ssh_private_key
  network_region = "eu-central"
  automatically_upgrade_os = false
  automatically_upgrade_k3s = true
  system_upgrade_use_drain = true
  system_upgrade_enable_eviction = false
  allow_scheduling_on_control_plane = false
  control_plane_nodepools = [
    {
      name         = "control-plane-fsn1"
      server_type  = "cpx21"
      location     = "fsn1"
      labels       = []
      taints       = []
      count        = 1
      backups      = true
      disable_ipv4 = true
      disable_ipv6 = true
    }
  ]
  agent_nodepools = []
  enable_wireguard = false
  load_balancer_type     = "lb11"
  load_balancer_location = "fsn1"
  nat_router = {
    server_type = "cpx21"
    location    = "fsn1"
    enable_sudo = false
    labels      = {}
  }
  autoscaler_nodepools = [
    {
      name        = "worker"
      server_type = "cx11"
      location    = "fsn1"
      min_nodes   = 1
      max_nodes   = 3
    }
  ]
  autoscaler_disable_ipv4 = true
  autoscaler_disable_ipv6 = true
  enable_delete_protection = {
    floating_ip   = false
    load_balancer = false
    volume        = true
  }
  disable_hetzner_csi = false
  ingress_controller = "nginx"
  enable_metrics_server = true
  restrict_outbound_traffic = false
  cni_plugin = "flannel"
  block_icmp_ping_in = true
  enable_cert_manager = true
  dns_servers = [
    "1.1.1.1",
    "1.0.0.1",
    "2606:4700:4700::1111",
  ]
  use_control_plane_lb = true
  control_plane_lb_type = "lb11"
  control_plane_lb_enable_public_interface = false
  create_kubeconfig = false
  create_kustomization = false
  kured_version = "1.19.0"
}

When deploying, we get these errors:

module.kube-hetzner.null_resource.kustomization: Still creating... [6m30s elapsed]
module.kube-hetzner.null_resource.kustomization (remote-exec): Waiting for load-balancer to get an IP...
module.kube-hetzner.null_resource.kustomization (remote-exec): Waiting for load-balancer to get an IP...
���
��� Error: IP not available (ip_not_available, 36b87b7432a0fa3ededbfa27a02d7016)
��� 
���   with module.kube-hetzner.hcloud_load_balancer_network.cluster[0],
���   on .terraform/modules/kube-hetzner/init.tf line 22, in resource "hcloud_load_balancer_network" "cluster":
���   22: resource "hcloud_load_balancer_network" "cluster" {
��� 
���
���
��� Error: remote-exec provisioner error
��� 
���   with module.kube-hetzner.null_resource.kustomization,
���   on .terraform/modules/kube-hetzner/init.tf line 426, in resource "null_resource" "kustomization":
���  426:   provisioner "remote-exec" {
��� 
��� error executing "/tmp/terraform_76873474.sh": Process exited with status
��� 124
���

At this time, I'm more interested in fixing the ip_not_available error, but I'll move on to the other one next.

I've searched this repo for existing issues about the ip_not_available error and the ones I found are either bugs that have been fixed, or unnecessary manual assignment of IP addresses, which I don't seem to be doing.

Any help with this problem would be greatly appreciated!

Update: following #1886 I've fixed the Kustomize error, but I'm still getting the ip_not_available error, only after a timeout of 6:30 minutes.

[Foxite]

I was able to fix this issue by adding an agent nodepool. I only had an autoscale nodepool, but it seems that this will cause Terraform to try and create a cluster without any nodes, presumably leaving the autoscale nodes to be created by k3s.

For some reason this causes the Hetzner load balancer to be unable to acquire an IP address. Looking at the load balancer in Hetzner in this situation, its status is gray as it has no targets. Adding a non-autoscaling node will add a target and everything goes through.