Full Documented ArgoCD Example

[Kaszanas]

Description

I have been fighting with a proper setup of ArgoCD for a better part of the day. I have still not reached the step where I would be able to log-in to ArgoCD running on my cluster.

The documentation is fragmented and the deployment path for ArgoCD is partly explained in README. Unfortunately I have no clue what are the best practices with this project and how it can interface with ArgoCD.

Examples include installing via Helm, while some discussion threads indicate that it is better to use Kustomize.

I think that @nikolaszimmermann initially proposed documenting such a use case.

Is there anyone knowledgeable on such a setup?

[Giuliopime]

I have a minimal example in my repo: https://github.com/Giuliopime/gport?tab=readme-ov-file#usage-1
disclaimer: I haven't tested it yet

[nikolaszimmermann]

Sorry for the late reply, we install ArgoCD using Helm releases by now:

The main.tf starts with this:

provider "hcloud" {
  token = var.hcloud_token
}

provider "kubernetes" {
  host                   = module.kube_hetzner.kubeconfig_data.host
  client_certificate     = module.kube_hetzner.kubeconfig_data.client_certificate
  client_key             = module.kube_hetzner.kubeconfig_data.client_key
  cluster_ca_certificate = module.kube_hetzner.kubeconfig_data.cluster_ca_certificate
}

provider "helm" {
  kubernetes = {
    host                   = module.kube_hetzner.kubeconfig_data.host
    client_certificate     = module.kube_hetzner.kubeconfig_data.client_certificate
    client_key             = module.kube_hetzner.kubeconfig_data.client_key
    cluster_ca_certificate = module.kube_hetzner.kubeconfig_data.cluster_ca_certificate
  }
}

module "kube_hetzner" {
  providers = {
    hcloud = hcloud
  }
  hcloud_token = var.hcloud_token

  # Then fill or edit the below values. Only the first values starting with a * are obligatory; the rest can remain with their default values, or you
  # could adapt them to your needs.
...

Besides the main.tf file, there's an argocd.tf file:

resource "helm_release" "argocd" {
  name = "argocd"

  repository = "https://argoproj.github.io/argo-helm"
  chart      = "argo-cd"
  version    = "8.2.5"

  namespace        = "argocd"
  create_namespace = true

  values = [
    yamlencode({
      configs = {
        repositories = {
          igalia = {
            name      = "SomeCustomRepo"
            url       = "ghcr.io/path/to/custom/charts"
            type      = "helm"
            username  = var.ghcr_registry_username
            password  = var.ghcr_registry_password
            enableOCI = "true"
          }
        }
      }
    })
  ]

  depends_on = [module.kube_hetzner]
}

data "kubernetes_secret" "argocd_admin_password" {
  metadata {
    name      = "argocd-initial-admin-secret"
    namespace = helm_release.argocd.namespace
  }
}

That's enough to deploy ArgoCD using Helm. Hope that helps.
We used to use kustomize commands deploying the yamls directly, but this has proven to be easier to maintain.

[Kaszanas]

Sorry for the late reply, we install ArgoCD using Helm releases by now:

The main.tf starts with this:

provider "hcloud" {
  token = var.hcloud_token
}

provider "kubernetes" {
  host                   = module.kube_hetzner.kubeconfig_data.host
  client_certificate     = module.kube_hetzner.kubeconfig_data.client_certificate
  client_key             = module.kube_hetzner.kubeconfig_data.client_key
  cluster_ca_certificate = module.kube_hetzner.kubeconfig_data.cluster_ca_certificate
}

provider "helm" {
  kubernetes = {
    host                   = module.kube_hetzner.kubeconfig_data.host
    client_certificate     = module.kube_hetzner.kubeconfig_data.client_certificate
    client_key             = module.kube_hetzner.kubeconfig_data.client_key
    cluster_ca_certificate = module.kube_hetzner.kubeconfig_data.cluster_ca_certificate
  }
}

module "kube_hetzner" {
  providers = {
    hcloud = hcloud
  }
  hcloud_token = var.hcloud_token

  # Then fill or edit the below values. Only the first values starting with a * are obligatory; the rest can remain with their default values, or you
  # could adapt them to your needs.
...

Besides the main.tf file, there's an argocd.tf file:

resource "helm_release" "argocd" {
  name = "argocd"

  repository = "https://argoproj.github.io/argo-helm"
  chart      = "argo-cd"
  version    = "8.2.5"

  namespace        = "argocd"
  create_namespace = true

  values = [
    yamlencode({
      configs = {
        repositories = {
          igalia = {
            name      = "SomeCustomRepo"
            url       = "ghcr.io/path/to/custom/charts"
            type      = "helm"
            username  = var.ghcr_registry_username
            password  = var.ghcr_registry_password
            enableOCI = "true"
          }
        }
      }
    })
  ]

  depends_on = [module.kube_hetzner]
}

data "kubernetes_secret" "argocd_admin_password" {
  metadata {
    name      = "argocd-initial-admin-secret"
    namespace = helm_release.argocd.namespace
  }
}

That's enough to deploy ArgoCD using Helm. Hope that helps. We used to use kustomize commands deploying the yamls directly, but this has proven to be easier to maintain.

That's great to hear! I hope to construct a repo where I can reproduce this. So far I have been applying only the created kube.tf. I think we have discussed seeing a full example at some point and how you manage to add repos / remove repos and how to properly migrate stuff when using such approach (#1068). What is the way you instantiate databases/storage/persistence? Sorry if some of my questions are misguided, I am not very experienced in infrastructure, and yet I am attempting to start and deploy some of my services as one-man-army.

[Kaszanas]

Also equally important, how do you handle ArgoCD updates?

[mysticaltech]

@Kaszanas Updates is up to you, especially for custom helm packages, we focus on the cluster itself.

@nikolaszimmermann @Giuliopime Interesting stuff folks, thanks for sharing.

@Kaszanas From all that has been shared here, a PR with an example targeting the examples folder would be welcome.

[vsalomaki]

@nikolaszimmermann I would be also interested to know what kind of challenges you've had, and also with with this;

We used to use kustomize commands deploying the yamls directly, but this has proven to be easier to maintain.

I have stumbled more or less on the same subject while testing my setup and re-creating the cluster more than a few times.
Previously I attempted to create a similar setup you are referring to, but using a kubeconfig-file as an input to provider.helm. It didn't quite work as planned, perhaps the way you have expressed would be good way to go.

Anyway, facing challenges with the "multilevel" installation of Helms, such as external-secrets, I created an approach of applying Kustomization in kube-hetzner using a map of "Kustomization-sets". There is a PR of that: #1924
The setup does work (in my setup! 😅 ) but would appreciate if you could drop some thoughts on the concept.

[Kaszanas]

@nikolaszimmermann I would be also interested to know what kind of challenges you've had, and also with with this;

We used to use kustomize commands deploying the yamls directly, but this has proven to be easier to maintain.

I have stumbled more or less on the same subject while testing my setup and re-creating the cluster more than a few times. Previously I attempted to create a similar setup you are referring to, but using a kubeconfig-file as an input to provider.helm. It didn't quite work as planned, perhaps the way you have expressed would be good way to go.

Anyway, facing challenges with the "multilevel" installation of Helms, such as external-secrets, I created an approach of applying Kustomization in kube-hetzner using a map of "Kustomization-sets". There is a PR of that: #1924 The setup does work (in my setup! 😅 ) but would appreciate if you could drop some thoughts on the concept.

See this is the issue for me, there are multiple ways of setting it up, and it is hard to forsee the issues one might arrive at in the later stages of deployment.

If you got some time I'd love to look at a repository and a workflow describing the different ways of handling ArgoCD (a form of a tutorial).

I was initially aiming at one node server (because I cannot break the bank on my initial setup), but later I'd like to be able to scale and set everything up for success. It is also hard to debug all of the issues considering that I have limited DevOPS knowledge. Even worse, I'd like to host some "internal" services which only I should be able to access (ArgoCD UI in itself should not be open to public), and other services which I'd like to be public-facing. Replicating this locally would be great to just test the workflows and reproduce testing scenarios and so on.

Your PR is interesting! But I am still missing a full example that presents a "Hello World!" service (Frontend, backend, database, load-balancer, local/dev/prod separation) using all of the goodness of this repository.

[Kaszanas]

@nikolaszimmermann Let me know if this is something that you are aware of ;)

[nikolaszimmermann]

@Kaszanas didn't follow this dud to holidays - what exactly are you looking for? 😇

[Kaszanas]

@Kaszanas didn't follow this dud to holidays - what exactly are you looking for? 😇

@nikolaszimmermann

The following (on a single node cluster)

0. Storing secrets for configuring the apps (typically using env variables passed to the containers, but I am not sure how that works with k3s. And how to populate them.

1. Exposing the ArgoCD UI safely to look if the application was successfully provisioned, setting different users etc.

2. Having Dev replica of the production release only for testing with the development team.

3. Updating Argo

And whatever else daily maintenance one could imagine 😅 overall what's best not to touch. And how to build this system in a way that in couple of years there's no hiccups.

I'll add the stuff which I've written before below in an edit. For now I'd like to try with some "Hello World" manifest examples.