0.6.0 Type Checking Issue

[anthonyh-quilt]

Description

I'm having an issue with 0.6.0- I can no longer use this: ${schema.metadata.namespace}

Observed Behavior:

      message: >-
        failed to validate node "certificate": failed to type-check template
        expression "schema.metadata.namespace" at path "spec.dnsNames[0]":
        ERROR: <input>:1:16: undefined field 'namespace'
         | schema.metadata.namespace
         | ...............^
      observedGeneration: 9
      reason: InvalidResourceGraph
      status: 'False'
      type: ResourceGraphAccepted

I have tried many different ways to try and "safely" reference namespace but the new validation is very restrictive... Or I have no idea what I am doing.

Expected Behavior:
can reference instance's deployed namespace in RGD resources

Reproduction Steps (Please include ResourceGraphDefinition and Instances files):
resource graph:

apiVersion: kro.run/v1alpha1
kind: ResourceGraphDefinition
metadata:
  name: tenant-tls
spec:
  schema:
    apiVersion: v1alpha1
    kind: TenantTLS
    spec:
      domain: string | default=example.com
      issuerRef:
        name: string | default=letsencrypt
        kind: string | default=ClusterIssuer
  resources:
    - id: certificate
      template:
        apiVersion: cert-manager.io/v1
        kind: Certificate
        metadata:
          name: tenant-env-cert
          annotations:
            argocd.argoproj.io/tracking-id: ${schema.metadata.?annotations["argocd.argoproj.io/tracking-id"].orValue("")}
        spec:
          secretName: tenant-env-cert
          issuerRef:
            name: ${schema.spec.issuerRef.name}
            kind: ${schema.spec.issuerRef.kind}
          dnsNames:
            - '*.${schema.metadata.namespace}.${schema.spec.domain}'
            - '${schema.metadata.namespace}.${schema.spec.domain}'

instance:

apiVersion: kro.run/v1alpha1
kind: TenantTLS
metadata:
  name: tenant-env-cert

Note:
I'm using an ArgoCD ApplicationSet that discovers instance resources like: apps/{tenant}/*/{env}/
And it will specify the target namespace like: {tenant}-{env}

Versions:

• kro version: 0.6.0
• Kubernetes Version (kubectl version):

Client Version: v1.30.2
Server Version: v1.32.9-eks-113cf36

---

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Which option describes the most your issue?

No response

[anthonyh-quilt]

As a side note, all argocd.argoproj.io/tracking-id annotations needed to be updated to include .orValue("") to satisfy the validator (see point 5 in Kro's FAQ)

[a-hilaly]

hi @anthonyh-quilt can you try ${schema.metadata.__namespace__} looks like our dependency on apiserver type conversion does some Escaping one special fields like namespace https://github.com/kubernetes/apiserver/blob/master/pkg/cel/escaping.go#L32

[a-hilaly]

I'm still exploring our options here, very likely we'll have to upstream a fix to apiserver

[anthonyh-quilt]

@a-hilaly the graph passes validation using ${schema.metadata.__namespace__} 👍

[anthonyh-quilt]

Fixing this has revealed another type checking issue and i'm uncertain if I should create a new issue for it or not...

failed to validate node "ingressroute": type mismatch in resource
"ingressroute" at path "spec.routes[0].middlewares": expression
"schema.spec.ingress.middlewares" returns type
"list(__type_schema.spec.ingress.middlewares.@idx)" but expected
"list(object)"

where:

apiVersion: kro.run/v1alpha1
kind: ResourceGraphDefinition
metadata:
  name: stateless-service-2
spec:
  schema:
    apiVersion: v1alpha1
    kind: StatelessService2
    types:
      Middleware:
        name: string
    spec:
      # <snip>
      # Ingress configuration
      ingress:
        middlewares: '[]Middleware | default=[]'
  resources:
    # <snip>
    - id: ingressroute
      includeWhen:
        - ${schema.spec.ingress.enabled}
      template:
        # <snip>
        spec:
          entryPoints:
            - websecure
          routes:
          - match: ...
            kind: Rule
            middlewares: ${schema.spec.ingress.middlewares}  # <- this no longer passes validation
            services:
            - name: ${schema.spec.name}
              port: 80
          tls:
            secretName: tenant-env-cert

[shabbskagalwala]

what got it to work for me is

'${has(schema.spec.ingress.middlewares) ? schema.spec.ingress.middlewares : []}'

and remove the default=[]

[anthonyh-quilt]

@shabbskagalwala well i'm unsure if it was the workaround or something local but my Kro controller is now going through panic crash loops... Will verify your workaround once i'm able to calm Kro down 😅

[anthonyh-quilt]

@shabbskagalwala unfortunately your workaround did not work for me. Are you running on latest Kro? (0.6.0)

[shabbskagalwala]

@anthonyh-quilt yes, on latest kro (0.6.0). what error do you see?

[anthonyh-quilt]

@shabbskagalwala similar error as before:

failed to validate node "ingressroute": type mismatch in resource
"ingressroute" at path "spec.routes[0].middlewares": expression
"has(schema.spec.ingress.middlewares) ? schema.spec.ingress.middlewares
: []" returns type "list(__type_schema.spec.ingress.middlewares.@idx)"
but expected "list(object)"

I also made sure to remove the default=[]

middlewares: '[]Middleware'

[shabbskagalwala]

yeah try a explain

kubectl explain <kind>.spec.ingress.middlewares

I think Kro does validations and what custom types return doesn't match what kubernetes expects which is a list(object)

so i think custom types might be an issue here