How to do a patch to delete items in a list that are in a HelmRelease's values?

[zanoni23]

So I've tried a delete patch on a HelmRelease but it doesn't work because I think kustomize doesn't know about the CRD and so it can't do it - ref kubernetes-sigs/kustomize#4596
Because the result I get is a literal "$patch" in my configMap and all other items in the list are removed.
And I can't use "op: remove" with an index for items in the list, as the list is not ordered so the patch may remove another item instead.
So how would I get the patch to use openapi and point it to the HelmRelease CRD schema, so it knows how to patch it properly?
Or should I be doing this some other way?

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: infra-phase3
spec:
  sourceRef:
    kind: GitRepository
    name: flux-system
  path: ./flux/infrastructure/phase3
  patches:
    # remove object replication metrics not used in dev/stg
    - patch: |-
        apiVersion: helm.toolkit.fluxcd.io/v2
        kind: HelmRelease
        metadata:
          name: promitor-scraper
        spec:
          values:
            metrics:
              - name: azure_storage_claims_pending_bytes_for_replication
                $patch: delete
              - name: azure_storage_claims_pending_operations_for_replication
                $patch: delete
      target:
        kind: HelmRelease
        name: promitor-scraper