Installs upstream nydus-snapshotter

[wainersm]

The operator installs nydus-snapshotter built in-house since release 0.8.0. However, recently the nydus-snapshotter project has implemented its own set of daemonSets and built binaries to deploy it on Kubernetes.

The Kata CI already uses nydus-snapshotter's provided daemonsets/images.

Likewise, the CoCo operator could be using the upstream nydus-snapshotter daemonsets and images, so to avoid duplication and maintenance burden.

Proposed by @fidencio in https://cloud-native.slack.com/archives/C066XLCU95Z/p1712331709471629

Cc @bpradipt @fitzthum @zvonkok

[wainersm]

@fidencio @fitzthum for 0.9.0? If yes then let's put in the board.

[fitzthum]

I am on the fence about this one for v0.9.0. We do still need to support forked containerd for enclave-cc and (different fork) for GPU.

[wainersm]

Hi @bpradipt @fitzthum @zvonkok @fidencio !

Now that 0.9.0-alpha0 is released, could we resume that discussion here?

When CoCo (0.8.0 release) became dependent on nydus-snapshotter we didn't have a better way to deploy that component in Kubernetes, so @fidencio implement the support in this operator. Now that nydus-snapshotter project provides the daemonsets and configurations itself; why would us continue to install it via CoCo operator?

[ldoktor]

Do you mean only to switch to containerd/nydus-snapshotter repo or to completely change the deployment? If the repo only then that should have been covered by #364

[fitzthum]

Whoops, I made a duplicate of this issue #369

[wainersm]

Whoops, I made a duplicate of this issue #369

Note actually a duplicate per se. I guess we still need the pre-reqs regardless if we drop nydus or not.

[wainersm]

Do you mean only to switch to containerd/nydus-snapshotter repo or to completely change the deployment? If the repo only then that should have been covered by #364

I meant remove the deployment completely from this operator. And assume that users will install nydus-snapshotter by their own as a requirement.

[fitzthum]

Note actually a duplicate per se. I guess we still need the pre-reqs regardless if we drop nydus or not.

Yeah I guess there are a few options. We could keep the containerd stuff for certain cases, for instances, but do nydus separately. Ideally we can get rid of the pre-reqs thing at some point tho.

[wainersm]

Cc @mkulke @stevenhorsman

[mkulke]

I tested the nydus ds a bit, they work well, but the process is a bit manual. how do we intend to consume/proces the upstream nydus manifests?

as part of some higher-level invocation (e.g in a makefile sitting next to the coco operator installation)?

managing a nydus ds with the operator? (peerpods is doing something like this in peerpodconfig-ctrl, could lead to overhead if the nydus installation needs tweaking in some cases)