Plans for K8s

[gregpakes]

I wanted to understand the plans around k8s in development and production.

At the moment, we use Aspire locally and we publish to AKS using manual manifests in gitops.

In production, I want to start using KEDA and have it monitor the Azure Service Bus and spin up a container to process messages as and when they arrive. That is all fine, my issue is the development side. The devs need the same experience locally. What is the recommendation here?

Are there plans to have the Aspire orchestrator operate with a local mini kubernetes cluster in the same way it works with Docker?

Or will we always have the discrepancy between dev and prod for things like this? At the moment, I am thinking that in dev we will just run that container as a persistent, always on container, but that isn't the reality of how it would work in prod.

The middle-ground at the moment seems to be to keep using Aspire Orchestrator with docker and run a mini kubernetes cluster (docker/orbstack) with KEDA installed to spin up the container mentioned above. The issue here is that I think it will be super unpopular with the devs as Aspire makes their current workflow so simple. This would complicate it somewhat and make it more manual.

Would love your thoughts.

[davidfowl]

Are there plans to have the Aspire orchestrator operate with a local mini kubernetes cluster in the same way it works with Docker?

No there are no plans here.

Or will we always have the discrepancy between dev and prod for things like this? At the moment, I am thinking that in dev we will just run that container as a persistent, always on container, but that isn't the reality of how it would work in prod.

For the forseeable future.

The middle-ground at the moment seems to be to keep using Aspire Orchestrator with docker and run a mini kubernetes cluster (docker/orbstack) with KEDA installed to spin up the container mentioned above. The issue here is that I think it will be super unpopular with the devs as Aspire makes their current workflow so simple. This would complicate it somewhat and make it more manual.

It would be interesting to see if this could be made simple.

The next thing we will be doing for k8s is making aspire publish robust and making aspire deploy work e2e.

There's a whole devex on top of k8s that I think you are trying to build here that we are not focusing on at least in the short term. That said, we'd love to learn from your exploration. It'll inform what we are able to do and help us evaluate what types of systems still needs lots of work.

[davidfowl]

You should be able to mutate the k8s manifest in C# via the model to do whatever equivalent you would do by hand just programmatically.

[gregpakes]

In the case of Argo Rollouts - this wouldn't be a mutation of the deployment manifest, it's a new rollout manifest which links to the deployment one.

So the c# code would know nothing about this. As I said, I just think you almost immediately step out of the happy path.

[davidfowl]

I think you'll just end up in k8s manifest patch and override hell. But I am not particularly experienced in k8s, I'm just going on my experience with aspirate.

Both things cant be true... That said, we haven't spent enough time here to give you a solid answer about what this looks like.

[YohanSciubukgian]

I’m wondering if Radius (https://radapp.io/) could complement Aspire by covering the Kubernetes ecosystem.
Have there been any updates/progresses on the POC or the discussions between the Aspire team and the Radius team since May? #811 (comment)

[davidfowl]

We're meeting again the new year.