Figure out testing strategy for localkube/k8s

[dlorenc]

We'll need to test localkube vs. kubernetes at HEAD (or as close to it as possible). This is non-trivial because localkube vendors and directly links in kubernetes packages.

One idea:

Have a CI job that updates all vendored kubernetes components from master daily and runs tests, then discards the changes. This job should alert on failures, which would indicate manual work will need to be done.

Weekly/monthly updates can be made and committed by an actual person.

cc @vishh Does this make sense?

[luxas]

+1 for a travis/GCE/shippable job that does this w/ godep and runs on every PR

[dlorenc]

If we go this route, it might be easiest to switch to glide. We can then use a version: master tag for k8s.io/kubernetes, which will cause glide up -u to get the new version of k8s packages each time.

@ethernetdan: any concerns with using glide instead of godep? Here's a proof of concept PR using glide and updating to the latest k8s bits for localkube: redspread/localkube#59

[vishh]

@dlorenc What does minikube require from core kube other than client libs? Are you thinking about localkube specifically? If yes, that is one of the reasons why I felt it might be better to include localkube in kubernetes core itself.
Updating vendored code is often times non-trivial. I'm concerned that the CI job might break often.
Should we instead update kube deps on kube releases only? There is an alpha release every two weeks I think.

[ethernetdan]

This is definitely something worth putting some effort into, I found that localkube's build frequently breaks between Kubernetes releases.

How hard would it be to use integration tests from Kube core? These would be really nice to ensure we don't break features.

@dlorenc: definitely interested in trying glide, my experience managing localkube's dependencies with Godep has been rocky

[vishh]

@dlorenc @ethernetdan Would it make sense to co-ordinate releases of minikube with main kube releases?

[ethernetdan]

@vishh I think that makes a lot of sense, otherwise we are going to be constantly playing catchup

[steveej]

I want to ask about a use-case and at the same time suggest to add this to this feature request: building/running a local kubernetes codebase for development. This would be very helpful for testing changes to kubernetes locally without messing with the dev (host) state. If there's already a method for that which I'm not aware of, I'm happy to learn about it. Otherwise I suggest to add this feature to minikube.

[aaron-prindle]

We have other issues for testing localkube and k8s, the last feature request we will track in another issue.
#558