New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Move Helm Downloads #5663
Comments
cc @helm/helm-core-maintainers |
Thanks for bringing this up, @mattfarina. I'm personally leaning towards object storage served with a CDN. Having control of the distribution of Helm releases would be great to have, as it allows us to control the distribution (like in China) and bandwidth (not hogging Github's CDN during spike loads). Additionally, we would be able to receive reliable download metrics of our content, and from what regions it's being downloaded from. As far as I know, GitHub does not provide those download metrics. Using the existing Helm Azure account would be great to burn some of that unused budget. :) Should we decide to go down this route, we should see what we can do about documenting the architecture of the Helm CDN. That way we can quickly rebuild it in case of emergency (or perform maintenance/updates to the system via PR). |
I'll just chime in as "Another Matt who agrees". CDN-fronted object storage is a good solution for this, and we do have Azure File as a storage option under our Helm account.
I think we should work on making the metrics public, if possible, even if that means taking a report and exporting it back to object storage as a data file. What level of metrics is appropriate for distributing?
*EDIT:* And can we target Helm 3 Alpha 1 as our first release this way?
|
I'll break up the Matt-fest a bit :) I think custom DNS is the most important part and should be prioritized as a first step. From there we can move things around as needed. CDN+Object storage sounds good. |
Any name suggestions?
I'm leaning towards get.helm.sh. |
/cc @idvoretskyi |
To update everyone... after talking with @idvoretskyi we're going to do three things.
|
I like the concept of DNS also but I would lean more towards download.helm.sh. I am however ok with get.helm.sh. Re to hosting, here is feedback I got after asking in the community: |
If jFrog provides adequate analytics, I'm fine with that. |
I've started work on the following:
Unfortunately, adding a CNAME record pointing get.helm.sh to the current bucket won't work. There are a few more steps involved. I'll look into this. In other news, I'm currently experimenting with Azure Blob Store and see what kind of analytics we can get from there. I'll report back once I find out more information on this. @mattfarina are you following up with JFrog on experimenting with Artifactory? If you are, perhaps in a few weeks we can share our results on a dev call to report our findings. If not, I'd be happy to experiment. I just need someone to show me where to start :) |
Since I raised about JFrog, I can investigate it if that's ok? @mattfarina What do you think or have you started investigating? |
After more investigation, it looks like more setup is involved to front our existing cloud storage buckets with a custom domain. From Google's docs:
Which would mean creating a load balancer in an account which we already have limited billing access to. With that in mind, it might make more sense to cut over sooner rather than later IMO. |
If we can't do an SSL redirect with Google, is that a show-stopper? Should we just try to pivot to jFrog (leaving old stuff at Google for legacy apps)?
Note that jFrog has a commercial Helm offering (and Rimas is a Helm co-founder). We might need to put together a quick sentence or two stating that CNCF is not officially endorsing jFrog's product, but that we do believe that jFrog is the right provider for our download needs. Or something like that.
Matt
…________________________________
From: Matthew Fisher <notifications@github.com>
Sent: Tuesday, May 7, 2019 3:42:23 PM
To: helm/helm
Cc: Matt Butcher; Team mention
Subject: Re: [helm/helm] Move Helm Downloads (#5663)
After more investigation, it looks like more setup is involved to front our existing cloud storage buckets with a custom domain. From Google's docs<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcloud.google.com%2Fstorage%2Fdocs%2Frequest-endpoints%23cname&data=01%7C01%7CMatt.Butcher%40microsoft.com%7C582a61d0b6144acd013d08d6d334e44b%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=HQgS3FRH5K1VhOEQasEVS16WfCxn7yt8T%2FO%2F2bayoik%3D&reserved=0>:
Note: You can use a CNAME redirect only with HTTP, not with HTTPS. To serve your content through a custom domain over SSL, you can set up a load balancer.
Which would mean creating a load balancer in an account which we already have limited billing access to.
With that in mind, it might make more sense to cut over sooner rather than later IMO.
—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fhelm%2Fhelm%2Fissues%2F5663%23issuecomment-490265936&data=01%7C01%7CMatt.Butcher%40microsoft.com%7C582a61d0b6144acd013d08d6d334e44b%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=D6po0lgPCE1pUAQtmp4JTRPD0zoVT3vrbykCCk%2BCbKU%3D&reserved=0>, or mute the thread<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAAVY2IF6M3QHOAMKIZL6B3PUHZT7ANCNFSM4HJPB3BQ&data=01%7C01%7CMatt.Butcher%40microsoft.com%7C582a61d0b6144acd013d08d6d334e44b%7C72f988bf86f141af91ab2d7cd011db47%7C1&sdata=6530Y6UHeSuNMa%2Bc3IqaUDCXSwaO81wtbiCeQWzR%2BLw%3D&reserved=0>.
|
There is a free JFrog Artifactory Cloud solution on GKE for OSS projects. This is the recommendation from the CNCF for CNCF projects. I can submit an application for helm if people think this is a good idea? |
Good point @mattfarina. Did @rimusz provide any feedback on additional hosting? |
JFrog Artifactory Cloud solution is on GCP not GKE :) |
Head of JFrog Developer Relations here. There should be no problem having your instance on Azure instead of GCP. |
@jbaruch Thanks for contacting us and the feedback. @rimusz Thanks for helping out and getting info on cloud hosting (and for correcting my mistake! :)) |
@helm/helm-core-maintainers how would you like to progress? Should I submit an application for the JFrog Artifactory Cloud solution? Let me know what you think. |
In the same time it's taken to figure out how to provision Artifactory, I've already gone ahead and:
I'll be more than happy to provide a demo tomorrow during the dev call to show how this is all set up. #5694 provides a way to publish Helm 3 assets over to this infrastructure moving forward. If we want to publish Helm 2 release assets here as well, I'd be more than happy to provide a PR to master, though I imagine we probably want to continue publishing to GCS for backwards compatibility concerns. |
As an example, here's what the final destination looks like: https://get.helm.sh/helm-v2.13.1-linux-amd64.tar.gz |
imho, experimentation is always valuable, but if JFrog is indeed the recommended way for CNCF projects, then we need to justify why we are NOT following their guidance. Let's minimise divergence here, unless there is a really good reason. |
The goal was to evaluate several systems and come up with a workable solution. Artifactory is one solution, Azure is another. There's several reasons why we should choose Azure in this case, in my opinion:
The last point is the most critical IMO. At the end of the day, it is the responsibility of the core maintainers to ensure that the system is running smoothly, and we're the ones on call to support this. This decision should be up to the core maintainers to make the call on what system they feel most comfortable going to production with. Given my personal experience being a Microsoft employee, I'm more comfortable supporting Azure in production. |
I should also point out that if we want download metrics for the first alpha release, we should land on a solution soon. We're zeroing in on a final release candidate and KubeCon EU is in 2 weeks. We should have something in place by that time. |
@henrynash I talked to the CNCF directly, and they do not prefer or require jFrog. They merely provide jFrog services. We also happen to have Azure services provided for us, as well as Google Cloud. CNCF leaves it up to projects to choose what technology they select. @mattfarina was in the same meeting and can confirm. |
Quick sampling of CNCF projects:
|
I did find one project (thanks @hickeyma for the pointer): GRPC. They use Artifactory for https://packages.grpc.io/. Regardless, the main motivation for moving was
The storage layer is just an implementation detail. |
So given @technosophos comment on the fact that this isn't a recommendation from the CNCF, then my point is mute :-). My general point was that, given that I assume we are looking towards graduation as well as an increasingly diverse maintainer membership as we grow, that our technology choices should try and follow the patterns that are out there in the CNCF (and other related foundations). |
@rimusz Thanks for the correction and filling things in. @jbaruch Would you have some time to talk about Artifactory? I have a bunch of questions. @henrynash Artifactory is an option rather than something that's recommended. In the meeting @technosophos and I were in we discussed both Artifactory and Azure storage (we already use Azure for some other things). There was no preference give to any solution. They just wanted us to be aware of the options. I am curious about Artifactory because I'm thinking more holistically than just tgz downloads and I'm interested in statistics. If we were going to go with the simpler solution that is less work for the immediate problem it would be Azure Storage. |
@henrynash I agree with @mattfarina and @technosophos as being an option and not the only solution to use. Sorry for any mis-understanding gleaned from comments above. |
@henrynash We do look at what the other CNCF projects are doing and talk with CNCF folks about it. I also tend to attend the TOC meeting to follow along with any direction they are discussing. |
@mattfarina excellent! |
Couple points to add to this:
|
Re CNCF, we already serve some CNCF projects (e.g. grpc) on Artifactory and looking forward to serve more :) |
As of #5709 we are now publishing Helm 3 release assets to https://get.helm.sh, so the original ask for this issue has been resolved. I assume we want to keep this open until we evaluate other systems though? |
@bacongobbler Might I suggest we keep this open until we launch the alpha1. After that we create a ticket to move if there is a reason and the reason be detailed in the issue for traceability. |
Sounds good to me. |
3.0.0-alpha.1 was released, so this can be closed. |
Helm downloads are currently served out of a bucket in GKE owned by the Kubernetes people at Google. This is not part of the Kubernetes infra in the CNCF, where Kubernetes is migrating the things to. The bucket is not in the control of the Helm project.
We should migrate downloads/assets to a new location run by the Helm project. Two possibilities immediately come to mind:
The pros to using GitHub Releases is that a) they are free to use and b) someone viewing the releases can find all the files. The cons to GitHub Releases are the extra complication in automation uploading the files and in the scripts that need to find the right file to download and install.
The pros to a custom storage bucket + DNS is that a) it's in more of our control and b) the scripts we have will be less complicated. The con is cost. But, we could do this on the existing Helm Azure account where we have the room.
We've been asked when we can move off the Google managed infrastructure so this is something we need to do.
Any thoughts on the alternative path to start using?
The text was updated successfully, but these errors were encountered: