Subject: [3.14] Release process change proposal: Add official Alpine Linux cloud images

From: Ariadne Conill <[email-masked]>
Date: Tue, 15 Dec 2020 02:29:08 GMT

Hello,

I believe it is time (beyond time, really) for us to start mastering official 
Alpine cloud images as part of the release process.  This is something that 
other distributions comparable to us have been doing for some time.

Additionally, due to the vacuum of official cloud images, there has been a few 
people offering custom Alpine images on AWS at prices that any reasonable 
person would consider predatory [1].  Multiple people have been lead at first 
glance to believe these activities were somehow connected to the project, 
which is also a problem [2].

This lead to community members banding together to create unofficial community 
AMIs for using Alpine on EC2 that are offered gratis [3], but those efforts have 
not had the official backing of the project, which continues to enable the 
predatory vendors to unjustly profit from our project and its reputation.

## Benefits to Alpine

Adopting the Alpine Linux EC2 images (and adjusting them to use cloud-init 
[4]) would give users a clear path to an Alpine image on the cloud provider of 
their choice, at least AWS, Google Cloud, Azure and Oracle would be targeted 
to begin with.  With a clear path to an officially supported image, predatory 
vendors would find it more difficult to continue misusing the Alpine name and 
reputation to sell their custom images and pass them off as official.

## Organizational Plan

These images would be maintained by a new Cloud team which would coordinate 
the image mastering process, as well as all packages relating to using, 
creating and publishing the images.

They would participate in the release management process, ensuring that Alpine 
images are generated and published automatically by the build servers when a 
release is generated.

## Support Requirements from Alpine Infrastructure Team

The resources relating to the the community AMI image effort would be 
transferred to the Infrastructure team.  This primarily consists of a small 
application that acts as a credential broker.

Storage for the images would also become managed by the infrastructure team, 
but most likely we could get the costs of doing so offset as other 
distributions have.

A GitLab team would be created for the Cloud team so that they may own 
packages and Git repositories in the Alpine project.

Ariadne

[1]: https://aws.amazon.com/marketplace/pp/B07Q4Q8KV5 is offered at a starting 
rate of $0.50/hour or $2500 per year, is outdated and has several CVEs in the 
image.  No support or refunds are offered for this image.

[2]: The latest example being https://twitter.com/mistersql/status/
1337907929832452099

[3]: https://github.com/mcrute/alpine-ec2-ami/tree/master/releases

[4]: cloud-init was recently brought to Alpine by Dermot Bradley (minimal in 
IRC) largely for this purpose.

---

From: Mike Crute <[email-masked]>
Date: Mon, 14 Dec 2020 19:04:52 -0800

On Tue, Dec 15, 2020 at 02:29:08AM +0000, Ariadne Conill wrote:
> Hello,
> 
> I believe it is time (beyond time, really) for us to start mastering 
> official Alpine cloud images as part of the release process.  This is 
> something that other distributions comparable to us have been doing 
> for some time.
> 
> Additionally, due to the vacuum of official cloud images, there has 
> been a few people offering custom Alpine images on AWS at prices that 
> any reasonable person would consider predatory [1].  Multiple people 
> have been lead at first glance to believe these activities were 
> somehow connected to the project, which is also a problem [2].
> 
> This lead to community members banding together to create unofficial 
> community AMIs for using Alpine on EC2 that are offered gratis [3], 
> but those efforts have not had the official backing of the project, 
> which continues to enable the predatory vendors to unjustly profit 
> from our project and its reputation.
> 
> ## Benefits to Alpine
> 
> Adopting the Alpine Linux EC2 images (and adjusting them to use 
> cloud-init [4]) would give users a clear path to an Alpine image on 
> the cloud provider of their choice, at least AWS, Google Cloud, Azure 
> and Oracle would be targeted to begin with.  With a clear path to an 
> officially supported image, predatory vendors would find it more 
> difficult to continue misusing the Alpine name and reputation to sell 
> their custom images and pass them off as official.
> 
> ## Organizational Plan
> 
> These images would be maintained by a new Cloud team which would 
> coordinate the image mastering process, as well as all packages 
> relating to using, creating and publishing the images.
> 
> They would participate in the release management process, ensuring 
> that Alpine images are generated and published automatically by the 
> build servers when a release is generated.
> 
> transferred to the Infrastructure team.  This primarily consists of a 
> small application that acts as a credential broker.
> 
> Storage for the images would also become managed by the infrastructure 
> team, but most likely we could get the costs of doing so offset as 
> other distributions have.
> 

Hi Ariadne,

I started the community images that you are referring to and am one of 
the maintainers of them. I completely support pulling these into the 
upstream Alpine project and treating them as official release. I would 
be happy to assist in doing that and, with my co-maintainers, continuing 
to maintain and release them as part of the Cloud team going forward.

The current images all reside in an AWS account dedicated to only the 
hosting and production of those images. Migrating it to Alpine should be 
trivial, and current users would not be required to change their 
orchestration scripts to continue using the official images.

There's a little bit of private infrastructure that facilitates secure 
credential management that would need to be open sourced and transferred 
to the Alpine Infrastructure team but that too should be pretty 
straightforward. That infrastructure will need to grow a bit to support 
multi-cloud use-cases in the future.

We've been primarily focused on stability and feature completeness for 
AWS at this point and I think we're on good footing there. I'd recommend 
releasing those images as official as soon as reasonably possible, then 
we can start to investigate the work that would be required to begin 
adding support for other clouds. This would also put us into a good 
position to advertise the images on the AWS marketplace as official 
Alpine images and to make the case for the removal of the existing, 
predatory, images.

I've opened issues in our GitHub project [0] to begin research on the 
other items you've mentioned.

[0] https://github.com/mcrute/alpine-ec2-ami/issues

~mike

---

From: Ariadne Conill <[email-masked]>
Date: Tue, 15 Dec 2020 06:07:59 GMT

Hello,

> to maintain and release them as part of the Cloud team going forward.

Yes, this is basically the idea I had in mind after talking to you about it on 
IRC.  I would prefer to see you lead this new team in fact, since you're 
already producing the community images anyway.

> multi-cloud use-cases in the future.

Most likely the credential broker software itself would still be maintained by 
the cloud team, with infra team handling the deployment of the software.  This 
is how it is done with other services managed by the infra team.

> Alpine images and to make the case for the removal of the existing,

Right that is my hope that we can have this for 3.14, and thus put that $2500/
year image out of business.  I think it is something achievable.  Obviously 
getting something together for 3.13 is not doable, since we are already close 
to release.

> [0] https://github.com/mcrute/alpine-ec2-ami/issues

We will probably want to move the github repo into the Alpine gitlab, but for 
now, we can work through issues over on github until the necessary 
infrastructure begins to be set up.

Right now, my main goal is to get everyone interested in making this happen 
talking to each other, so we can move forward with establishing a more formal 
relationship with AWS, Azure, Google, Oracle and whoever else's cloud we want 
our images on.

Ariadne

---

From: Mike Crute <[email-masked]>
Date: Tue, 15 Dec 2020 08:37:19 -0800

On Tue, Dec 15, 2020 at 06:07:59AM +0000, Ariadne Conill wrote:
>> I started the community images that you are referring to and am one 
>> of the maintainers of them. I completely support pulling these into 
>> the upstream Alpine project and treating them as official release. I 
>> would be happy to assist in doing that and, with my co-maintainers, 
>> continuing to maintain and release them as part of the Cloud team 
>> going forward.
>
> Yes, this is basically the idea I had in mind after talking to you 
> about it on IRC.  I would prefer to see you lead this new team in 
> fact, since you're already producing the community images anyway.

That works for me. I'm happy to lead the cloud team and to get these 
images pulled into Alpine proper. I've spoken with my co-maintainer and 
he's onboard as well.

>> We've been primarily focused on stability and feature completeness 
>> for AWS at this point and I think we're on good footing there. I'd 
>> recommend releasing those images as official as soon as reasonably 
>> possible, then we can start to investigate the work that would be 
>> required to begin adding support for other clouds. This would also 
>> put us into a good position to advertise the images on the AWS 
>> marketplace as official Alpine images and to make the case for the 
>> removal of the existing, predatory, images.
>
> Right that is my hope that we can have this for 3.14, and thus put 
> that $2500/ year image out of business.  I think it is something 
> achievable.  Obviously getting something together for 3.13 is not 
> doable, since we are already close to release.

What makes this not doable for 3.13? If we can have images ready the day 
or week of the 3.13 release, would that make this workable? I'm happy to 
start getting things ready for a migration in 3.14 but would prefer, if 
possible, to get these officially released sooner rather than later.

> We will probably want to move the github repo into the Alpine gitlab, 
> but for now, we can work through issues over on github until the 
> necessary infrastructure begins to be set up.

Agreed. I suggest that we work in parallel to get the Alpine infra setup 
and then migrate everything over. Until that's ready we'll continue to 
work on GitHub.

~mike

---

From: Natanael Copa <[email-masked]>
Date: Wed, 16 Dec 2020 16:28:43 +0100

On Tue, 15 Dec 2020 08:37:19 -0800

> images pulled into Alpine proper. I've spoken with my co-maintainer and 

Awesome!
 
> possible, to get these officially released sooner rather than later.

The only thing is that we are already late with 3.13 release and we
want avoid anything that delays the release. But if can manage to
on-board you to the Alpine infra team and you can do the glue, then I
don't see anything preventing it for 3.13.

---

From: Natanael Copa <[email-masked]>
Date: Tue, 15 Dec 2020 08:26:09 +0100

On Tue, 15 Dec 2020 02:29:08 GMT

> other distributions comparable to us have been doing for some time.

Yes. But lets get 3.13 out first?

... 

> reputation to sell their custom images and pass them off as official.

I'd like to have linode on the list early too.

I think it would be great if we could have alpine based images for CI
systems, like github actions:

```
jobs:
  build:
    name: Build
    runs-on: alpine-latest
    ...
```

I suppose having cloud images is the first step.

>     that Alpine images are generated and published automatically by
>     the build servers when a release is generated.

I suspect we may need more people on the infra team for that. 

> Storage for the images would also become managed by the
>     infrastructure team, but most likely we could get the costs of
>     doing so offset as other distributions have.

We may need to get more disk space for this on our master mirror.

> A GitLab team would be created for the Cloud team so that they may
>     own packages and Git repositories in the Alpine project.
> 
> Ariadne
> 
> [1]: https://aws.amazon.com/marketplace/pp/B07Q4Q8KV5 is offered at a
>     starting rate of $0.50/hour or $2500 per year, is outdated and
>     has several CVEs in the image.  No support or refunds are offered
>     for this image.
> 
> [4]: cloud-init was recently brought to Alpine by Dermot Bradley
>     (minimal in IRC) largely for this purpose.
>

---

From: Ariadne Conill <[email-masked]>
Date: Tue, 15 Dec 2020 09:17:06 GMT

On Tuesday, December 15, 2020 12:26:09 AM MST Natanael Copa wrote:
> > something that other distributions comparable to us have been doing for
> > some time.

Right, hince why I propose it for 3.14.  It is good to start planning 
initiatives like this early, so that we determine resource requirements and 
fulfill them ahead of time.

> > [4]) would give users a clear path to an Alpine image on the cloud
> > provider of their choice, at least AWS, Google Cloud, Azure and Oracle
> > would be targeted to begin with.  With a clear path to an officially
> > supported image, predatory vendors would find it more difficult to
> > continue misusing the Alpine name and reputation to sell their custom
> > images and pass them off as official.

Would linode actually use our image?  My understanding is that linode prefer 
to master their own images.

> I suppose having cloud images is the first step.

GitHub actions already support Alpine (and any other image that is delivered 
by Docker), but it is kind of weird in how it works.  See ifupdown-ng GitHub 
actions for details there.

> I suspect we may need more people on the infra team for that.

Some of the people who have been working on the community AMI effort would 
likely be interested in volunteering there.

> We may need to get more disk space for this on our master mirror.

In general, the storage for the images are provided by the cloud providers, as 
you upload the mastered images to each cloud provider as an AMI or whatever.  
Thus storing those cloud images would not require us to store anything new on 
our mirrors.

Ariadne

---

From: Mike Crute <[email-masked]>
Date: Tue, 15 Dec 2020 08:30:34 -0800

On Tue, Dec 15, 2020 at 09:17:06AM +0000, Ariadne Conill wrote:
>> On Tue, 15 Dec 2020 02:29:08 GMT Ariadne Conill <[email-masked]> wrote:
>>
> Would linode actually use our image?  My understanding is that linode 
> prefer to master their own images.

Having Linode available would be nice. At a quick glance I don't see 
anything in their API that supports custom images, though I recall there 
was some roundabout way to upload them for a single account several 
years ago. We may need to talk to a human on their side to make that 
happen, if anyone has contacts over there. Once we migrate to GitLab I'll 
throw together issues for each cloud provider we want to support and we 
can use that as a jumping off point for research and prioritization of 
support.

> GitHub actions already support Alpine (and any other image that is 
> delivered by Docker), but it is kind of weird in how it works.  See 
> ifupdown-ng GitHub actions for details there.

I thought most CI systems were docker based these days. I guess we could 
look into this going forward. I would recommend not making this a 
blocker for a 3.14 release of the images though since there's a lot more 
surface area to cover in the CI space and a limited number of hands to 
cover it.

> Some of the people who have been working on the community AMI effort 
> would likely be interested in volunteering there.

I'd be happy to join the infra team as well to make this happen. I'm 
already personally hosting all of the AWS AMI infra as it stands so I'm 
pretty familiar with the space.

>> > The resources relating to the the community AMI image effort would 
>> > be transferred to the Infrastructure team.  This primarily consists 
>> > of a small application that acts as a credential broker.
>> >
> In general, the storage for the images are provided by the cloud 
> providers, as you upload the mastered images to each cloud provider as 
> an AMI or whatever. Thus storing those cloud images would not require 
> us to store anything new on our mirrors.

That's correct, at least until we get to Qemu images (which are on my 
wish list). But the cloud providers proper have their own storage. This 
does come with some fractional USD cost but currently it's under $5/mo 
for the global AWS image set.

~mike

---

From: jake buchholz <[email-masked]>
Date: Tue, 15 Dec 2020 09:05:40 -0800

Hi, I’m Jake, the other main person involved with the “alpine-ec2-ami” project.  I’d also like to officially join this “Officialization”  effort (cloud/infra).

>>> On Tue, 15 Dec 2020 02:29:08 GMT Ariadne Conill <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
>> Would linode actually use our image?  My understanding is that linode prefer to master their own images.
> 
> Having Linode available would be nice. At a quick glance I don't see anything in their API that supports custom images, though I recall there was some roundabout way to upload them for a single account several years ago. We may need to talk to a human on their side to make that happen, if anyone has contacts over there. Once we migrate to GitLab I'll throw together issues for each cloud provider we want to support and we can use that as a jumping off point for research and prioritization of support.

It would appear that Linode does support Packer-built images...
* https://www.linode.com/docs/guides/how-to-use-linode-packer-builder/ <https://www.linode.com/docs/guides/how-to-use-linode-packer-builder/>

…and Packer has a Linode builder…
* https://www.packer.io/docs/builders/linode <https://www.packer.io/docs/builders/linode>

…so creating Alpine images for Linode (or any other cloud provider that Packer supports) should be doable.

Having some providers “bless” our images as Official and make them publicly available will likely involve working with them to make it happen.

j

---

From: Ariadne Conill <[email-masked]>
Date: Wed, 16 Dec 2020 12:31:03 GMT

On Tuesday, December 15, 2020 10:05:40 AM MST jake buchholz wrote:
> Hi, I’m Jake, the other main person involved with the “alpine-ec2-ami”
> project.  I’d also like to officially join this “Officialization”  effort
> (cloud/infra).
> >>> On Tue, 15 Dec 2020 02:29:08 GMT Ariadne Conill
> >>> <[email-masked] <mailto:[email-masked]>> wrote:
> >>> 
> >> prefer to master their own images.> 
> > was some roundabout way to upload them for a single account several years
> > ago. We may need to talk to a human on their side to make that happen, if
> > anyone has contacts over there. Once we migrate to GitLab I'll throw
> > together issues for each cloud provider we want to support and we can use
> > that as a jumping off point for research and prioritization of support.
> * https://www.linode.com/docs/guides/how-to-use-linode-packer-builder/
> <https://www.linode.com/docs/guides/how-to-use-linode-packer-builder/>
> 
> * https://www.packer.io/docs/builders/linode
> <https://www.packer.io/docs/builders/linode>
> 
> Having some providers “bless” our images as Official and make them publicly
> available will likely involve working with them to make it happen.

I guess what I am saying is, I can go to Linode and create an Alpine VM 
already.  Why would they use our image over the image they already generate?

Ariadne

---

From: jake buchholz <[email-masked]>
Date: Tue, 15 Dec 2020 09:07:36 -0800

j

---

From: Natanael Copa <[email-masked]>
Date: Wed, 16 Dec 2020 22:32:46 +0100

On Tue, 15 Dec 2020 08:30:34 -0800

> pretty familiar with the space.

We had a video call with Mike and Mike is now officially on the infra team.

Welcome!

Thank you very much for helping us with official cloud images.

-nc

---

From: Ariadne Conill <[email-masked]>
Date: Wed, 16 Dec 2020 21:37:32 GMT

Hi,

> Thank you very much for helping us with official cloud images.

Sorry I missed the meeting -- I don't know whether the problem was KMail or 
Google meet, but the invite said the wrong time.

Are we going to be able to make the deadline for 3.13 or will it be postponed 
until 3.14?

Ariadne

---

From: Natanael Copa <[email-masked]>
Date: Wed, 16 Dec 2020 22:42:00 +0100

> 16. des. 2020 kl. 22:37 skrev Ariadne Conill <[email-masked]>:
> 
> Hi,
> 

Hi! No worries.

We are aming for 3.13. Seems like there is very litte overhead to make that happen from our end.

Initially only the AWS AMI will be supported and well try support for more cloud providers for 3.14.

-nc