>> Do you have a plan to provide the vulnerability information as a> parsable format (JSON, YAML, etc.) officially? Or, as I have already> developed the program, is there anything I can help?
You mean something like https://github.com/alpinelinux/alpine-secdb ?
-carlo
On Tue, 23 Jul 2019 16:42:43 +0900
Teppei Fukuda <knqyf263@gmail.com> wrote:
> Hi Copa,> > Thank you for the quick response! I will watch the issues of the> aports repository.> > By the way, I developed the crawler to save the vulnerability> information of Alpine as JSON format like the following.> Data: https://github.com/knqyf263/vuln-list/tree/master/alpine> Program: https://github.com/knqyf263/vuln-list-update/tree/master/alpine
Oh, nice!
> Do you have a plan to provide the vulnerability information as a> parsable format (JSON, YAML, etc.) officially? Or, as I have already> developed the program, is there anything I can help?
Short answer:
- we have someone who helps us report unfixed issues
- we lack someone that retro actively reports fixed things
- we want make it as simple as possible for those who actually fixes
things (eg, we don't want those people to do heavy manual work to be
able to push fixes)
- we want things to be automated as much as possible
Longer answer:
We are definitively interested in an advisory program, and we have
talked about it. The main problem so far is that we don't have the
needed manpower (yet).
The most important thing is to actually fix the issues. We do have a
working system to detect and make sure unfixed issues gets fixed.
The next step is to announce this in one way or the other. This we
don't have in place yet. It was possible to follow using atoms in
redmine or similar. We haven't worked out 100% how to do this in gitlab
yet.
But keep in mind that the reason that those issues are reported in our
bugtracker, is to make sure they are fixed, not to let people know that
they need to update. This means that if someone do an update that
happens to include a security fix, we may not bother file an issue
for it afterwards. So you will not see all issues by follow gitlab (it
was the same with redmine)
We do try to record which version fixes CVE issues, and to make that
simple we do that as comments in the APKBUILDs. Those are formatted as
yaml, so it is possible to extract that information. We do that with
alpine-secdb. (https://git.alpinelinux.org/alpine-secdb) this is still
a manual process, and should be automated.
Note that the alpine-secdb was created because there are 3rd party
security scanners that detects open source components and their version
number and compare that to a CVE database and flags vulnerable
versions. Those scanners does not detect when we backport patches. So
we use this as whitelist for such scanners. This means that this list
is not necessarily complete either.
I would like to combine the data we have and have some sort of advisory
program. It would be great if someone could help with this.
Thanks!
-nc
> > Thanks,> Teppei> > 2019*7*23*(*) 16:12 Natanael Copa <ncopa@alpinelinux.org>:> >> > Hi Teppei!> >> > On Tue, 23 Jul 2019 15:51:28 +0900> > Teppei Fukuda <knqyf263@gmail.com> wrote:> > > > > Hi> > >> > > I've watched the security related issues in Redmine.> > > https://bugs.alpinelinux.org/projects/alpine/issues> > >> > > I saw the following announcement.> > > "Migration from Redmine to GitLab"> > > https://lists.alpinelinux.org/~alpine/devel/%20%3CCA+cSEmPRYLv45t4+z-BsRBHyV5M0c2BisPFrjNDmUtPd28Mm_w%40mail.gmail.com%3E> > >> > > Currently, where should we watch to know the security issues? Is it below?> > > https://gitlab.alpinelinux.org/alpine/aports/issues?scope=all&utf8=%E2%9C%93&state=all&label_name[]=Security> >> > Yes, this is the current way to find the security issues. As you> > understand we are still trying to work out all details. We are open to> > ideas how to make it better.> >> > Thanks!> > > > >> > > Best regards,> > > Teppei Fukuda (@knqyf263) > >
Hi Carlo,
Yes, it is. However, alpine-secdb is database of backported fixes as
README says.
>It is not a complete database of all security issues in Alpine.
I need a complete database of all security issues.
2019年7月23日(火) 17:40 Carlo Landmeter <clandmeter@alpinelinux.org>:
>> >> > Do you have a plan to provide the vulnerability information as a> > parsable format (JSON, YAML, etc.) officially? Or, as I have already> > developed the program, is there anything I can help?>> You mean something like https://github.com/alpinelinux/alpine-secdb ?>>> -carlo
On Tue, 23 Jul 2019 17:54:40 +0900
Teppei Fukuda <knqyf263@gmail.com> wrote:
> Hi Carlo,> > Yes, it is. However, alpine-secdb is database of backported fixes as> README says.> >It is not a complete database of all security issues in Alpine. > > I need a complete database of all security issues.
We currently don't have that. I do think we have much or maybe even
most of the needed data, but its spread.
We need someone who can figure out the pieces that is missing and find
a way to collect and store it in a way that makes it as simple as
possible to fix and roll out fixes.
We could for example use the secfixes comments in APKBUILD and data
from gitlab issues and generate a database from that, and have someone
fill in the missing data, or we could turn it around, have someone
collect all the data in a database and generate issues from that and
maybe automatically add secfixes comments from it.
But we need someone who can investigate and come up with a good plan.
-nc
Hi Copa,
I appreciate your polite explanation. I understand.
Currently, my program collects the following data:
1. the secfixes comments in APKBUILD
2. alpine-secdb (maybe the same as above)
3. the security tickets of Redmine (will be replaced with the issues of GitLab)
4. git diff APKBUILD (only commits related with the above issues)
I think we can generate the security advisories by checking all git
log like No. 4. For example, the following commit fixes
CVE-2019-13636.
https://github.com/alpinelinux/aports/pull/9642/files
Watching this diff of main/patch/APKBUILD, we can detect the version
update from 2.7.6-r4 to 2.7.6-r5. This is my source code doing it.
https://github.com/knqyf263/vuln-list-update/blob/d8aefa60155637561a8a2d3feb486bbb675c996c/alpine/alpine.go#L404-L450
I know this way is not perfect. There may be false positive/negative.
However, this process can be automated and the maintenance cost is
low. It may be a good way as a first step of the security advisory. It
is better if the format of the commit message is fixed. e.g.
[os_version] pkgname: fix CVE-ID.
I want the security database of Alpine strongly and can help you in
the task of investigating it and writing an automation program. But,
It is difficult to do manual operation (e.g. I continue to fill the
security information manually).
Best,
Teppei
2019年7月23日(火) 18:18 Natanael Copa <ncopa@alpinelinux.org>:
>> On Tue, 23 Jul 2019 17:54:40 +0900> Teppei Fukuda <knqyf263@gmail.com> wrote:>> > Hi Carlo,> >> > Yes, it is. However, alpine-secdb is database of backported fixes as> > README says.> > >It is not a complete database of all security issues in Alpine.> >> > I need a complete database of all security issues.>> We currently don't have that. I do think we have much or maybe even> most of the needed data, but its spread.>> We need someone who can figure out the pieces that is missing and find> a way to collect and store it in a way that makes it as simple as> possible to fix and roll out fixes.>> We could for example use the secfixes comments in APKBUILD and data> from gitlab issues and generate a database from that, and have someone> fill in the missing data, or we could turn it around, have someone> collect all the data in a database and generate issues from that and> maybe automatically add secfixes comments from it.>> But we need someone who can investigate and come up with a good plan.>> -nc
Hi Copa,
My apologies for keeping asking you questions, but I have one more question.
When I've been following the recent security related commits of
alpine/aports, I've noticed that you always write secfixes comment
even if they were not backported fixes. Was this defined as a rule?
As I mentioned before, I hope that security advisories of Alpine will
be provided. So I would like to help if I can do anything.
I know you are so busy. It would be appreciated if I discuss this with
you when you have time.
Best regards,
Teppei
2019年7月23日(火) 16:56 Teppei Fukuda <knqyf263@gmail.com>:
>> Hi Copa,>> I appreciate your polite explanation. I understand.>> Currently, my program collects the following data:> 1. the secfixes comments in APKBUILD> 2. alpine-secdb (maybe the same as above)> 3. the security tickets of Redmine (will be replaced with the issues of GitLab)> 4. git diff APKBUILD (only commits related with the above issues)>> I think we can generate the security advisories by checking all git> log like No. 4. For example, the following commit fixes> CVE-2019-13636.> https://github.com/alpinelinux/aports/pull/9642/files>> Watching this diff of main/patch/APKBUILD, we can detect the version> update from 2.7.6-r4 to 2.7.6-r5. This is my source code doing it.> https://github.com/knqyf263/vuln-list-update/blob/d8aefa60155637561a8a2d3feb486bbb675c996c/alpine/alpine.go#L404-L450>> I know this way is not perfect. There may be false positive/negative.> However, this process can be automated and the maintenance cost is> low. It may be a good way as a first step of the security advisory. It> is better if the format of the commit message is fixed. e.g.> [os_version] pkgname: fix CVE-ID.>> I want the security database of Alpine strongly and can help you in> the task of investigating it and writing an automation program. But,> It is difficult to do manual operation (e.g. I continue to fill the> security information manually).>> Best,> Teppei>> 2019年7月23日(火) 18:18 Natanael Copa <ncopa@alpinelinux.org>:> >> > On Tue, 23 Jul 2019 17:54:40 +0900> > Teppei Fukuda <knqyf263@gmail.com> wrote:> >> > > Hi Carlo,> > >> > > Yes, it is. However, alpine-secdb is database of backported fixes as> > > README says.> > > >It is not a complete database of all security issues in Alpine.> > >> > > I need a complete database of all security issues.> >> > We currently don't have that. I do think we have much or maybe even> > most of the needed data, but its spread.> >> > We need someone who can figure out the pieces that is missing and find> > a way to collect and store it in a way that makes it as simple as> > possible to fix and roll out fixes.> >> > We could for example use the secfixes comments in APKBUILD and data> > from gitlab issues and generate a database from that, and have someone> > fill in the missing data, or we could turn it around, have someone> > collect all the data in a database and generate issues from that and> > maybe automatically add secfixes comments from it.> >> > But we need someone who can investigate and come up with a good plan.> >> > -nc
Hi,
On Sun, 11 Aug 2019 14:29:50 -1000
Teppei Fukuda <knqyf263@gmail.com> wrote:
> Hi Copa,> > My apologies for keeping asking you questions, but I have one more question.> > When I've been following the recent security related commits of> alpine/aports, I've noticed that you always write secfixes comment> even if they were not backported fixes. Was this defined as a rule?
No, it was never an expressed or documented rule or request or
anything. The community have just filled in the secfixes data, so it
seems that is the direction things are "naturally" going.
We have recently also moved to gitlab and have tried various ways to
report the issues. Gitlab has made things simpler, for example we are
now using one issue with tasklist of affected branches. We have also
started to add the commit data where the issue is fixed. This seems to
work relatively well and combined with the secfixes data, this is a
good step forward to an advisory program.
> As I mentioned before, I hope that security advisories of Alpine will> be provided. So I would like to help if I can do anything.
Can you please have a look at the recent security fixes in gitlab and
see what you think, and what we could do differently.
https://gitlab.alpinelinux.org/alpine/aports/issues?scope=all&utf8=%E2%9C%93&state=closed&label_name[]=Security
I specifically wonder how to report multiple CVEs that affect different
branches. See for example
https://gitlab.alpinelinux.org/alpine/aports/issues/10699
Do you think that tracking the security data in secfixes comments in
APKBUILDs and the reported security issues in gitlab is sufficient? We
could probably start also report the issues that we have fixed already,
with the commit with the fix.
> > I know you are so busy. It would be appreciated if I discuss this with> you when you have time.
Would you like to have a video conference meeting?
-nc
>> > Best regards,> Teppei> > 2019*7*23*(*) 16:56 Teppei Fukuda <knqyf263@gmail.com>:> >> > Hi Copa,> >> > I appreciate your polite explanation. I understand.> >> > Currently, my program collects the following data:> > 1. the secfixes comments in APKBUILD> > 2. alpine-secdb (maybe the same as above)> > 3. the security tickets of Redmine (will be replaced with the> > issues of GitLab) 4. git diff APKBUILD (only commits related with> > the above issues)> >> > I think we can generate the security advisories by checking all git> > log like No. 4. For example, the following commit fixes> > CVE-2019-13636.> > https://github.com/alpinelinux/aports/pull/9642/files> >> > Watching this diff of main/patch/APKBUILD, we can detect the version> > update from 2.7.6-r4 to 2.7.6-r5. This is my source code doing it.> > https://github.com/knqyf263/vuln-list-update/blob/d8aefa60155637561a8a2d3feb486bbb675c996c/alpine/alpine.go#L404-L450> >> > I know this way is not perfect. There may be false> > positive/negative. However, this process can be automated and the> > maintenance cost is low. It may be a good way as a first step of> > the security advisory. It is better if the format of the commit> > message is fixed. e.g. [os_version] pkgname: fix CVE-ID.> >> > I want the security database of Alpine strongly and can help you in> > the task of investigating it and writing an automation program. But,> > It is difficult to do manual operation (e.g. I continue to fill the> > security information manually).> >> > Best,> > Teppei> >> > 2019*7*23*(*) 18:18 Natanael Copa <ncopa@alpinelinux.org>: > > >> > > On Tue, 23 Jul 2019 17:54:40 +0900> > > Teppei Fukuda <knqyf263@gmail.com> wrote:> > > > > > > Hi Carlo,> > > >> > > > Yes, it is. However, alpine-secdb is database of backported> > > > fixes as README says. > > > > >It is not a complete database of all security issues in> > > > >Alpine. > > > >> > > > I need a complete database of all security issues. > > >> > > We currently don't have that. I do think we have much or maybe> > > even most of the needed data, but its spread.> > >> > > We need someone who can figure out the pieces that is missing and> > > find a way to collect and store it in a way that makes it as> > > simple as possible to fix and roll out fixes.> > >> > > We could for example use the secfixes comments in APKBUILD and> > > data from gitlab issues and generate a database from that, and> > > have someone fill in the missing data, or we could turn it> > > around, have someone collect all the data in a database and> > > generate issues from that and maybe automatically add secfixes> > > comments from it.> > >> > > But we need someone who can investigate and come up with a good> > > plan.> > >> > > -nc
Hi Copa,
Thank you for the kind response.
I looked over all recent commits. I think the reported security issues
in GitLab is sufficient as a first step. Of course, there are many
better ways such as preparing a system for vulnerability management.
However, it is hard to maintain. I believe the most important thing is
sustainability.
For maintainers, it should be easier to just write markdown for
Issue/PR. Therefore, I think it is enough to parse markdown and
generate security advisories. We need to fix the format of the
security issues. The GiaLab "description templates" may be a good idea
to fix the format.
https://docs.gitlab.com/ee/user/project/description_templates.html
If possible, let's have a video meeting!
Best regards,
Teppei
2019年8月14日(水) 11:01 Natanael Copa <ncopa@alpinelinux.org>:
>> Hi,>> On Sun, 11 Aug 2019 14:29:50 -1000> Teppei Fukuda <knqyf263@gmail.com> wrote:>> > Hi Copa,> >> > My apologies for keeping asking you questions, but I have one more question.> >> > When I've been following the recent security related commits of> > alpine/aports, I've noticed that you always write secfixes comment> > even if they were not backported fixes. Was this defined as a rule?>> No, it was never an expressed or documented rule or request or> anything. The community have just filled in the secfixes data, so it> seems that is the direction things are "naturally" going.>> We have recently also moved to gitlab and have tried various ways to> report the issues. Gitlab has made things simpler, for example we are> now using one issue with tasklist of affected branches. We have also> started to add the commit data where the issue is fixed. This seems to> work relatively well and combined with the secfixes data, this is a> good step forward to an advisory program.>> > As I mentioned before, I hope that security advisories of Alpine will> > be provided. So I would like to help if I can do anything.>> Can you please have a look at the recent security fixes in gitlab and> see what you think, and what we could do differently.> https://gitlab.alpinelinux.org/alpine/aports/issues?scope=all&utf8=%E2%9C%93&state=closed&label_name[]=Security>> I specifically wonder how to report multiple CVEs that affect different> branches. See for example> https://gitlab.alpinelinux.org/alpine/aports/issues/10699>> Do you think that tracking the security data in secfixes comments in> APKBUILDs and the reported security issues in gitlab is sufficient? We> could probably start also report the issues that we have fixed already,> with the commit with the fix.>> >> > I know you are so busy. It would be appreciated if I discuss this with> > you when you have time.>> Would you like to have a video conference meeting?>> -nc>> >> >> > Best regards,> > Teppei> >> > 2019*7*23*(*) 16:56 Teppei Fukuda <knqyf263@gmail.com>:> > >> > > Hi Copa,> > >> > > I appreciate your polite explanation. I understand.> > >> > > Currently, my program collects the following data:> > > 1. the secfixes comments in APKBUILD> > > 2. alpine-secdb (maybe the same as above)> > > 3. the security tickets of Redmine (will be replaced with the> > > issues of GitLab) 4. git diff APKBUILD (only commits related with> > > the above issues)> > >> > > I think we can generate the security advisories by checking all git> > > log like No. 4. For example, the following commit fixes> > > CVE-2019-13636.> > > https://github.com/alpinelinux/aports/pull/9642/files> > >> > > Watching this diff of main/patch/APKBUILD, we can detect the version> > > update from 2.7.6-r4 to 2.7.6-r5. This is my source code doing it.> > > https://github.com/knqyf263/vuln-list-update/blob/d8aefa60155637561a8a2d3feb486bbb675c996c/alpine/alpine.go#L404-L450> > >> > > I know this way is not perfect. There may be false> > > positive/negative. However, this process can be automated and the> > > maintenance cost is low. It may be a good way as a first step of> > > the security advisory. It is better if the format of the commit> > > message is fixed. e.g. [os_version] pkgname: fix CVE-ID.> > >> > > I want the security database of Alpine strongly and can help you in> > > the task of investigating it and writing an automation program. But,> > > It is difficult to do manual operation (e.g. I continue to fill the> > > security information manually).> > >> > > Best,> > > Teppei> > >> > > 2019*7*23*(*) 18:18 Natanael Copa <ncopa@alpinelinux.org>:> > > >> > > > On Tue, 23 Jul 2019 17:54:40 +0900> > > > Teppei Fukuda <knqyf263@gmail.com> wrote:> > > >> > > > > Hi Carlo,> > > > >> > > > > Yes, it is. However, alpine-secdb is database of backported> > > > > fixes as README says.> > > > > >It is not a complete database of all security issues in> > > > > >Alpine.> > > > >> > > > > I need a complete database of all security issues.> > > >> > > > We currently don't have that. I do think we have much or maybe> > > > even most of the needed data, but its spread.> > > >> > > > We need someone who can figure out the pieces that is missing and> > > > find a way to collect and store it in a way that makes it as> > > > simple as possible to fix and roll out fixes.> > > >> > > > We could for example use the secfixes comments in APKBUILD and> > > > data from gitlab issues and generate a database from that, and> > > > have someone fill in the missing data, or we could turn it> > > > around, have someone collect all the data in a database and> > > > generate issues from that and maybe automatically add secfixes> > > > comments from it.> > > >> > > > But we need someone who can investigate and come up with a good> > > > plan.> > > >> > > > -nc>
Speaking only about aesthetics here, but for me having APKBUILD with lots of CVE comments is not welcoming and make it harder to read. Especially packaged with lots of vulns.
Detaching the CVE list from the APKBUILD itself could do wonders for readability.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Saturday, August 17, 2019 11:29 PM, Teppei Fukuda <knqyf263@gmail.com> wrote:
> Hi Copa,>> Thank you for the kind response.>> I looked over all recent commits. I think the reported security issues> in GitLab is sufficient as a first step. Of course, there are many> better ways such as preparing a system for vulnerability management.> However, it is hard to maintain. I believe the most important thing is> sustainability.>> For maintainers, it should be easier to just write markdown for> Issue/PR. Therefore, I think it is enough to parse markdown and> generate security advisories. We need to fix the format of the> security issues. The GiaLab "description templates" may be a good idea> to fix the format.> https://docs.gitlab.com/ee/user/project/description_templates.html>> If possible, let's have a video meeting!>> Best regards,> Teppei>> 2019年8月14日(水) 11:01 Natanael Copa ncopa@alpinelinux.org:>> > Hi,> > On Sun, 11 Aug 2019 14:29:50 -1000> > Teppei Fukuda knqyf263@gmail.com wrote:> >> > > Hi Copa,> > > My apologies for keeping asking you questions, but I have one more question.> > > When I've been following the recent security related commits of> > > alpine/aports, I've noticed that you always write secfixes comment> > > even if they were not backported fixes. Was this defined as a rule?> >> > No, it was never an expressed or documented rule or request or> > anything. The community have just filled in the secfixes data, so it> > seems that is the direction things are "naturally" going.> > We have recently also moved to gitlab and have tried various ways to> > report the issues. Gitlab has made things simpler, for example we are> > now using one issue with tasklist of affected branches. We have also> > started to add the commit data where the issue is fixed. This seems to> > work relatively well and combined with the secfixes data, this is a> > good step forward to an advisory program.> >> > > As I mentioned before, I hope that security advisories of Alpine will> > > be provided. So I would like to help if I can do anything.> >> > Can you please have a look at the recent security fixes in gitlab and> > see what you think, and what we could do differently.> > https://gitlab.alpinelinux.org/alpine/aports/issues?scope=all&utf8=✓&state=closed&label_name[]=Security> > I specifically wonder how to report multiple CVEs that affect different> > branches. See for example> > https://gitlab.alpinelinux.org/alpine/aports/issues/10699> > Do you think that tracking the security data in secfixes comments in> > APKBUILDs and the reported security issues in gitlab is sufficient? We> > could probably start also report the issues that we have fixed already,> > with the commit with the fix.> >> > > I know you are so busy. It would be appreciated if I discuss this with> > > you when you have time.> >> > Would you like to have a video conference meeting?> > -nc> >> > > Best regards,> > > Teppei> > > 2019723*(*) 16:56 Teppei Fukuda knqyf263@gmail.com:> > >> > > > Hi Copa,> > > > I appreciate your polite explanation. I understand.> > > > Currently, my program collects the following data:> > > >> > > > 1. the secfixes comments in APKBUILD> > > > 2. alpine-secdb (maybe the same as above)> > > > 3. the security tickets of Redmine (will be replaced with the> > > > issues of GitLab) 4. git diff APKBUILD (only commits related with> > > > the above issues)> > > >> > > >> > > > I think we can generate the security advisories by checking all git> > > > log like No. 4. For example, the following commit fixes> > > > CVE-2019-13636.> > > > https://github.com/alpinelinux/aports/pull/9642/files> > > > Watching this diff of main/patch/APKBUILD, we can detect the version> > > > update from 2.7.6-r4 to 2.7.6-r5. This is my source code doing it.> > > > https://github.com/knqyf263/vuln-list-update/blob/d8aefa60155637561a8a2d3feb486bbb675c996c/alpine/alpine.go#L404-L450> > > > I know this way is not perfect. There may be false> > > > positive/negative. However, this process can be automated and the> > > > maintenance cost is low. It may be a good way as a first step of> > > > the security advisory. It is better if the format of the commit> > > > message is fixed. e.g. [os_version] pkgname: fix CVE-ID.> > > > I want the security database of Alpine strongly and can help you in> > > > the task of investigating it and writing an automation program. But,> > > > It is difficult to do manual operation (e.g. I continue to fill the> > > > security information manually).> > > > Best,> > > > Teppei> > > > 2019723*(*) 18:18 Natanael Copa ncopa@alpinelinux.org:> > > >> > > > > On Tue, 23 Jul 2019 17:54:40 +0900> > > > > Teppei Fukuda knqyf263@gmail.com wrote:> > > > >> > > > > > Hi Carlo,> > > > > > Yes, it is. However, alpine-secdb is database of backported> > > > > > fixes as README says.> > > > > >> > > > > > > It is not a complete database of all security issues in> > > > > > > Alpine.> > > > > >> > > > > > I need a complete database of all security issues.> > > > >> > > > > We currently don't have that. I do think we have much or maybe> > > > > even most of the needed data, but its spread.> > > > > We need someone who can figure out the pieces that is missing and> > > > > find a way to collect and store it in a way that makes it as> > > > > simple as possible to fix and roll out fixes.> > > > > We could for example use the secfixes comments in APKBUILD and> > > > > data from gitlab issues and generate a database from that, and> > > > > have someone fill in the missing data, or we could turn it> > > > > around, have someone collect all the data in a database and> > > > > generate issues from that and maybe automatically add secfixes> > > > > comments from it.> > > > > But we need someone who can investigate and come up with a good> > > > > plan.> > > > > -nc