~alpine/devel

10 5

[3.15] System change proposal: Use netbsd-curses instead of GNU ncurses

Ariadne Conill
Details
Message ID
<e12847f2-4cea-c3e8-84c3-e98b92553f8e@dereferenced.org>
DKIM signature
missing
Download raw message
Hello,

This proposal is largely centered around normalizing our package set with 
other musl distributions, such as Sabotage and Adelie, which already ship 
netbsd-curses in favor of GNU ncurses.

## GNU ncurses vs netbsd-curses

* GNU ncurses is presently the standard curses library on Linux systems.

Some applications may access private mechanisms inside the curses 
implementation and need patching.

* Both libraries implement the POSIX curses API.

This means, 99% of applications won't need patching and should be 
basically compatible.

* ncurses ships a huge terminfo database.

Ncurses presently ships a database which takes 7MiB on disk.  We split 
this into `ncurses-terminfo` and `ncurses-terminfo-base` to reduce the 
installed size to only 216KiB by default.

netbsd-curses precompiles the terminfo database into a small CDB format 
database.

* ncurses is large.

At present, we try to avoid using curses support in programs which 
optionally use it, because ncurses is a heavy dependency (ncurses-libs is 
500KiB, plus the 216KiB base terminfo database, plus 284KiB for tools). 
netbsd-curses is only 1.3MiB on disk for everything and can be split in 
the same way to reduce size.

* GNU ncurses code quality is lacking.

GNU ncurses is littered with macros and other layers of preprocessing. 
This makes it hard to understand the source code, which makes hotfixing 
security vulnerabilities a potential challenge.

netbsd-curses is written in straightforward C89/C99, one of the preferred 
languages for tools and base system components in Alpine.  It is much 
easier to hack on than GNU ncurses.

Simpler code also makes for more portable code in general.

## Benefits to Alpine

Alpine benefits from switching to a curses implementation that is 
actively developed on musl systems, in addition to the other benefits 
outlined above.

## Contingency Plan

We will do an experimental rebuild with netbsd-curses-dev providing 
ncurses-dev.  If this goes well, then we will switch to netbsd-curses.

If there is a problem, then we will try to fix netbsd-curses first, 
failing that, we will revert back to GNU ncurses.

Ariadne
Led
Details
Message ID
<CAAO3vZ_AexCkJyibKPmxMWeacdXBU7GZgYUK8uraYpYmOdwUEw@mail.gmail.com>
In-Reply-To
<e12847f2-4cea-c3e8-84c3-e98b92553f8e@dereferenced.org> (view parent)
DKIM signature
missing
Download raw message
> This proposal is largely centered around normalizing our package set with
> other musl distributions, such as Sabotage and Adelie, which already ship
> netbsd-curses in favor of GNU ncurses.
>

Why are you trying to harm Alpine? Because you hate GNU? Why do you hate
GNU?
Led
Details
Message ID
<CAAO3vZ_EDSvMtM5Xg0qK3aAKi3nhaXYLHsUiHSfy4hvO1oXSmA@mail.gmail.com>
In-Reply-To
<e12847f2-4cea-c3e8-84c3-e98b92553f8e@dereferenced.org> (view parent)
DKIM signature
missing
Download raw message
> This proposal is largely centered around normalizing our package set with
> other musl distributions, such as Sabotage and Adelie, which already ship
> netbsd-curses in favor of GNU ncurses.


Why are you trying to harm Alpine? Because you hate GNU? Why do you hate GNU?
Max Rees
Details
Message ID
<3f0b5db7-d905-9ed4-3911-1f1352c60d9a@me.com>
In-Reply-To
<e12847f2-4cea-c3e8-84c3-e98b92553f8e@dereferenced.org> (view parent)
DKIM signature
missing
Download raw message

On 4/6/21 2:17 PM, Ariadne Conill wrote:
> Hello,
> 
> This proposal is largely centered around normalizing our package set 
> with other musl distributions, such as Sabotage and Adelie, which 
> already ship netbsd-curses in favor of GNU ncurses.

As a point of order, Adélie currently does not use NetBSD curses - we 
had been planning to switch at some point I believe (which is probably 
where you got the idea), but we are still on ncurses at the moment.

The rest of your proposal sounds sound to me and in alignment with what 
we had (informally) discussed at Adélie.

Max "[[sroracle]]" Rees
Ariadne Conill
Details
Message ID
<39816f25-5ecf-cbbb-b2c-bf5fdbb6c2fb@dereferenced.org>
In-Reply-To
<CAAO3vZ_EDSvMtM5Xg0qK3aAKi3nhaXYLHsUiHSfy4hvO1oXSmA@mail.gmail.com> (view parent)
DKIM signature
missing
Download raw message
Hello,

On Tue, 6 Apr 2021, Led wrote:

>> This proposal is largely centered around normalizing our package set with
>> other musl distributions, such as Sabotage and Adelie, which already ship
>> netbsd-curses in favor of GNU ncurses.
>
>
> Why are you trying to harm Alpine?

The proposal outlines the benefits and risks to Alpine.  By using a better 
maintained ncurses implementation that is also simpler and smaller, this 
proposal is in line with the fundamental design values of the project: 
Small, Simple and Secure.  Or in other words, Alpine is improved by this 
proposal.

> Because you hate GNU? Why do you hate GNU?

These are bad faith arguments.  My desire is to ensure Alpine ships the 
best possible implementations that are aligned with our design values. 
Unfortunately, those implementations are increasingly not GNU ones, 
because of the way the GNU project approaches portability and other 
concerns.

If you have technical evidence to demonstrate that GNU ncurses is superior 
to the NetBSD implementation for Alpine, then please feel free to submit 
those arguments, but bad faith arguments are unhelpful and unwelcome.

Ariadne
Details
Message ID
<8fe3e82a56ea5f6127383b69f09e93f8c33abc3d.camel@gmail.com>
In-Reply-To
<e12847f2-4cea-c3e8-84c3-e98b92553f8e@dereferenced.org> (view parent)
DKIM signature
missing
Download raw message
On Tue, 2021-04-06 at 12:17 -0600, Ariadne Conill wrote:
> Hello,
> 
> This proposal is largely centered around normalizing our package set
> with 
> other musl distributions, such as Sabotage and Adelie, which already
> ship 
> netbsd-curses in favor of GNU ncurses.
> 
> ## GNU ncurses vs netbsd-curses
> 
> * GNU ncurses is presently the standard curses library on Linux
> systems.
> 
> Some applications may access private mechanisms inside the curses 
> implementation and need patching.
> 
> * Both libraries implement the POSIX curses API.
> 
> This means, 99% of applications won't need patching and should be 
> basically compatible.
> 
> * ncurses ships a huge terminfo database.
> 
> Ncurses presently ships a database which takes 7MiB on disk.  We
> split 
> this into `ncurses-terminfo` and `ncurses-terminfo-base` to reduce
> the 
> installed size to only 216KiB by default.
> 
> netbsd-curses precompiles the terminfo database into a small CDB
> format 
> database.
> 
> * ncurses is large.
> 
> At present, we try to avoid using curses support in programs which 
> optionally use it, because ncurses is a heavy dependency (ncurses-
> libs is 
> 500KiB, plus the 216KiB base terminfo database, plus 284KiB for
> tools). 
> netbsd-curses is only 1.3MiB on disk for everything and can be split
> in 
> the same way to reduce size.
> 
> * GNU ncurses code quality is lacking.
> 
> GNU ncurses is littered with macros and other layers of
> preprocessing. 
> This makes it hard to understand the source code, which makes
> hotfixing 
> security vulnerabilities a potential challenge.
> 
> netbsd-curses is written in straightforward C89/C99, one of the
> preferred 
> languages for tools and base system components in Alpine.  It is much
> easier to hack on than GNU ncurses.
> 
> Simpler code also makes for more portable code in general.
> 
> ## Benefits to Alpine
> 
> Alpine benefits from switching to a curses implementation that is 
> actively developed on musl systems, in addition to the other benefits
> outlined above.
> 
> ## Contingency Plan
> 
> We will do an experimental rebuild with netbsd-curses-dev providing 
> ncurses-dev.  If this goes well, then we will switch to netbsd-
> curses.
> 
> If there is a problem, then we will try to fix netbsd-curses first, 
> failing that, we will revert back to GNU ncurses.
> 
> Ariadne

Hello

+1

Feel free to ping if help is necessary in anything related to aports

Regards
Leo
Led
Details
Message ID
<CAAO3vZ91BrPp2rR0nZP1M8P8zMsRiCNJA5+1BZ+PaoE_=2acfg@mail.gmail.com>
In-Reply-To
<39816f25-5ecf-cbbb-b2c-bf5fdbb6c2fb@dereferenced.org> (view parent)
DKIM signature
missing
Download raw message
> If you have technical evidence to demonstrate that GNU ncurses is superior
> to the NetBSD implementation for Alpine, then please feel free to submit
> those arguments, but bad faith arguments are unhelpful and unwelcome.

https://invisible-island.net/ncurses/ncurses-netbsd.html

-- 
Led.
Ariadne Conill
Details
Message ID
<6bb47924-f023-acf6-87e6-9ab7ff885db@dereferenced.org>
In-Reply-To
<CAAO3vZ91BrPp2rR0nZP1M8P8zMsRiCNJA5+1BZ+PaoE_=2acfg@mail.gmail.com> (view parent)
DKIM signature
missing
Download raw message
Hi,

On Wed, 7 Apr 2021, Led wrote:

>> If you have technical evidence to demonstrate that GNU ncurses is superior
>> to the NetBSD implementation for Alpine, then please feel free to submit
>> those arguments, but bad faith arguments are unhelpful and unwelcome.
>
> https://invisible-island.net/ncurses/ncurses-netbsd.html

Many of the arguments there are poorly formulated, and Roy Marples has 
made rebuttals to the majority of them.  At any rate, I would rather have 
a curses implementation that is cleanly written with asymptotic behavior 
than a highly-optimized curses implementation that is not cleanly written.

Ariadne
Wolf
Details
Message ID
<20210407114719.qcup2wefxby523ez@wolfsden.cz>
In-Reply-To
<e12847f2-4cea-c3e8-84c3-e98b92553f8e@dereferenced.org> (view parent)
DKIM signature
missing
Download raw message
Hello,

On 2021-04-06 12:17:33 -0600, Ariadne Conill wrote:
> * Both libraries implement the POSIX curses API.
> 
> This means, 99% of applications won't need patching and should be basically
> compatible.

(Incomplete) list of differences at [0] seems to contain only
hard-to-debug, easy-to-miss type of things.

> new_item

This will just introduce memory bloat since every string in items will
now be present twice.

> post_menu

Will just not work (and return error no one will likely check) instead
of truncating (which might be fine for most software).

> getnstr

Not sure what the implications here are, but sounds like buffer
overflow.

All these things by themselves are fairly reasonable, however if
software assumes ncurses behaviour, I'm afraid they will be hard to
notice since none of them should lead to hard crash, but instead just to
worse/suboptimal behavior.

Is there a plan to go over whole aports to make sure none of the
programs uses these functions the wrong way for openbsd-curses? What
about new packages? I'm not sure it is possible to have some kind of
linter for this.

W.

[0] https://wiki.netbsd.org/curses_in_netbsd/

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.
Led
Details
Message ID
<CAAO3vZ-RV=8J=cJhZOrT_btcaYRxzx+QxFOF5ah5TO_mGwE8XQ@mail.gmail.com>
In-Reply-To
<20210407114719.qcup2wefxby523ez@wolfsden.cz> (view parent)
DKIM signature
missing
Download raw message
IMHO It looks like a plan to "fix" the unbroken things.

ср, 7 квіт. 2021 о 14:50 Wolf <wolf@wolfsden.cz> пише:
>
> Hello,
>
> On 2021-04-06 12:17:33 -0600, Ariadne Conill wrote:
> > * Both libraries implement the POSIX curses API.
> >
> > This means, 99% of applications won't need patching and should be basically
> > compatible.
>
> (Incomplete) list of differences at [0] seems to contain only
> hard-to-debug, easy-to-miss type of things.
>
> > new_item
>
> This will just introduce memory bloat since every string in items will
> now be present twice.
>
> > post_menu
>
> Will just not work (and return error no one will likely check) instead
> of truncating (which might be fine for most software).
>
> > getnstr
>
> Not sure what the implications here are, but sounds like buffer
> overflow.
>
> All these things by themselves are fairly reasonable, however if
> software assumes ncurses behaviour, I'm afraid they will be hard to
> notice since none of them should lead to hard crash, but instead just to
> worse/suboptimal behavior.
>
> Is there a plan to go over whole aports to make sure none of the
> programs uses these functions the wrong way for openbsd-curses? What
> about new packages? I'm not sure it is possible to have some kind of
> linter for this.
>
> W.
>
> [0] https://wiki.netbsd.org/curses_in_netbsd/
>
> --
> There are only two hard things in Computer Science:
> cache invalidation, naming things and off-by-one errors.



-- 
Led.
Ariadne Conill
Details
Message ID
<222542a7-4875-9620-21aa-e7f169a548f@dereferenced.org>
In-Reply-To
<20210407114719.qcup2wefxby523ez@wolfsden.cz> (view parent)
DKIM signature
missing
Download raw message
Hello,

On Wed, 7 Apr 2021, Wolf wrote:

> Hello,
>
> On 2021-04-06 12:17:33 -0600, Ariadne Conill wrote:
>> * Both libraries implement the POSIX curses API.
>>
>> This means, 99% of applications won't need patching and should be basically
>> compatible.
>
> (Incomplete) list of differences at [0] seems to contain only
> hard-to-debug, easy-to-miss type of things.
>
>> new_item
>
> This will just introduce memory bloat since every string in items will
> now be present twice.

I don't think the impact of this is that heavy, and having multiple copies 
of a string is preferable from a security PoV, as it means there's not 
multiple places where a single string can be corrupted with a write.

>> post_menu
>
> Will just not work (and return error no one will likely check) instead
> of truncating (which might be fine for most software).

This should probably be fine.

>> getnstr
>
> Not sure what the implications here are, but sounds like buffer
> overflow.
>
> All these things by themselves are fairly reasonable, however if
> software assumes ncurses behaviour, I'm afraid they will be hard to
> notice since none of them should lead to hard crash, but instead just to
> worse/suboptimal behavior.
>
> Is there a plan to go over whole aports to make sure none of the
> programs uses these functions the wrong way for openbsd-curses? What
> about new packages? I'm not sure it is possible to have some kind of
> linter for this.

It is not possible to have a linter, but as previously noted, other 
distributions have adopted this curses implementation and have largely not 
had problems.

Ariadne
Reply to thread Export thread (mbox)