Subject: [3.15] System change proposal: Use netbsd-curses instead of GNU ncurses

From: Ariadne Conill <[email-masked]>
Date: Tue, 6 Apr 2021 12:17:33 -0600 (MDT)

Hello,

This proposal is largely centered around normalizing our package set with 
other musl distributions, such as Sabotage and Adelie, which already ship 
netbsd-curses in favor of GNU ncurses.

## GNU ncurses vs netbsd-curses

* GNU ncurses is presently the standard curses library on Linux systems.

Some applications may access private mechanisms inside the curses 
implementation and need patching.

* Both libraries implement the POSIX curses API.

This means, 99% of applications won't need patching and should be 
basically compatible.

* ncurses ships a huge terminfo database.

Ncurses presently ships a database which takes 7MiB on disk.  We split 
this into `ncurses-terminfo` and `ncurses-terminfo-base` to reduce the 
installed size to only 216KiB by default.

netbsd-curses precompiles the terminfo database into a small CDB format 
database.

* ncurses is large.

At present, we try to avoid using curses support in programs which 
optionally use it, because ncurses is a heavy dependency (ncurses-libs is 
500KiB, plus the 216KiB base terminfo database, plus 284KiB for tools). 
netbsd-curses is only 1.3MiB on disk for everything and can be split in 
the same way to reduce size.

* GNU ncurses code quality is lacking.

GNU ncurses is littered with macros and other layers of preprocessing. 
This makes it hard to understand the source code, which makes hotfixing 
security vulnerabilities a potential challenge.

netbsd-curses is written in straightforward C89/C99, one of the preferred 
languages for tools and base system components in Alpine.  It is much 
easier to hack on than GNU ncurses.

Simpler code also makes for more portable code in general.

## Benefits to Alpine

Alpine benefits from switching to a curses implementation that is 
actively developed on musl systems, in addition to the other benefits 
outlined above.

## Contingency Plan

We will do an experimental rebuild with netbsd-curses-dev providing 
ncurses-dev.  If this goes well, then we will switch to netbsd-curses.

If there is a problem, then we will try to fix netbsd-curses first, 
failing that, we will revert back to GNU ncurses.

Ariadne

---

From: Max Rees <[email-masked]>
Date: Tue, 6 Apr 2021 16:30:23 -0400

On 4/6/21 2:17 PM, Ariadne Conill wrote:
> Hello,
> 
> This proposal is largely centered around normalizing our package set 
> with other musl distributions, such as Sabotage and Adelie, which 
> already ship netbsd-curses in favor of GNU ncurses.

As a point of order, Adélie currently does not use NetBSD curses - we 
had been planning to switch at some point I believe (which is probably 
where you got the idea), but we are still on ncurses at the moment.

The rest of your proposal sounds sound to me and in alignment with what 
we had (informally) discussed at Adélie.

Max "[[sroracle]]" Rees

---

From: Led <[email-masked]>
Date: Tue, 6 Apr 2021 23:07:41 +0300

> netbsd-curses in favor of GNU ncurses.

Why are you trying to harm Alpine? Because you hate GNU? Why do you hate GNU?

---

From: Ariadne Conill <[email-masked]>
Date: Tue, 6 Apr 2021 15:06:21 -0600 (MDT)

Hello,

On Tue, 6 Apr 2021, Led wrote:

> Why are you trying to harm Alpine?

The proposal outlines the benefits and risks to Alpine.  By using a better 
maintained ncurses implementation that is also simpler and smaller, this 
proposal is in line with the fundamental design values of the project: 
Small, Simple and Secure.  Or in other words, Alpine is improved by this 
proposal.

> Because you hate GNU? Why do you hate GNU?

These are bad faith arguments.  My desire is to ensure Alpine ships the 
best possible implementations that are aligned with our design values. 
Unfortunately, those implementations are increasingly not GNU ones, 
because of the way the GNU project approaches portability and other 
concerns.

If you have technical evidence to demonstrate that GNU ncurses is superior 
to the NetBSD implementation for Alpine, then please feel free to submit 
those arguments, but bad faith arguments are unhelpful and unwelcome.

Ariadne

---

From: Led <[email-masked]>
Date: Wed, 7 Apr 2021 00:44:52 +0300

> those arguments, but bad faith arguments are unhelpful and unwelcome.

https://invisible-island.net/ncurses/ncurses-netbsd.html

-- 
Led.

---

From: Ariadne Conill <[email-masked]>
Date: Tue, 6 Apr 2021 16:12:02 -0600 (MDT)

Hi,

> https://invisible-island.net/ncurses/ncurses-netbsd.html

Many of the arguments there are poorly formulated, and Roy Marples has 
made rebuttals to the majority of them.  At any rate, I would rather have 
a curses implementation that is cleanly written with asymptotic behavior 
than a highly-optimized curses implementation that is not cleanly written.

Ariadne

---

From: Led <[email-masked]>
Date: Tue, 6 Apr 2021 22:59:32 +0300

> netbsd-curses in favor of GNU ncurses.

Why are you trying to harm Alpine? Because you hate GNU? Why do you hate
GNU?

---

From: maxice8 <[email-masked]>
Date: Tue, 06 Apr 2021 18:39:12 -0300

On Tue, 2021-04-06 at 12:17 -0600, Ariadne Conill wrote:
> with 

Hello

+1

Feel free to ping if help is necessary in anything related to aports

Regards
Leo

---

From: Wolf <[email-masked]>
Date: Wed, 7 Apr 2021 13:47:19 +0200

Hello,

On 2021-04-06 12:17:33 -0600, Ariadne Conill wrote:
> This means, 99% of applications won't need patching and should be basically
> compatible.

(Incomplete) list of differences at [0] seems to contain only
hard-to-debug, easy-to-miss type of things.

> new_item

This will just introduce memory bloat since every string in items will
now be present twice.

> post_menu

Will just not work (and return error no one will likely check) instead
of truncating (which might be fine for most software).

> getnstr

Not sure what the implications here are, but sounds like buffer
overflow.

All these things by themselves are fairly reasonable, however if
software assumes ncurses behaviour, I'm afraid they will be hard to
notice since none of them should lead to hard crash, but instead just to
worse/suboptimal behavior.

Is there a plan to go over whole aports to make sure none of the
programs uses these functions the wrong way for openbsd-curses? What
about new packages? I'm not sure it is possible to have some kind of
linter for this.

W.

[0] https://wiki.netbsd.org/curses_in_netbsd/

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

---

From: Led <[email-masked]>
Date: Wed, 7 Apr 2021 18:33:10 +0300

IMHO It looks like a plan to "fix" the unbroken things.

-- 
Led.

---

From: Ariadne Conill <[email-masked]>
Date: Wed, 7 Apr 2021 10:32:15 -0600 (MDT)

Hello,

> This will just introduce memory bloat since every string in items will

I don't think the impact of this is that heavy, and having multiple copies 
of a string is preferable from a security PoV, as it means there's not 
multiple places where a single string can be corrupted with a write.

> of truncating (which might be fine for most software).

This should probably be fine.

> about new packages? I'm not sure it is possible to have some kind of

It is not possible to have a linter, but as previously noted, other 
distributions have adopted this curses implementation and have largely not 
had problems.

Ariadne