MIME-Version: 1.0
From: Roman Shaposhnik <roman@zededa.com>
Date: Mon, 17 May 2021 11:45:46 -0700
Message-ID: <CAMmSBy-3=MYWhjSVJ1aiMy01mNZxNkERxfvFUpAs6Uwf0sdFgw@mail.gmail.com>
Subject: Proposal for next steps in support of riscv64 in Alpine
To: ~alpine/devel@lists.alpinelinux.org
Content-Type: text/plain; charset="UTF-8"

Hi!

on advice from the other thread I've completed the evaluation
and porting of the main part of aports. The good news is that
out of 1529 packages in main only 50 of them didn't build (the list
can be found at the bottom of this email). Interestingly enough,
some of them (at least about 20) didn't build because a download
link for the source code appears to be down -- not sure what's
up with that.

The rest of the packages (all 5Gb of them) are now uploaded to:
  https://eve-alpine-packages.s3.amazonaws.com/edge/{main,community,testing}/riscv64
This includes pretty much all of main and quite a bit of community
and testing.

I'm still doing light testing but so far it looks good. At this point, I'd
like to propose that we consider including official support of riscv64
platform into some future release of Alpine Linux. I'd be willing to
do most of the lift here, but obviously we need to decide:
   1. What release that may be
   2. What amount of build bandwidth (in terms of hardware) can be
dedicated to this

I'll leave y'all to comment on #1, and I will only point out on #2 that
while my packages at the AWS link above are OK for casual use,
I'd be the first one to say it makes me very uncomfortable to have
any official packages to be built on "some dude's box". If we all agree
that the packages need to be built on the official Alpine infrastructure
then the ask on that side will be pretty moderate: in my experiments,
you can comfortably build on a 16-core configuration and while it takes
a long time to build all of main (about 5 days) the load, on average, stays
at about 600% (this is with qemu-user).

Now, for the meatiest part of discussion: the changes I had to make to
aports to make it happen. Here they are all squashed into a single commit
to make it appear in the same place:
    https://github.com/rvs/aports/commit/e2f5df6681d325510d181d8cc6d6d4f8b39998e1

I'd like to start pushing those into Alpine's master and will start
preparing MRs
this week (please let me know if there are any gotchas I need to be aware
of -- since this will be my first time doing MRs for Alpine).

The MRs will be bucketed in roughly X categories:
    0. Pretty non-controversial stuff that is basically around improving
    core Alpine code to handle riscv64 (like updates to bootstrap.sh
    or scripts in general). Same goes for missing riscv64 in case
    statements and such.

    1. Tests (AKA check section) that are failing on riscv64. The bad
     news is that there's a handful of these, the good news is that in
     almost all the cases it only comes down to a few tests in the suite
     failing and those almost always are failing b/c qemu-user is leaking
     some abstraction (like lookups in /proc/self or some suh). I feel that
     it would be fair to just disable that check altogether (as is done for
     a lot of other architectures like ppc, etc.). Holler if you disagree -- but
     I honestly don't think we have resources to fix all the tests upfront.
     Also making riscv64 Alpine available to general public seems important
     enough to live with a few of those !check statements.

     2. bundled config.sub not being recent enough to recognize riscv64.
     The workaround I used for now is:
               cp /usr/share/abuild/config.sub .
      in the build() function. On top of that:
            2.1. I can make it conditional based on riscv64
            2.2. I can, possibly (if there's a lot desire), turn it into patches
                   against bundled config.sub. This, however will increase
                   the maintenance cost and is effectively equivalent to the
                   above cp statement anyway.

      3. There's a few packages that come out with TEXTRELs and thus
      fail the postcheck() by default. They seem to be fully functional
      otherwise. For now, I'm proposing to disable the postcheck based
      on riscv64 case statement. Please let me know if there's a fundamental
      disagreement with this approach.

       4. There's an issue with gcc toolchain around libatomic that still
       affect a few packages:
https://github.com/riscv/riscv-gnu-toolchain/issues/183#issuecomment-253721765
       For now I'm just planning to conditionally pass an equivalent
       of -latomic to whatever needs to see it.

       5. Newer autotools seem to be more strict in what it accepts as input.
       Hence a few packages that run autoreconf -vif or an equivalent need
       to be lightly patched when it comes to their configure.ac and such.

       6. luajit, nodejs and java are still not available on riscv64
-- hence all
       the packages that depend on those will be turned off. Luajit is
in particularly
       interesting since packages don't really depend on it so much as they just
       want to test with it. I propose we make that dependency optional based
       on riscv64.

        7. Finally, for whatever reason sha512 sums have changed for downloaded
        artifacts (a good cluster is around acf-* packages). Not sure
what's up with
        that, but it surely doesn't seem riscv64 related (and btw, all
of them built fine
        when I manually updated the checksums).

Thanks,
Roman.

1/50 1436/1486 main/asterisk 18.2.2-r0
2/50 1437/1486 main/acf-asterisk 0.7.0-r4
3/50 1438/1486 main/lua-feedparser 0.71-r3
4/50 1439/1486 main/logcheck 1.3.22-r0
5/50 1440/1486 main/hylafaxplus 7.0.3-r1
6/50 1441/1486 main/quagga 1.2.4-r5
7/50 1442/1486 main/acf-quagga 0.10.1-r3
8/50 1443/1486 main/kamailio 5.4.5-r0
9/50 1444/1486 main/acf-kamailio 0.10.0-r4
10/50 1445/1486 main/pdnsd 1.2.9a-r5
11/50 1446/1486 main/dmvpn 1.4.0-r0
12/50 1447/1486 main/hylafax 6.0.7-r3
13/50 1448/1486 main/rp-pppoe 3.14-r0
14/50 1449/1486 main/freeradius 3.0.21-r8
15/50 1450/1486 main/acf-freeradius3 0.3.1-r2
16/50 1451/1486 main/acf-clamav 0.8.0-r4
17/50 1452/1486 main/dhcp 4.4.2-r3
18/50 1453/1486 main/acf-dhcp 0.9.1-r2
19/50 1454/1486 main/lxc 4.0.8-r0
20/50 1455/1486 main/lua-lxc 3.0.2-r1
21/50 1456/1486 main/nagios-plugins 2.3.2-r2
22/50 1457/1486 main/sofia-sip 1.13.3-r0
23/50 1458/1486 main/freeswitch 1.10.6-r0
24/50 1459/1486 main/rsyslog 8.2012.0-r0
25/50 1460/1486 main/bacula 9.6.7-r1
26/50 1461/1486 main/smokeping 2.7.3-r5
27/50 1462/1486 main/openvpn-auth-ldap 2.0.4-r3
28/50 1463/1486 main/apache-mod-auth-radius 1.6.0-r0
29/50 1464/1486 main/xtables-addons-lts 5.10.32-r0
30/50 1465/1486 main/sipp 3.6.0-r0
31/50 1466/1486 main/lua-resty-lrucache 0.09-r1
32/50 1467/1486 main/linux-lts 5.12.0-r0
33/50 1468/1486 main/squid 5.0.5-r0
34/50 1469/1486 main/spice 0.14.3-r0
35/50 1470/1486 main/procps 3.3.17-r0
36/50 1471/1486 main/bash-completion 2.11-r3
37/50 1472/1486 main/dovecot 2.3.14-r0
38/50 1473/1486 main/dconf 0.40.0-r0
39/50 1474/1486 main/pam-pgsql 0.7.3.2-r1
40/50 1475/1486 main/py3-lxc 3.0.4-r2
41/50 1476/1486 main/psqlodbc 13.00.0000-r0
42/50 1477/1486 main/netcf 0.2.8-r6
43/50 1478/1486 main/acf-dovecot 0.6.0-r4
44/50 1479/1486 main/pound 2.8-r0
45/50 1480/1486 main/lua-resty-core 0.1.21-r0
46/50 1481/1486 main/nginx 1.20.0-r0
47/50 1482/1486 main/acf-freeswitch-vmail 0.6.2-r4
48/50 1483/1486 main/acf-freeswitch 0.8.0-r4
49/50 1484/1486 main/lua-imlib2 0.1-r2
50/50 1485/1486 main/acf-squid 0.11.0-r4
main built:     50
main tried:     50
main deleted:   0
main total built:       1486
main total relevant aports:     1486
main total aports:      1529