31 4

[alpine-devel] Porting Alpine scripts

Tuan M. Hoang
Details
Message ID
<155a8e57d76.f37920c130996.6728092923369604578@flatglobe.org>
Sender timestamp
1467417460
DKIM signature
missing
Download raw message
Hi,

I am trying to create a Docker image of Alpine targeting s390x architecture.

Natanael suggested me Timo's previous work at : http://dev.alpinelinux.org/~tteras/bootstrap/ to bootstrap Alpine on new arch.

I have 2 questions related to this, hope you could shed some light.

Q1:
Running createcross-toolchain.sh, at step 5, building gcc pass 2 : (experimenting with targeting x86)

    $ ABUILD_CONF=$ABUILD_CREATECROSS_CONF BOOTSTRAP=nolibc CTARGET_LIBC=musl abuild -r || return 1
(this makes sure --disable-shared is passed to configure script)

Then at step 6, it introduces a conflict between host's gcc and newly installed/built gcc-pass2-$CTARGET

ERROR: unsatisfiable constraints:
    gcc-5.3.0-r0:
        conflicts: gcc-pass2-i486-alpine-linux-musl-5.2.0-r0[so:libcc1.so.0=0.0.0]
        satisfies: g++-5.3.0-r0[gcc=5.3.0-r0] build-base-0.4-r1[gcc]
    gcc-pass2-i486-alpine-linux-musl-5.2.0-r0:
        conflicts: gcc-5.3.0-r0[so:libcc1.so.0=0.0.0]
        satisfies: world[gcc-pass2-i486-alpine-linux-musl]

How could I solve this? Should I build gcc-pass-2 with different prefix than /usr/ to avoid conflict dynamic libs? If I do so, will it be any different than a manually/traditionally built cross-toolchain ? (mentioned below)


Q2:
I have built a cross toolchains targeting s390x (gcc, musl, binutils, busybox, apk-tools), based on Gregor Richards's musl-cross with some patches.
If I specifies this cross toolchains in the crossbuild-alpine-bootstrap.sh , rather than doing
    ${SUDO_APK} --repository $REPODEST_HOST/main add gcc-$CTARGET g++-$CTARGET gcc-gnat-$CTARGET || return 1

then will there be any differences ? I have not try this (havent have cross-built gnat yet) and just want to know what you think about it.

Thank you,
Tuan



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Justin Cormack
Details
Message ID
<CAK4o1Ww77SRVoTP8kMcaHSHWhPwn+4oZ+F9SYF6vJ1xMk386qw@mail.gmail.com>
In-Reply-To
<155a8e57d76.f37920c130996.6728092923369604578@flatglobe.org> (view parent)
Sender timestamp
1467524480
DKIM signature
missing
Download raw message
I was not aware of a Musl s390x port, is there one?

Justin
On 1 Jul 2016 4:57 p.m., "Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> Hi,
>
> I am trying to create a Docker image of Alpine targeting s390x
> architecture.
>
> Natanael suggested me Timo's previous work at :
> http://dev.alpinelinux.org/~tteras/bootstrap/ to bootstrap Alpine on new
> arch.
>
> I have 2 questions related to this, hope you could shed some light.
>
> Q1:
> Running createcross-toolchain.sh, at step 5, building gcc pass 2 :
> (experimenting with targeting x86)
>
>     $ ABUILD_CONF=$ABUILD_CREATECROSS_CONF BOOTSTRAP=nolibc
> CTARGET_LIBC=musl abuild -r || return 1
> (this makes sure --disable-shared is passed to configure script)
>
> Then at step 6, it introduces a conflict between host's gcc and newly
> installed/built gcc-pass2-$CTARGET
>
> ERROR: unsatisfiable constraints:
>     gcc-5.3.0-r0:
>         conflicts:
> gcc-pass2-i486-alpine-linux-musl-5.2.0-r0[so:libcc1.so.0=0.0.0]
>         satisfies: g++-5.3.0-r0[gcc=5.3.0-r0] build-base-0.4-r1[gcc]
>     gcc-pass2-i486-alpine-linux-musl-5.2.0-r0:
>         conflicts: gcc-5.3.0-r0[so:libcc1.so.0=0.0.0]
>         satisfies: world[gcc-pass2-i486-alpine-linux-musl]
>
> How could I solve this? Should I build gcc-pass-2 with different prefix
> than /usr/ to avoid conflict dynamic libs? If I do so, will it be any
> different than a manually/traditionally built cross-toolchain ? (mentioned
> below)
>
>
> Q2:
> I have built a cross toolchains targeting s390x (gcc, musl, binutils,
> busybox, apk-tools), based on Gregor Richards's musl-cross with some
> patches.
> If I specifies this cross toolchains in the crossbuild-alpine-bootstrap.sh
> , rather than doing
>     ${SUDO_APK} --repository $REPODEST_HOST/main add gcc-$CTARGET
> g++-$CTARGET gcc-gnat-$CTARGET || return 1
>
> then will there be any differences ? I have not try this (havent have
> cross-built gnat yet) and just want to know what you think about it.
>
> Thank you,
> Tuan
>
>
>
> ---
> Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> Help:         alpine-devel+help@lists.alpinelinux.org
> ---
>
>
Tuan M. Hoang
Details
Message ID
<155afe2764a.10cb9409959067.3829969527295319337@flatglobe.org>
In-Reply-To
<CAK4o1Ww77SRVoTP8kMcaHSHWhPwn+4oZ+F9SYF6vJ1xMk386qw@mail.gmail.com> (view parent)
Sender timestamp
1467534702
DKIM signature
missing
Download raw message
Hi,

The experimental Musl s390x port is at : https://github.com/koorogi/musl/tree/s390x

Tuan

 ---- On Sun, 03 Jul 2016 12:41:20 +0700 Justin Cormack <justin@specialbusservice.com> wrote ---- 
 > I was not aware of a Musl s390x port, is there one?
 >  Justin
 >  
 >  On 1 Jul 2016 4:57 p.m., "Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:
 >  Hi,
 >  
 >  I am trying to create a Docker image of Alpine targeting s390x architecture.
 >  
 >  Natanael suggested me Timo's previous work at : http://dev.alpinelinux.org/~tteras/bootstrap/ to bootstrap Alpine on new arch.
 >  
 >  I have 2 questions related to this, hope you could shed some light.
 >  
 >  Q1:
 >  Running createcross-toolchain.sh, at step 5, building gcc pass 2 : (experimenting with targeting x86)
 >  
 >      $ ABUILD_CONF=$ABUILD_CREATECROSS_CONF BOOTSTRAP=nolibc CTARGET_LIBC=musl abuild -r || return 1
 >  (this makes sure --disable-shared is passed to configure script)
 >  
 >  Then at step 6, it introduces a conflict between host's gcc and newly installed/built gcc-pass2-$CTARGET
 >  
 >  ERROR: unsatisfiable constraints:
 >      gcc-5.3.0-r0:
 >          conflicts: gcc-pass2-i486-alpine-linux-musl-5.2.0-r0[so:libcc1.so.0=0.0.0]
 >          satisfies: g++-5.3.0-r0[gcc=5.3.0-r0] build-base-0.4-r1[gcc]
 >      gcc-pass2-i486-alpine-linux-musl-5.2.0-r0:
 >          conflicts: gcc-5.3.0-r0[so:libcc1.so.0=0.0.0]
 >          satisfies: world[gcc-pass2-i486-alpine-linux-musl]
 >  
 >  How could I solve this? Should I build gcc-pass-2 with different prefix than /usr/ to avoid conflict dynamic libs? If I do so, will it be any different than a manually/traditionally built cross-toolchain ? (mentioned below)
 >  
 >  
 >  Q2:
 >  I have built a cross toolchains targeting s390x (gcc, musl, binutils, busybox, apk-tools), based on Gregor Richards's musl-cross with some patches.
 >  If I specifies this cross toolchains in the crossbuild-alpine-bootstrap.sh , rather than doing
 >      ${SUDO_APK} --repository $REPODEST_HOST/main add gcc-$CTARGET g++-$CTARGET gcc-gnat-$CTARGET || return 1
 >  
 >  then will there be any differences ? I have not try this (havent have cross-built gnat yet) and just want to know what you think about it.
 >  
 >  Thank you,
 >  Tuan
 >  
 >  
 >  
 >  ---
 >  Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
 >  Help:         alpine-devel+help@lists.alpinelinux.org
 >  ---
 >  
 >  



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20160703225319.0ce46f62@vostro>
In-Reply-To
<155a8e57d76.f37920c130996.6728092923369604578@flatglobe.org> (view parent)
Sender timestamp
1467575598
DKIM signature
missing
Download raw message
On Sat, 02 Jul 2016 06:57:40 +0700
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> I am trying to create a Docker image of Alpine targeting s390x
> architecture.
> 
> Natanael suggested me Timo's previous work at :
> http://dev.alpinelinux.org/~tteras/bootstrap/ to bootstrap Alpine on
> new arch.
> 
> I have 2 questions related to this, hope you could shed some light.
> 
> Q1:
> Running createcross-toolchain.sh, at step 5, building gcc pass 2 :
> (experimenting with targeting x86)
> 
>     $ ABUILD_CONF=$ABUILD_CREATECROSS_CONF BOOTSTRAP=nolibc
> CTARGET_LIBC=musl abuild -r || return 1 (this makes sure
> --disable-shared is passed to configure script)
> 
> Then at step 6, it introduces a conflict between host's gcc and newly
> installed/built gcc-pass2-$CTARGET
> 
> ERROR: unsatisfiable constraints:
>     gcc-5.3.0-r0:
>         conflicts:
> gcc-pass2-i486-alpine-linux-musl-5.2.0-r0[so:libcc1.so.0=0.0.0]
> satisfies: g++-5.3.0-r0[gcc=5.3.0-r0] build-base-0.4-r1[gcc]
> gcc-pass2-i486-alpine-linux-musl-5.2.0-r0: conflicts:
> gcc-5.3.0-r0[so:libcc1.so.0=0.0.0] satisfies:
> world[gcc-pass2-i486-alpine-linux-musl]
> 
> How could I solve this? Should I build gcc-pass-2 with different
> prefix than /usr/ to avoid conflict dynamic libs? If I do so, will it
> be any different than a manually/traditionally built
> cross-toolchain ? (mentioned below)

Sounds like your aports is from edge, but you are running 3.3 or 3.2
stable alpine. Then the gcc versions do not match, and you get this.

> Q2:
> I have built a cross toolchains targeting s390x (gcc, musl, binutils,
> busybox, apk-tools), based on Gregor Richards's musl-cross with some
> patches. If I specifies this cross toolchains in the
> crossbuild-alpine-bootstrap.sh , rather than doing ${SUDO_APK}
> --repository $REPODEST_HOST/main add gcc-$CTARGET g++-$CTARGET
> gcc-gnat-$CTARGET || return 1
> 
> then will there be any differences ? I have not try this (havent have
> cross-built gnat yet) and just want to know what you think about it.

It's probably ok for gcc/g++ part, but you are right, Ada might be
problem for full bootstrap.

As heads up: I'm currently looking at gcc 6.1; and also bootstrapping
armv7. I have already some changes coming up for gcc/musl and some
other packages for fixing bootstrapping. But seems there's been some
bitrot since last bootstrap (for armhf some years ago) and the scripts
need fixing.

Cheers,
Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<155b28d7dfa.ca28fe6d67425.5841089419777612642@flatglobe.org>
In-Reply-To
<20160703225319.0ce46f62@vostro> (view parent)
Sender timestamp
1467579465
DKIM signature
missing
Download raw message
 ---- On Mon, 04 Jul 2016 02:53:18 +0700 Timo Teras <timo.teras@iki.fi> wrote ---- 

 > Sounds like your aports is from edge, but you are running 3.3 or 3.2 
 > stable alpine. Then the gcc versions do not match, and you get this. 
 >  

I am on v3.4 with gcc5.3-r0. Tried gcc5.3-r0 APKBUILD file but the same problem.
 
>From what I understand your gcc-pass-2 and gcc-final are actually gcc-pass-1 and gcc-pass-2 in traditional way : 
    1. build gcc-pass-1 and binutils using host (x86_64) 's gcc+libc targeting new target (x86). 
    2. use that gcc-pass-1 to build new target's libc. 
    3. build gcc-pass-2 using new libc
(please correct me if I am wrong)
So, after building gcc-pass-2 (step 1 above) in the script, we install it in the host system then use it to build musl targeting new target (step 2 above). But if you install gcc-pass-2 in the host system at /usr/, it will produce my conflicts problem. In the traditional way, we just pass something like CROSS_COMPILE=i486-linux-musl- when running musl's configure script. This is why I suggested install gcc-pass-2 into different prefix so it won't conflict with the host's gcc. Just wonder how did you do last time.

I did try to delete host's gcc to install only gcc-pass-2 but building musl requires build-base package and in turn requires host's gcc).

 > It's probably ok for gcc/g++ part, but you are right, Ada might be 
 > problem for full bootstrap. 
 >  
 > As heads up: I'm currently looking at gcc 6.1; and also bootstrapping 
 > armv7. I have already some changes coming up for gcc/musl and some 
 > other packages for fixing bootstrapping. But seems there's been some 
 > bitrot since last bootstrap (for armhf some years ago) and the scripts 
 > need fixing. 
 >  

I also suspect that Alpine has involved a lot since 2013 and your scripts might be outdated to current aports system. Would you mind sharing your current/working bootstrap on present aports system?

Thanks,
Tuan



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20160705172656.788ec089@vostro.util.wtbts.net>
In-Reply-To
<155b28d7dfa.ca28fe6d67425.5841089419777612642@flatglobe.org> (view parent)
Sender timestamp
1467728816
DKIM signature
missing
Download raw message
On Mon, 04 Jul 2016 03:57:45 +0700
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

>  ---- On Mon, 04 Jul 2016 02:53:18 +0700 Timo Teras
> <timo.teras@iki.fi> wrote ---- 
> 
>  > Sounds like your aports is from edge, but you are running 3.3 or
>  > 3.2 stable alpine. Then the gcc versions do not match, and you get
>  > this. 
> 
> I am on v3.4 with gcc5.3-r0. Tried gcc5.3-r0 APKBUILD file but the
> same problem. 
> From what I understand your gcc-pass-2 and gcc-final are actually
> gcc-pass-1 and gcc-pass-2 in traditional way :
> 1. build gcc-pass-1 and binutils using host (x86_64) 's gcc+libc
> targeting new target (x86).
> 2. use that gcc-pass-1 to build new target's libc.
> 3. build gcc-pass-2 using new libc (please correct me if I am wrong)

Well, upon in the beginning it was 3 pass. You need pass 1 to build gcc
without libc. Then you could build glibc headers. Then you build gcc
pass 2 with glibc headers so it targets right c-library, and you can
build full glibc. Finally you can build the final gcc as pass 3. Maybe
this was simplified too.

> So, after building gcc-pass-2 (step 1 above) in the script, we
> install it in the host system then use it to build musl targeting new
> target (step 2 above). But if you install gcc-pass-2 in the host
> system at /usr/, it will produce my conflicts problem. In the
> traditional way, we just pass something like
> CROSS_COMPILE=i486-linux-musl- when running musl's configure script.
> This is why I suggested install gcc-pass-2 into different prefix so
> it won't conflict with the host's gcc. Just wonder how did you do
> last time.
> I did try to delete host's gcc to install only gcc-pass-2 but
> building musl requires build-base package and in turn requires host's
> gcc).

Seems there was bitrot in the cross building support. abuild was
improved, and upstream packaged changed.

I made same fixes to various places now. And will push all aports
changes today/tomorrow. After gcc 6.1 is done.

I'm playing with armv7 target, and got:
 - full cross complier done (createcross-toolchain.sh runs)
 - cross building boot image works up to building openssl; seems
   something changed upstream and needs fixing

I hope to get the scripts / aports' minimal cross-building support to
the point that bootstrapping works again.

/Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20160706172836.74bab5d1@vostro>
In-Reply-To
<20160705172656.788ec089@vostro.util.wtbts.net> (view parent)
Sender timestamp
1467815315
DKIM signature
missing
Download raw message
On Tue, 5 Jul 2016 17:26:56 +0300
Timo Teras <timo.teras@iki.fi> wrote:

> Seems there was bitrot in the cross building support. abuild was
> improved, and upstream packaged changed.
> 
> I made same fixes to various places now. And will push all aports
> changes today/tomorrow. After gcc 6.1 is done.
> 
> I'm playing with armv7 target, and got:
>  - full cross complier done (createcross-toolchain.sh runs)
>  - cross building boot image works up to building openssl; seems
>    something changed upstream and needs fixing
> 
> I hope to get the scripts / aports' minimal cross-building support to
> the point that bootstrapping works again.

I pushed all of my aports changed now, and copied slightly modified
bootstrap scripts out.

On edge host, with edge aports, and abuild from git (needed one minor
fix); I managed to bootstrap armv7 from x86_64 host now.

mkinitfs is not cross-compilable at the moment yet. But basically the
script should mostly work now.

/Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<155e6497a8e.11ad265cf269464.7355344611305011615@flatglobe.org>
In-Reply-To
<20160706172836.74bab5d1@vostro> (view parent)
Sender timestamp
1468447423
DKIM signature
missing
Download raw message
 ---- On Wed, 06 Jul 2016 21:28:35 +0700 Timo Teras <timo.teras@iki.fi> wrote ---- 
 > On Tue, 5 Jul 2016 17:26:56 +0300 
 > Timo Teras <timo.teras@iki.fi> wrote: 
 >  
 > > Seems there was bitrot in the cross building support. abuild was 
 > > improved, and upstream packaged changed. 
 > >  
 > > I made same fixes to various places now. And will push all aports 
 > > changes today/tomorrow. After gcc 6.1 is done. 
 > >  
 > > I'm playing with armv7 target, and got: 
 > >  - full cross complier done (createcross-toolchain.sh runs) 
 > >  - cross building boot image works up to building openssl; seems 
 > >    something changed upstream and needs fixing 
 > >  
 > > I hope to get the scripts / aports' minimal cross-building support to 
 > > the point that bootstrapping works again. 
 >  
 > I pushed all of my aports changed now, and copied slightly modified 
 > bootstrap scripts out. 
 >  
 > On edge host, with edge aports, and abuild from git (needed one minor 
 > fix); I managed to bootstrap armv7 from x86_64 host now. 
 >  
 > mkinitfs is not cross-compilable at the moment yet. But basically the 
 > script should mostly work now. 
 >  
 > /Timo 
 > 

Would you mind sharing your building log ? I just need to make sure my steps are correct.
Thank you.



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<155f440c618.129c9f0aa5155.5728200182511575692@flatglobe.org>
In-Reply-To
<155e6497a8e.11ad265cf269464.7355344611305011615@flatglobe.org> (view parent)
Sender timestamp
1468681733
DKIM signature
missing
Download raw message
Patch: +5 -1
Hi,

I have successfully built armv7 cross-toolchain using createcross-toolchain.sh.

About crossbuild-alpine-bootstrap.sh, I have 2 questions:

1. At cross-building busybox (in the for loop), there was an error of missing linux/kd.h header, even though /usr/include/linux/kd.h is present in my Alpine build machine. So I have to install the newly cross-built linux-headers package from previous loop, like this:

diff --git a/crossbuild-alpine-bootstrap.sh b/crossbuild-alpine-bootstrap.sh
--- a/crossbuild-alpine-bootstrap.sh
+++ b/crossbuild-alpine-bootstrap.sh
@@ -1,6 +1,6 @@
 #!/bin/sh
 
-TARGET_ARCH=aarch64
+export TARGET_ARCH=armv7
 
 export ABUILD_CREATECROSS_CONF=$PWD/abuild-createcross-$TARGET_ARCH.conf
 export ABUILD_CONF=$PWD/abuild-crossbuild-$TARGET_ARCH.conf
@@ -47,6 +47,10 @@ for PKG in linux-headers musl libc-dev \
 	cd $APORTS/main/$PKG
 	BOOTSTRAP=bootimage abuild -r || exit 1
 
+	if [ "$PKG" = "linux-headers" ]; then
+		${SUDO_APK} -u --root "$CBUILDROOT" --repository "$REPODEST_TARGET/main" --arch "$TARGET_ARCH" add linux-headers | exit 1
+	fi
+
 	if [ "$PKG" = "libc-dev" ]; then
 		# install libc-dev, as implicit but mandatory dependency
 		${SUDO_APK} -u --root "$CBUILDROOT" add libc-dev || exit 1

How about your procedures?

2. At cross-building binutils step, I have the error : 

./a.out: cannot execute binary file: Exec format error

I checked the configure log (http://pastebin.com/t9QS9DZZ) and it looks like the build, host, target machines are all 'armv7-alpine-linux-muslgnueabihf', which is incorrect I guess. I think the build machine should be 'x86_64-alpine-linux-musl' because we are using a cross-compiler running on x86_64 targeting armv7. So from its point of view, the build machine is x86_64, it builds program to run on host machine (armv7), and I am not really sure what to pass as target machine, maybe armv7 too.

If I try to pass to binutils configure script with '--build=x86_64-alpine-linux-musl --host=armv7-alpine-linux-muslgnueabihf --target=armv7-alpine-linux-muslgnueabihf ' then it configure and build just fine.

Doing the same strategy with make, it failed.

What do you think ?

3. I am curious about CBUILD variable. Where is it defined? Can't find in your script nor (any) APKBUILD nor APKBUILD reference documentation/webpage. In cross-building binutils case, it is set to 'armv7-alpine-linux-muslgnueabihf', same as CHOST.

By the way, your work is beautiful.
Thank you.


 ---- On Thu, 14 Jul 2016 05:03:43 +0700 Tuan M. Hoang <tmhoang@flatglobe.org> wrote ---- 
 >  
 > Would you mind sharing your building log ? I just need to make sure my steps are correct. 
 > Thank you. 
 >  
 > --- 
 > Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org 
 > Help:         alpine-devel+help@lists.alpinelinux.org 
 > --- 
 >  
 > 



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20160718084038.23773fca@vostro.util.wtbts.net>
In-Reply-To
<155f440c618.129c9f0aa5155.5728200182511575692@flatglobe.org> (view parent)
Sender timestamp
1468820438
DKIM signature
missing
Download raw message
On Sat, 16 Jul 2016 22:08:53 +0700
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> I have successfully built armv7 cross-toolchain using
> createcross-toolchain.sh.

Nice.

> About crossbuild-alpine-bootstrap.sh, I have 2 questions:
> 
> 1. At cross-building busybox (in the for loop), there was an error of
> missing linux/kd.h header, even though /usr/include/linux/kd.h is
> present in my Alpine build machine. So I have to install the newly
> cross-built linux-headers package from previous loop, like this:
> 
> diff --git a/crossbuild-alpine-bootstrap.sh
> b/crossbuild-alpine-bootstrap.sh --- a/crossbuild-alpine-bootstrap.sh
> +++ b/crossbuild-alpine-bootstrap.sh
> @@ -1,6 +1,6 @@
>  #!/bin/sh
>  
> -TARGET_ARCH=aarch64
> +export TARGET_ARCH=armv7
>  
>  export
> ABUILD_CREATECROSS_CONF=$PWD/abuild-createcross-$TARGET_ARCH.conf
> export ABUILD_CONF=$PWD/abuild-crossbuild-$TARGET_ARCH.conf @@ -47,6
> +47,10 @@ for PKG in linux-headers musl libc-dev \ cd
> $APORTS/main/$PKG BOOTSTRAP=bootimage abuild -r || exit 1
>  
> +	if [ "$PKG" = "linux-headers" ]; then
> +		${SUDO_APK} -u --root "$CBUILDROOT" --repository
> "$REPODEST_TARGET/main" --arch "$TARGET_ARCH" add linux-headers |
> exit 1
> +	fi
> +
>  	if [ "$PKG" = "libc-dev" ]; then
>  		# install libc-dev, as implicit but mandatory
> dependency ${SUDO_APK} -u --root "$CBUILDROOT" add libc-dev || exit 1
> 
> How about your procedures?

busybox apkbuild has:
makedepends_host="linux-headers"

So when cross building abuild should have automatically installed this
for you. It does it for me.

> 2. At cross-building binutils step, I have the error : 
> 
> ./a.out: cannot execute binary file: Exec format error
> 
> I checked the configure log (http://pastebin.com/t9QS9DZZ) and it
> looks like the build, host, target machines are all
> 'armv7-alpine-linux-muslgnueabihf', which is incorrect I guess. I
> think the build machine should be 'x86_64-alpine-linux-musl' because
> we are using a cross-compiler running on x86_64 targeting armv7. So
> from its point of view, the build machine is x86_64, it builds
> program to run on host machine (armv7), and I am not really sure what
> to pass as target machine, maybe armv7 too.
> 
> If I try to pass to binutils configure script with
> '--build=x86_64-alpine-linux-musl
> --host=armv7-alpine-linux-muslgnueabihf
> --target=armv7-alpine-linux-muslgnueabihf ' then it configure and
> build just fine.
> 
> Doing the same strategy with make, it failed.
> 
> What do you think ?

Sounds like you figured out the root of the problem. CBUILD needs to be
the build host's triplet, or things will not work.

Usually it is configured in /etc/abuild.conf, and that's what my
scripts expect. E.g. my abuild-crossbuild-armv7.conf has:
  # inherit CBUILD, and other basic defines
  source /etc/abuild.conf

It expects CBUILD=<system triplet> to be in /etc/abuild.conf.

This is also used by abuild to deduce if cross building or not, and
probably caused the problem #1 too.


> 3. I am curious about CBUILD variable. Where is it defined? Can't
> find in your script nor (any) APKBUILD nor APKBUILD reference
> documentation/webpage. In cross-building binutils case, it is set to
> 'armv7-alpine-linux-muslgnueabihf', same as CHOST.

See above.

> By the way, your work is beautiful.

Thank you.

We managed to bootstrap aarch64 recently. We have it running on qemu
and on real hw too. I believe when it comes up, it could be used for
aarch64 and possibly armv7 builds.

/Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<1560fbd5a78.dbf5aa7b48820.4632348656468847927@flatglobe.org>
In-Reply-To
<20160718084038.23773fca@vostro.util.wtbts.net> (view parent)
Sender timestamp
1469142882
DKIM signature
missing
Download raw message
 ---- On Mon, 18 Jul 2016 12:40:38 +0700 Timo Teras <timo.teras@iki.fi> wrote ---- 

 > Usually it is configured in /etc/abuild.conf, and that's what my 
 > scripts expect. E.g. my abuild-crossbuild-armv7.conf has: 
 >   # inherit CBUILD, and other basic defines 
 >   source /etc/abuild.conf 
 >  
 > It expects CBUILD=<system triplet> to be in /etc/abuild.conf. 
 >  
 > This is also used by abuild to deduce if cross building or not, and 
 > probably caused the problem #1 too. 
 >  

Problem 1 : (to keeptrack easier)
This is the content of my /etc/abuild.conf : http://pastebin.com/Pq6vYpvD
So strange that there is no trace of CBUILD variable, even CARCH, CTARGET_ARCH. I keep getting CARCH=unknown error. Then I made some modification as in : https://github.com/tmh1999/alpine-bootstrap-s390x/commit/2aadde7982a9ba679bd01734705321d9dde76d97

AFAIK, CARCH is the build machine architecture (x86_64) and you get it by running $ abuild -A. But in my modification, it is armv7.

 > We managed to bootstrap aarch64 recently. We have it running on qemu 
 > and on real hw too. I believe when it comes up, it could be used for 
 > aarch64 and possibly armv7 builds. 

Sounds great !


Question 1 :
Until now, I also have successfully built a cross-compiler targeting s390x architecture, with above modification to the script and few more modification in aports/main/gcc/APKBUILD and aports/main/musl/APKBUILD files ( I use the experimental s390x port of musl ). I know this is not your problem but I would like to ask you since you are more experienced in this problem than me :)

This is the modification I made to aports/main/gcc/APKBUILD file :

--- a/main/gcc/APKBUILD
+++ b/main/gcc/APKBUILD
@@ -392,7 +427,9 @@ libatomic() {
         mkdir -p "$subpkgdir"/usr/lib
         mv "$pkgdir"/usr/lib/libatomic.so.* "$subpkgdir"/usr/lib/ || \
                 cp -a "$pkgdir"/usr/$CTARGET/lib/libatomic.so.* "$subpkgdir"/usr/lib/ || \
+                cp -a "$pkgdir"/usr/$CTARGET/lib64/libatomic.so.* "$subpkgdir"/usr/lib/ || \
                 return 1
 }
 
 libcxx() {
@@ -403,7 +440,9 @@ libcxx() {
 	mkdir -p "$subpkgdir"/usr/lib
 	mv "$pkgdir"/usr/lib/libstdc++.so.* "$subpkgdir"/usr/lib/ || \
 		cp -a "$pkgdir"/usr/$CTARGET/lib/libstdc++.so.* "$subpkgdir"/usr/lib/ || \
+		cp -a "$pkgdir"/usr/$CTARGET/lib64/libstdc++.so.* "$subpkgdir"/usr/lib/ || \
 		return 1
 }
 
 gpp() {
@@ -452,7 +491,9 @@ libgcc() {
 	mkdir -p "$subpkgdir"/usr/lib
 	mv "$pkgdir"/usr/lib/libgcc_s.so.* "$subpkgdir"/usr/lib/ || \
 		cp -a "$pkgdir"/usr/$CTARGET/lib/libgcc_s.so.* "$subpkgdir"/usr/lib/ || \
+		cp -a "$pkgdir"/usr/$CTARGET/lib64/libgcc_s.so.* "$subpkgdir"/usr/lib/ || \
 		return 1
 }

>From what I understand, when we --disable-multilib when configuring gcc, lib/ and lib64/ wouldn't be existed at the same time. But in my case they are all there and the *.so files are in lib64/ directory. This workaround helps the creatcross-toolchain.sh but when running crossbuild-toolchain.sh I faced a lot of failures from this lib/, lib64/ thing and the 'unknown' described above. What do you think ?


Question 2:

So when you started porting armv7 and aarch, which files in the aports tree you had to make changes ? Theoretically I can check in the git log but that would be a lot, I think :(

Thank you.



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20160722084514.7c01123f@vostro.util.wtbts.net>
In-Reply-To
<1560fbd5a78.dbf5aa7b48820.4632348656468847927@flatglobe.org> (view parent)
Sender timestamp
1469166314
DKIM signature
missing
Download raw message
On Fri, 22 Jul 2016 06:14:42 +0700
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

>  ---- On Mon, 18 Jul 2016 12:40:38 +0700 Timo Teras
> <timo.teras@iki.fi> wrote ---- 
> 
>  > Usually it is configured in /etc/abuild.conf, and that's what my 
>  > scripts expect. E.g. my abuild-crossbuild-armv7.conf has: 
>  >   # inherit CBUILD, and other basic defines 
>  >   source /etc/abuild.conf 
>  >  
>  > It expects CBUILD=<system triplet> to be in /etc/abuild.conf. 
>  >  
>  > This is also used by abuild to deduce if cross building or not,
>  > and probably caused the problem #1 too. 
>  >    
> 
> Problem 1 : (to keeptrack easier)
> This is the content of my /etc/abuild.conf :
> http://pastebin.com/Pq6vYpvD So strange that there is no trace of
> CBUILD variable, even CARCH, CTARGET_ARCH.

My scripts expect that you add CBUILD there. Though it's trivial to get
it from $(gcc -dumpmachine), I think that should be added to abuild.
I'll probably do that soon. I think most of the environment variables
set the cross .conf files should in fact be set in abuild already. I'll
look in to doing that as well.

Though, we are currently looking into improving the build system a bit
more to support cross-building better, including supporting it directly
from abuild command line and building things automatically in chroot.
This needs a bit of design, but is on the way.

> I keep getting CARCH=unknown error. Then I made some modification as
> in :
> https://github.com/tmh1999/alpine-bootstrap-s390x/commit/2aadde7982a9ba679bd01734705321d9dde76d97
>
> AFAIK, CARCH is the build machine architecture (x86_64) and you get
> it by running $ abuild -A. But in my modification, it is armv7.

abuild will deduce CARCH from CHOST. You need to add s390x support to
abuild. See commit:
http://git.alpinelinux.org/cgit/abuild/commit/?id=580dd46c588ef71bc852d2b9a5bb4426a2c47665
 
> Question 1 :
> Until now, I also have successfully built a cross-compiler targeting
> s390x architecture, with above modification to the script and few
> more modification in aports/main/gcc/APKBUILD and
> aports/main/musl/APKBUILD files ( I use the experimental s390x port
> of musl ). I know this is not your problem but I would like to ask
> you since you are more experienced in this problem than me :)
> 
> This is the modification I made to aports/main/gcc/APKBUILD file :
> 
> --- a/main/gcc/APKBUILD
> +++ b/main/gcc/APKBUILD
> @@ -392,7 +427,9 @@ libatomic() {
>          mkdir -p "$subpkgdir"/usr/lib
>          mv "$pkgdir"/usr/lib/libatomic.so.* "$subpkgdir"/usr/lib/ ||
> \ cp -a "$pkgdir"/usr/$CTARGET/lib/libatomic.so.*
> "$subpkgdir"/usr/lib/ || \
> +                cp -a "$pkgdir"/usr/$CTARGET/lib64/libatomic.so.*
> "$subpkgdir"/usr/lib/ || \ return 1
>  }
>  
>  libcxx() {
> @@ -403,7 +440,9 @@ libcxx() {
>  	mkdir -p "$subpkgdir"/usr/lib
>  	mv "$pkgdir"/usr/lib/libstdc++.so.* "$subpkgdir"/usr/lib/ ||
> \ cp -a "$pkgdir"/usr/$CTARGET/lib/libstdc++.so.*
> "$subpkgdir"/usr/lib/ || \
> +		cp -a "$pkgdir"/usr/$CTARGET/lib64/libstdc++.so.*
> "$subpkgdir"/usr/lib/ || \ return 1
>  }
>  
>  gpp() {
> @@ -452,7 +491,9 @@ libgcc() {
>  	mkdir -p "$subpkgdir"/usr/lib
>  	mv "$pkgdir"/usr/lib/libgcc_s.so.* "$subpkgdir"/usr/lib/ || \
>  		cp -a "$pkgdir"/usr/$CTARGET/lib/libgcc_s.so.*
> "$subpkgdir"/usr/lib/ || \
> +		cp -a "$pkgdir"/usr/$CTARGET/lib64/libgcc_s.so.*
> "$subpkgdir"/usr/lib/ || \ return 1
>  }
> 
> From what I understand, when we --disable-multilib when configuring
> gcc, lib/ and lib64/ wouldn't be existed at the same time. But in my
> case they are all there and the *.so files are in lib64/ directory.
> This workaround helps the creatcross-toolchain.sh but when running
> crossbuild-toolchain.sh I faced a lot of failures from this lib/,
> lib64/ thing and the 'unknown' described above. What do you think ?

The lib64 is gcc default, and on x86_64 and aarch64 it's similar. We
patch the lib location for these architectures, see:
http://git.alpinelinux.org/cgit/aports/tree/main/gcc/gcc-pure64.patch

You probably need to add similar change for gcc/config/s390/t-linux64.

> Question 2:
> 
> So when you started porting armv7 and aarch, which files in the
> aports tree you had to make changes ? Theoretically I can check in
> the git log but that would be a lot, I think :(

Not too many. You probably need (aarch64 commit referred):

- abuild to recognize the triplet (the commit refferred earlier)
  http://git.alpinelinux.org/cgit/abuild/commit/?id=580dd46c588ef71bc852d2b9a5bb4426a2c47665
  
- apk-tools to use the same arch name
  http://git.alpinelinux.org/cgit/apk-tools/commit/?id=06ae5fdfdccd0c8e6d5501d93666bd915d2604d1

- gcc to be configured correct for the arch
  http://git.alpinelinux.org/cgit/aports/commit/main/gcc?id=db1d2c069e66b3bf9962d76b0b48bfdcfc8d0384
  (the ada change is generic, you'll need the _arch_configure +
   gcc-pure64.patch update)

And update APKBUILDs for the packages that are arch aware; for
minimal bootstrap it includes:

- linux-headers
  http://git.alpinelinux.org/cgit/aports/commit/?id=f483f49692b97950b680d987fe31f8ddb9277842

- openssl
  http://git.alpinelinux.org/cgit/aports/commit/?id=896f06e3b0b07200957d0389e2156d0ee4e099cb

- linux-vanilla (or the kernel you need, including creating kernel configuration)
  http://git.alpinelinux.org/cgit/aports/commit/?id=baa04a75170112f9713dd5ad72a5732f9bfaeaac

Some packages might also need other kind of patching, e.g. if configure
does not recognize the new triplet.

Cheers,
Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<15614c77014.1200f0e697958.371787825394349914@flatglobe.org>
In-Reply-To
<20160722084514.7c01123f@vostro.util.wtbts.net> (view parent)
Sender timestamp
1469227429
DKIM signature
missing
Download raw message
 ---- On Fri, 22 Jul 2016 12:45:14 +0700 Timo Teras <timo.teras@iki.fi> wrote ---- 
 >  
 > My scripts expect that you add CBUILD there. Though it's trivial to get 
 > it from $(gcc -dumpmachine), I think that should be added to abuild. 
 > I'll probably do that soon. I think most of the environment variables 
 > set the cross .conf files should in fact be set in abuild already. I'll 
 > look in to doing that as well. 
 >  

For some commands, running without CBUILD being set is okay, some are not. I cannot remember which is which but If I see them again, I will tell you.

 > Though, we are currently looking into improving the build system a bit 
 > more to support cross-building better, including supporting it directly 
 > from abuild command line and building things automatically in chroot. 
 > This needs a bit of design, but is on the way. 
 >  

That's really good to know. I would like to help in that process, or try it out and give feedback.

 > abuild will deduce CARCH from CHOST. You need to add s390x support to 
 > abuild. See commit: 
 > http://git.alpinelinux.org/cgit/abuild/commit/?id=580dd46c588ef71bc852d2b9a5bb4426a2c47665 
 > 

This one is very helpful. I had to set CARCH to some commands around for a while even though I do *guess* it should be deduced from CHOST, by reading /usr/bin/abuild.
   
 > The lib64 is gcc default, and on x86_64 and aarch64 it's similar. We 
 > patch the lib location for these architectures, see: 
 > http://git.alpinelinux.org/cgit/aports/tree/main/gcc/gcc-pure64.patch 
 >  
 > You probably need to add similar change for gcc/config/s390/t-linux64. 
 >  

Looks like gold to me. Progressing on it.

 > > Question 2: 
 > >  
 > > So when you started porting armv7 and aarch, which files in the 
 > > aports tree you had to make changes ? Theoretically I can check in 
 > > the git log but that would be a lot, I think :( 
 >  
 > Not too many. You probably need (aarch64 commit referred): 
 >  
 > - abuild to recognize the triplet (the commit refferred earlier) 
 >   http://git.alpinelinux.org/cgit/abuild/commit/?id=580dd46c588ef71bc852d2b9a5bb4426a2c47665 
 >    
 > - apk-tools to use the same arch name 
 >   http://git.alpinelinux.org/cgit/apk-tools/commit/?id=06ae5fdfdccd0c8e6d5501d93666bd915d2604d1 
 >  
 > - gcc to be configured correct for the arch 
 >   http://git.alpinelinux.org/cgit/aports/commit/main/gcc?id=db1d2c069e66b3bf9962d76b0b48bfdcfc8d0384 
 >   (the ada change is generic, you'll need the _arch_configure + 
 >    gcc-pure64.patch update) 
 >  
 > And update APKBUILDs for the packages that are arch aware; for 
 > minimal bootstrap it includes: 
 >  
 > - linux-headers 
 >   http://git.alpinelinux.org/cgit/aports/commit/?id=f483f49692b97950b680d987fe31f8ddb9277842 
 >  
 > - openssl 
 >   http://git.alpinelinux.org/cgit/aports/commit/?id=896f06e3b0b07200957d0389e2156d0ee4e099cb 
 >  
 > - linux-vanilla (or the kernel you need, including creating kernel configuration) 
 >   http://git.alpinelinux.org/cgit/aports/commit/?id=baa04a75170112f9713dd5ad72a5732f9bfaeaac 
 >  
 > Some packages might also need other kind of patching, e.g. if configure 
 > does not recognize the new triplet. 
 >  

It's much easier to have some existed work to follow. I have patched some APKBUILD myself along the way (gcc, musl, apk-tools, libfetch, openssl ...) and it is kind of painful, when you are new to Alpine. I will use your work and suggestion and come back after few days.

Thank you so much, Timo.

Hoang

 > Cheers, 
 > Timo 
 >  
 >  
 > --- 
 > Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org 
 > Help:         alpine-devel+help@lists.alpinelinux.org 
 > --- 
 >  
 > 



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20160723092945.0680e7fd@vostro>
In-Reply-To
<15614c77014.1200f0e697958.371787825394349914@flatglobe.org> (view parent)
Sender timestamp
1469255384
DKIM signature
missing
Download raw message
On Sat, 23 Jul 2016 05:43:49 +0700
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

>  ---- On Fri, 22 Jul 2016 12:45:14 +0700 Timo Teras
> <timo.teras@iki.fi> wrote ---- 
>  >  
>  > My scripts expect that you add CBUILD there. Though it's trivial
>  > to get it from $(gcc -dumpmachine), I think that should be added
>  > to abuild. I'll probably do that soon. I think most of the
>  > environment variables set the cross .conf files should in fact be
>  > set in abuild already. I'll look in to doing that as well. 
>  >    
> 
> For some commands, running without CBUILD being set is okay, some are
> not. I cannot remember which is which but If I see them again, I will
> tell you.

I just pushed few commits to abuild, that set CBUILD by default
properly. It also fixes dependency installation for host/target to not
run scripts, that would become error later on the build process too.

Please try latest abuild git version if it helps.

>  > Though, we are currently looking into improving the build system a
>  > bit more to support cross-building better, including supporting it
>  > directly from abuild command line and building things
>  > automatically in chroot. This needs a bit of design, but is on the
>  > way. 
> 
> That's really good to know. I would like to help in that process, or
> try it out and give feedback.

I actually want first fix cross building to be supported properly. So
just doing "CHOST=aarch64 abuild -r" would to cross-compile aarch64
version of package on any build system.

I have first patch ready for this that makes all the cross-compile
script config entries built-in to abuild.

The second patch is make abuild properly aware of subpkg's arch. This
is tricky since when make cross toolchain we are generating packages
for the host and the target. But I got this almost done.

The above two will greatly simplify the bootstrap scripts, which I'll
probably dump to aports along with instructions. We've been getting lot
of questions for bootstrapping recently...

>  > abuild will deduce CARCH from CHOST. You need to add s390x support
>  > to abuild. See commit: 
>  > http://git.alpinelinux.org/cgit/abuild/commit/?id=580dd46c588ef71bc852d2b9a5bb4426a2c47665 
> 
> This one is very helpful. I had to set CARCH to some commands around
> for a while even though I do *guess* it should be deduced from CHOST,
> by reading /usr/bin/abuild. 

Could you share what target triplet you are using? And what CARCH? I'll
be happy to push these to abuild early, even if everything is not
working.

>  > The lib64 is gcc default, and on x86_64 and aarch64 it's similar.
>  > We patch the lib location for these architectures, see: 
>  > http://git.alpinelinux.org/cgit/aports/tree/main/gcc/gcc-pure64.patch 
>  >  
>  > You probably need to add similar change for
>  > gcc/config/s390/t-linux64. 
> 
> Looks like gold to me. Progressing on it.
> 
>  > > Question 2: 
>  > >  
>  > > So when you started porting armv7 and aarch, which files in the 
>  > > aports tree you had to make changes ? Theoretically I can check
>  > > in the git log but that would be a lot, I think :(   
>  >  
>  > Not too many. You probably need (aarch64 commit referred): 
>  >  
>  > - abuild to recognize the triplet (the commit refferred earlier) 
>  >   http://git.alpinelinux.org/cgit/abuild/commit/?id=580dd46c588ef71bc852d2b9a5bb4426a2c47665 
>  >    
>  > - apk-tools to use the same arch name 
>  >   http://git.alpinelinux.org/cgit/apk-tools/commit/?id=06ae5fdfdccd0c8e6d5501d93666bd915d2604d1 
>  >  
>  > - gcc to be configured correct for the arch 
>  >   http://git.alpinelinux.org/cgit/aports/commit/main/gcc?id=db1d2c069e66b3bf9962d76b0b48bfdcfc8d0384 
>  >   (the ada change is generic, you'll need the _arch_configure + 
>  >    gcc-pure64.patch update) 
>  >  
>  > And update APKBUILDs for the packages that are arch aware; for 
>  > minimal bootstrap it includes: 
>  >  
>  > - linux-headers 
>  >   http://git.alpinelinux.org/cgit/aports/commit/?id=f483f49692b97950b680d987fe31f8ddb9277842 
>  >  
>  > - openssl 
>  >   http://git.alpinelinux.org/cgit/aports/commit/?id=896f06e3b0b07200957d0389e2156d0ee4e099cb 
>  >  
>  > - linux-vanilla (or the kernel you need, including creating kernel
>  > configuration)
>  > http://git.alpinelinux.org/cgit/aports/commit/?id=baa04a75170112f9713dd5ad72a5732f9bfaeaac 
>  > Some packages might also need other kind of patching, e.g. if
>  > configure does not recognize the new triplet. 
>  >    
> 
> It's much easier to have some existed work to follow. I have patched
> some APKBUILD myself along the way (gcc, musl, apk-tools, libfetch,
> openssl ...) and it is kind of painful, when you are new to Alpine. I
> will use your work and suggestion and come back after few days.

Understand. Porting things to new systems is usually not easy. It's
often done by few people only and the documentation is scarce.

This thread has been useful also for us to understand what questions
come across. I'll probably collect things from here and dump them to
the readme along with the enhanced bootstrap scripts.

Thanks,
Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<1562340dbde.12a594a5120275.3795912911296475631@flatglobe.org>
In-Reply-To
<20160723092945.0680e7fd@vostro> (view parent)
Sender timestamp
1469470268
DKIM signature
missing
Download raw message
 ---- On Sat, 23 Jul 2016 13:29:44 +0700 Timo Teras <timo.teras@iki.fi> wrote ---- 

 > Could you share what target triplet you are using? And what CARCH? I'll 
 > be happy to push these to abuild early, even if everything is not 
 > working. 

Target triplet : s390x-alpine-linux-musl
CARCH : s390x (but for linux-headers, it is s390)

Fortunately, I have successfully run crossbuild-bootstrap targeting s390x, but I need to remove paxmark lines in gcc's APKBUILD as it returns an unknown error (creat-cross runs just fine). AFAIK, it is about security concerns, not system's functionality, so for now I guess it is okay. What do you think?

When I try either to (1) install newly cross-built packages into the sysroot-s390x-alpine-linux-musl/ in the build machine or (2) install Alpine in a chroot in the target device using packages from packages-s390x-alpine-linux-musl/main/s390x/ , they both generate following error: http://pastebin.com/WDJyCaxT. Any chance this thing comes from removing paxmark ? I try to pass --verbose to apk-build but cannot see any change. How did you manage to bootstrap on the real target device?


 > I just pushed few commits to abuild, that set CBUILD by default 
 > properly. It also fixes dependency installation for host/target to not 
 > run scripts, that would become error later on the build process too. 
 >  
 > Please try latest abuild git version if it helps. 
 
I am using it now.

 > >  > Though, we are currently looking into improving the build system a 
 > >  > bit more to support cross-building better, including supporting it 
 > >  > directly from abuild command line and building things 
 > >  > automatically in chroot. This needs a bit of design, but is on the 
 > >  > way.  
 > >  
 > > That's really good to know. I would like to help in that process, or 
 > > try it out and give feedback. 
 >  
 > I actually want first fix cross building to be supported properly. So 
 > just doing "CHOST=aarch64 abuild -r" would to cross-compile aarch64 
 > version of package on any build system. 
 >  
 > I have first patch ready for this that makes all the cross-compile 
 > script config entries built-in to abuild. 
 >  
 > The second patch is make abuild properly aware of subpkg's arch. This 
 > is tricky since when make cross toolchain we are generating packages 
 > for the host and the target. But I got this almost done. 
 >  
 > The above two will greatly simplify the bootstrap scripts, which I'll 
 > probably dump to aports along with instructions. We've been getting lot 
 > of questions for bootstrapping recently... 
 >  

Hum.. That looks quite complicated. Will definitely come back to this afterwards.





---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<156260c81a5.f458ee7355285.6767768145040849925@flatglobe.org>
In-Reply-To
<1562340dbde.12a594a5120275.3795912911296475631@flatglobe.org> (view parent)
Sender timestamp
1469517169
DKIM signature
missing
Download raw message
(updated, please ignore previous email)

 >  ---- On Sat, 23 Jul 2016 13:29:44 +0700 Timo Teras <timo.teras@iki.fi> wrote ----  
 >  
 > Could you share what target triplet you are using? And what CARCH? I'll  
 > be happy to push these to abuild early, even if everything is not  
 > working.  

Target triplet : s390x-alpine-linux-musl 
CARCH : s390x (but for linux-headers, it is s390) 

1) Fortunately, I have successfully run crossbuild script targeting s390x with following :

a) After running creat-cross script, my sysroot's lib directory looks like this:

edge2:~/s390x/$ ls -al sysroot-s390x-alpine-linux-musl/lib
total 12
drwxr-sr-x    3 root     edge          4096 Jul 26 12:54 .
drwxr-sr-x    8 edge     edge          4096 Jul 26 12:54 ..
drwxr-sr-x    3 root     edge          4096 Jul 26 12:54 apk
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 ld-musl-s390x.so.1 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            18 Jul 26 12:54 libc.musl-s390x.so.1 -> ld-musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libc.so.6 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libcrypt.so.1 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libm.so.6 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libpthread.so.0 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 librt.so.1 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libutil.so.1 -> /lib/libc.musl-s390x.so.1

Both ld-musl-s390x.so.1 and /lib/libc.musl-s390x.so.1 do not exist, so all of above are broken links. Then I have to do some thing like this:

edge2:~/s390x$ abuild-apk --root /home/edge/s390x/sysroot-s390x-alpine-linux-musl add /home/edge/s390x/packages-s390x-alpine-linux-musl/main/s390x/musl-1.1.15-r0.apk
(1/1) Replacing musl (1.1.15-r0 -> 1.1.15-r0)
OK: 23 MiB in 6 packages

And then new layout looks like:

edge2:~/s390x/sysroot-s390x-alpine-linux-musl/lib$ ls -al
total 3892
drwxr-sr-x    3 root     edge          4096 Jul 26 13:42 .
drwxr-sr-x    8 edge     edge          4096 Jul 26 12:54 ..
drwxr-sr-x    3 root     edge          4096 Jul 26 12:54 apk
-rwxr-xr-x    1 root     root       3972272 Jul 26 13:17 ld-musl-s390x.so.1
lrwxrwxrwx    1 root     root            18 Jul 26 13:42 libc.musl-s390x.so.1 -> ld-musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libc.so.6 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libcrypt.so.1 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libm.so.6 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libpthread.so.0 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 librt.so.1 -> /lib/libc.musl-s390x.so.1
lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libutil.so.1 -> /lib/libc.musl-s390x.so.1

If I don't do so, later crossbuild process will fail. Please check https://github.com/tmh1999/alpine-bootstrap-s390x for my latest scripts/config.

b) Then I run crossbuild script, and I need remove paxmark lines in gcc's APKBUILD as it returns an unknown error (while creat-cross script runs just fine). AFAIK, it is about security concerns, not system's functionality, so for now I guess it is okay.


2) After 1b) is good, I try either to (*) install newly cross-built packages into the sysroot-s390x-alpine-linux-musl/ in the build machine or (**) install Alpine in a chroot in the target device using packages from packages-s390x-alpine-linux-musl/main/s390x/ , they both generate following error: http://pastebin.com/WDJyCaxT. Any chance this thing comes from removing paxmark in 1b) and create-cross build error in 1a) ? I try to pass --verbose to apk-build but cannot see any change. How did you manage to bootstrap on the real target device? 


 > I just pushed few commits to abuild, that set CBUILD by default  
 > properly. It also fixes dependency installation for host/target to not  
 > run scripts, that would become error later on the build process too.  
 >   
 > Please try latest abuild git version if it helps.  

I am using it now. 

 > >  > Though, we are currently looking into improving the build system a  
 > >  > bit more to support cross-building better, including supporting it  
 > >  > directly from abuild command line and building things  
 > >  > automatically in chroot. This needs a bit of design, but is on the  
 > >  > way.   
 > >   
 > > That's really good to know. I would like to help in that process, or  
 > > try it out and give feedback.  
 >   
 > I actually want first fix cross building to be supported properly. So  
 > just doing "CHOST=aarch64 abuild -r" would to cross-compile aarch64  
 > version of package on any build system.  
 >   
 > I have first patch ready for this that makes all the cross-compile  
 > script config entries built-in to abuild.  
 >   
 > The second patch is make abuild properly aware of subpkg's arch. This  
 > is tricky since when make cross toolchain we are generating packages  
 > for the host and the target. But I got this almost done.  
 >   
 > The above two will greatly simplify the bootstrap scripts, which I'll  
 > probably dump to aports along with instructions. We've been getting lot  
 > of questions for bootstrapping recently...  

Hum.. That looks quite complicated. Will definitely come back to this afterwards. 



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20160726194654.69b12731@vostro>
In-Reply-To
<156260c81a5.f458ee7355285.6767768145040849925@flatglobe.org> (view parent)
Sender timestamp
1469551614
DKIM signature
missing
Download raw message
I'm working on fixing the abuild to be cross-build aware, and bundling
the bootstrap script as part of aports.git. I expect to publish all of
this work later today or tomorrow.

The abuild specific parts are pushed out to:
http://git.alpinelinux.org/cgit/abuild/log/?h=cross-build

On Tue, 26 Jul 2016 14:12:49 +0700
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> (updated, please ignore previous email)
> 
>  >  ---- On Sat, 23 Jul 2016 13:29:44 +0700 Timo Teras
>  > <timo.teras@iki.fi> wrote ---- 
>  > Could you share what target triplet you are using? And what CARCH?
>  > I'll be happy to push these to abuild early, even if everything is
>  > not working.    
> 
> Target triplet : s390x-alpine-linux-musl 
> CARCH : s390x (but for linux-headers, it is s390) 

Thanks. Added this to the cross-build branch of abuild.

> 1) Fortunately, I have successfully run crossbuild script targeting
> s390x with following :
> 
> a) After running creat-cross script, my sysroot's lib directory looks
> like this:
> 
> edge2:~/s390x/$ ls -al sysroot-s390x-alpine-linux-musl/lib
> total 12
> drwxr-sr-x    3 root     edge          4096 Jul 26 12:54 .
> drwxr-sr-x    8 edge     edge          4096 Jul 26 12:54 ..
> drwxr-sr-x    3 root     edge          4096 Jul 26 12:54 apk
> lrwxrwxrwx    1 root     root            25 Jul 26 12:54
> ld-musl-s390x.so.1 -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1
> root     root            18 Jul 26 12:54 libc.musl-s390x.so.1 ->
> ld-musl-s390x.so.1 lrwxrwxrwx    1 root     root            25 Jul 26
> 12:54 libc.so.6 -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root
> root            25 Jul 26 12:54 libcrypt.so.1
> -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root
> 25 Jul 26 12:54 libm.so.6 -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx
> 1 root     root            25 Jul 26 12:54 libpthread.so.0
> -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root
> 25 Jul 26 12:54 librt.so.1 -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx
> 1 root     root            25 Jul 26 12:54 libutil.so.1
> -> /lib/libc.musl-s390x.so.1
> 
> Both ld-musl-s390x.so.1 and /lib/libc.musl-s390x.so.1 do not exist,
> so all of above are broken links. Then I have to do some thing like
> this:
> 
> edge2:~/s390x$ abuild-apk
> --root /home/edge/s390x/sysroot-s390x-alpine-linux-musl
> add /home/edge/s390x/packages-s390x-alpine-linux-musl/main/s390x/musl-1.1.15-r0.apk
> (1/1) Replacing musl (1.1.15-r0 -> 1.1.15-r0) OK: 23 MiB in 6 packages
> 
> And then new layout looks like:
> 
> edge2:~/s390x/sysroot-s390x-alpine-linux-musl/lib$ ls -al
> total 3892
> drwxr-sr-x    3 root     edge          4096 Jul 26 13:42 .
> drwxr-sr-x    8 edge     edge          4096 Jul 26 12:54 ..
> drwxr-sr-x    3 root     edge          4096 Jul 26 12:54 apk
> -rwxr-xr-x    1 root     root       3972272 Jul 26 13:17
> ld-musl-s390x.so.1 lrwxrwxrwx    1 root     root            18 Jul 26
> 13:42 libc.musl-s390x.so.1 -> ld-musl-s390x.so.1 lrwxrwxrwx    1
> root     root            25 Jul 26 12:54 libc.so.6
> -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root
> 25 Jul 26 12:54 libcrypt.so.1 -> /lib/libc.musl-s390x.so.1
> lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libm.so.6
> -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root
> 25 Jul 26 12:54 libpthread.so.0 -> /lib/libc.musl-s390x.so.1
> lrwxrwxrwx    1 root     root            25 Jul 26 12:54 librt.so.1
> -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root
> 25 Jul 26 12:54 libutil.so.1 -> /lib/libc.musl-s390x.so.1
> 
> If I don't do so, later crossbuild process will fail. Please check
> https://github.com/tmh1999/alpine-bootstrap-s390x for my latest
> scripts/config.

Thanks, I believe this should be fixed in the new bootstrap.sh.

You seem to have some patches there. Would you like to send them as git
formatted commits, so you get your name as author of the commit?

> b) Then I run crossbuild script, and I need remove paxmark lines in
> gcc's APKBUILD as it returns an unknown error (while creat-cross
> script runs just fine). AFAIK, it is about security concerns, not
> system's functionality, so for now I guess it is okay.

Your kernel is probably built without XATTR support. paxmark requires
XATTR enabled kernel.

> 2) After 1b) is good, I try either to (*) install newly cross-built
> packages into the sysroot-s390x-alpine-linux-musl/ in the build
> machine or (**) install Alpine in a chroot in the target device using
> packages from packages-s390x-alpine-linux-musl/main/s390x/ , they
> both generate following error: http://pastebin.com/WDJyCaxT. Any
> chance this thing comes from removing paxmark in 1b) and create-cross
> build error in 1a) ? I try to pass --verbose to apk-build but cannot
> see any change. How did you manage to bootstrap on the real target
> device? 

Add --no-scripts so it won't try to execute foreign binaries. Though,
if you want to build a boot image from that, additional fixups are
likely needed. I hope to rewamp the boot image creation scripts
(alpine-iso.git) as set of scripts bundled in aports.git/scripts after
cross build fixes.

>  > >  > Though, we are currently looking into improving the build
>  > >  > system a bit more to support cross-building better, including
>  > >  > supporting it directly from abuild command line and building
>  > >  > things automatically in chroot. This needs a bit of design,
>  > >  > but is on the way.     
>  > >   
>  > > That's really good to know. I would like to help in that
>  > > process, or try it out and give feedback.    
>  >   
>  > I actually want first fix cross building to be supported properly.
>  > So just doing "CHOST=aarch64 abuild -r" would to cross-compile
>  > aarch64 version of package on any build system.  
>  >   
>  > I have first patch ready for this that makes all the
>  > cross-compile script config entries built-in to abuild.  
>  >   
>  > The second patch is make abuild properly aware of subpkg's arch.
>  > This is tricky since when make cross toolchain we are generating
>  > packages for the host and the target. But I got this almost done.  
>  >   
>  > The above two will greatly simplify the bootstrap scripts, which
>  > I'll probably dump to aports along with instructions. We've been
>  > getting lot of questions for bootstrapping recently...    
> 
> Hum.. That looks quite complicated. Will definitely come back to this
> afterwards. 

The abuild changes are complicated. The new bootstrap.sh is clean and
simplified. You'll love it :)

/Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<15635238f98.e963f442298215.3433654478481171006@flatglobe.org>
In-Reply-To
<20160726194654.69b12731@vostro> (view parent)
Sender timestamp
1469770338
DKIM signature
missing
Download raw message
 ---- On Tue, 26 Jul 2016 23:46:54 +0700 Timo Teras <timo.teras@iki.fi> wrote ---- 
 > I'm working on fixing the abuild to be cross-build aware, and bundling 
 > the bootstrap script as part of aports.git. I expect to publish all of 
 > this work later today or tomorrow. 
 >  
 > The abuild specific parts are pushed out to: 
 > http://git.alpinelinux.org/cgit/abuild/log/?h=cross-build 
 >  

The scripts/bootstrap.sh + new abuild (cross-build branch) commits look much cleaner.

 > On Tue, 26 Jul 2016 14:12:49 +0700 
 > "Tuan M. Hoang" <tmhoang@flatglobe.org> wrote: 
 >  
 > > (updated, please ignore previous email) 
 > >  
 > > 1) Fortunately, I have successfully run crossbuild script targeting 
 > > s390x with following : 
 > >  
 > > a) After running creat-cross script, my sysroot's lib directory looks 
 > > like this: 
 > >  
 > > edge2:~/s390x/$ ls -al sysroot-s390x-alpine-linux-musl/lib 
 > > total 12 
 > > drwxr-sr-x    3 root     edge          4096 Jul 26 12:54 . 
 > > drwxr-sr-x    8 edge     edge          4096 Jul 26 12:54 .. 
 > > drwxr-sr-x    3 root     edge          4096 Jul 26 12:54 apk 
 > > lrwxrwxrwx    1 root     root            25 Jul 26 12:54 
 > > ld-musl-s390x.so.1 -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 
 > > root     root            18 Jul 26 12:54 libc.musl-s390x.so.1 -> 
 > > ld-musl-s390x.so.1 lrwxrwxrwx    1 root     root            25 Jul 26 
 > > 12:54 libc.so.6 -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root 
 > > root            25 Jul 26 12:54 libcrypt.so.1 
 > > -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root 
 > > 25 Jul 26 12:54 libm.so.6 -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx 
 > > 1 root     root            25 Jul 26 12:54 libpthread.so.0 
 > > -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root 
 > > 25 Jul 26 12:54 librt.so.1 -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx 
 > > 1 root     root            25 Jul 26 12:54 libutil.so.1 
 > > -> /lib/libc.musl-s390x.so.1 
 > >  
 > > Both ld-musl-s390x.so.1 and /lib/libc.musl-s390x.so.1 do not exist, 
 > > so all of above are broken links. Then I have to do some thing like 
 > > this: 
 > >  
 > > edge2:~/s390x$ abuild-apk 
 > > --root /home/edge/s390x/sysroot-s390x-alpine-linux-musl 
 > > add /home/edge/s390x/packages-s390x-alpine-linux-musl/main/s390x/musl-1.1.15-r0.apk 
 > > (1/1) Replacing musl (1.1.15-r0 -> 1.1.15-r0) OK: 23 MiB in 6 packages 
 > >  
 > > And then new layout looks like: 
 > >  
 > > edge2:~/s390x/sysroot-s390x-alpine-linux-musl/lib$ ls -al 
 > > total 3892 
 > > drwxr-sr-x    3 root     edge          4096 Jul 26 13:42 . 
 > > drwxr-sr-x    8 edge     edge          4096 Jul 26 12:54 .. 
 > > drwxr-sr-x    3 root     edge          4096 Jul 26 12:54 apk 
 > > -rwxr-xr-x    1 root     root       3972272 Jul 26 13:17 
 > > ld-musl-s390x.so.1 lrwxrwxrwx    1 root     root            18 Jul 26 
 > > 13:42 libc.musl-s390x.so.1 -> ld-musl-s390x.so.1 lrwxrwxrwx    1 
 > > root     root            25 Jul 26 12:54 libc.so.6 
 > > -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root 
 > > 25 Jul 26 12:54 libcrypt.so.1 -> /lib/libc.musl-s390x.so.1 
 > > lrwxrwxrwx    1 root     root            25 Jul 26 12:54 libm.so.6 
 > > -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root 
 > > 25 Jul 26 12:54 libpthread.so.0 -> /lib/libc.musl-s390x.so.1 
 > > lrwxrwxrwx    1 root     root            25 Jul 26 12:54 librt.so.1 
 > > -> /lib/libc.musl-s390x.so.1 lrwxrwxrwx    1 root     root 
 > > 25 Jul 26 12:54 libutil.so.1 -> /lib/libc.musl-s390x.so.1 
 > >  
 > > If I don't do so, later crossbuild process will fail. Please check 
 > > https://github.com/tmh1999/alpine-bootstrap-s390x for my latest 
 > > scripts/config. 
 >  
 > Thanks, I believe this should be fixed in the new bootstrap.sh. 

I still have this problem with newbootstrap.sh and do the above workaround. Trying to find the cause of this.

 > You seem to have some patches there. Would you like to send them as git 
 > formatted commits, so you get your name as author of the commit? 

I would love to. But for now musl commits might not be clean since I need to fetch musl.tar.gz from my github repo (which I packaged from : https://github.com/koorogi/musl/tree/s390x ). In order to have it as clean as current musl APKBUILD, we need to wait until s390x is officially merged, which takes quite a time. Other commits are cool and I will submit in 1 day or 2. Need to read the Alpine contribution guide first.

 > > b) Then I run crossbuild script, and I need remove paxmark lines in 
 > > gcc's APKBUILD as it returns an unknown error (while creat-cross 
 > > script runs just fine). AFAIK, it is about security concerns, not 
 > > system's functionality, so for now I guess it is okay. 
 >  
 > Your kernel is probably built without XATTR support. paxmark requires 
 > XATTR enabled kernel. 

Hum... I cannot find any resources/guideline on the Internet. Would you might shed some light on this ?

 > > 2) After 1b) is good, I try either to (*) install newly cross-built 
 > > packages into the sysroot-s390x-alpine-linux-musl/ in the build 
 > > machine or (**) install Alpine in a chroot in the target device using 
 > > packages from packages-s390x-alpine-linux-musl/main/s390x/ , they 
 > > both generate following error: http://pastebin.com/WDJyCaxT. Any 
 > > chance this thing comes from removing paxmark in 1b) and create-cross 
 > > build error in 1a) ? I try to pass --verbose to apk-build but cannot 
 > > see any change. How did you manage to bootstrap on the real target 
 > > device?  
 >  
 > Add --no-scripts so it won't try to execute foreign binaries. Though, 
 > if you want to build a boot image from that, additional fixups are 
 > likely needed. I hope to rewamp the boot image creation scripts 
 > (alpine-iso.git) as set of scripts bundled in aports.git/scripts after 
 > cross build fixes. 

Confirmed.

 > >  > >  > Though, we are currently looking into improving the build 
 > >  > >  > system a bit more to support cross-building better, including 
 > >  > >  > supporting it directly from abuild command line and building 
 > >  > >  > things automatically in chroot. This needs a bit of design, 
 > >  > >  > but is on the way.      
 > >  > >    
 > >  > > That's really good to know. I would like to help in that 
 > >  > > process, or try it out and give feedback.     
 > >  >    
 > >  > I actually want first fix cross building to be supported properly. 
 > >  > So just doing "CHOST=aarch64 abuild -r" would to cross-compile 
 > >  > aarch64 version of package on any build system.   
 > >  >    
 > >  > I have first patch ready for this that makes all the 
 > >  > cross-compile script config entries built-in to abuild.   
 > >  >    
 > >  > The second patch is make abuild properly aware of subpkg's arch. 
 > >  > This is tricky since when make cross toolchain we are generating 
 > >  > packages for the host and the target. But I got this almost done.   
 > >  >    
 > >  > The above two will greatly simplify the bootstrap scripts, which 
 > >  > I'll probably dump to aports along with instructions. We've been 
 > >  > getting lot of questions for bootstrapping recently...     
 > >  
 > > Hum.. That looks quite complicated. Will definitely come back to this 
 > > afterwards.  
 >  
 > The abuild changes are complicated. The new bootstrap.sh is clean and 
 > simplified. You'll love it :) 
 >  
 > /Timo 
 >  

I am trying to read abuild source code and understand its mechanism. Love to get along.
Cheers,



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<20161116023639.23b44cf1.tmhoang@flatglobe.org>
In-Reply-To
<15635238f98.e963f442298215.3433654478481171006@flatglobe.org> (view parent)
Sender timestamp
1479238599
DKIM signature
missing
Download raw message
Hi,

I am sorry for a (very) long time break on this topic. Guess this time I
would finally help make Alpine support s390x.

>  ---- On Tue, 26 Jul 2016 23:46:54 +0700 Timo Teras
> <timo.teras@iki.fi> wrote ---- 
> > b) Then I run crossbuild script, and I need remove paxmark lines
> > in gcc's APKBUILD as it returns an unknown error (while
> > creat-cross script runs just fine). AFAIK, it is about security
> > concerns, not system's functionality, so for now I guess it is
> > okay.   
>  
> Your kernel is probably built without XATTR support. paxmark
> requires XATTR enabled kernel.   

I guess my x86_64 machine running grsec kernel (linux-grsec package) is
XATTR-enabled. I looked a little bit closer on the build log and see
this : http://sprunge.us/EIVE. When I try to run those $ paxctl
manually (code from /usr/sbin/paxmark), they just pass alright
with no output on stdout nor stderr. I also tried adding
--enable-xattr-support to configure script in gcc APKBUILD, but it won't
help. What do you think ?


Another problem I am having is when cross-compiling linux-vanilla
package using aports/scripts/bootstrap.sh : http://sprunge.us/AAcA. I
am reading abuild source code to find the cause but still nothing new.
It'd be nice if you help me to have a look.

Thanks,
Tuan



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20161117082440.7f5e0fa0@vostro.util.wtbts.net>
In-Reply-To
<20161116023639.23b44cf1.tmhoang@flatglobe.org> (view parent)
Sender timestamp
1479363880
DKIM signature
missing
Download raw message
On Wed, 16 Nov 2016 02:36:39 +0700
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> I am sorry for a (very) long time break on this topic. Guess this
> time I would finally help make Alpine support s390x.
> 
> >  ---- On Tue, 26 Jul 2016 23:46:54 +0700 Timo Teras
> > <timo.teras@iki.fi> wrote ----   
> > > b) Then I run crossbuild script, and I need remove paxmark lines
> > > in gcc's APKBUILD as it returns an unknown error (while
> > > creat-cross script runs just fine). AFAIK, it is about security
> > > concerns, not system's functionality, so for now I guess it is
> > > okay.     
> >  
> > Your kernel is probably built without XATTR support. paxmark
> > requires XATTR enabled kernel.     
> 
> I guess my x86_64 machine running grsec kernel (linux-grsec package)
> is XATTR-enabled. I looked a little bit closer on the build log and
> see this : http://sprunge.us/EIVE. When I try to run those $ paxctl
> manually (code from /usr/sbin/paxmark), they just pass alright
> with no output on stdout nor stderr. I also tried adding
> --enable-xattr-support to configure script in gcc APKBUILD, but it
> won't help. What do you think ?

The kernel should be xattr enabled. What filesystem are you using?
Perhaps there's some filesystem level knob (kernel config) or
limitation.

> Another problem I am having is when cross-compiling linux-vanilla
> package using aports/scripts/bootstrap.sh : http://sprunge.us/AAcA. I
> am reading abuild source code to find the cause but still nothing new.
> It'd be nice if you help me to have a look.

Bootstrap script was not yet updated for libressl change. I'll take a
look at this. I think the only change needed is to build libressl
instead of openssl. I'll push fix for this soon.


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<20161119041548.765e6513.tmhoang@flatglobe.org>
In-Reply-To
<20161117082440.7f5e0fa0@vostro.util.wtbts.net> (view parent)
Sender timestamp
1479503748
DKIM signature
missing
Download raw message
On Thu, 17 Nov 2016 08:24:40 +0200
Timo Teras <timo.teras@iki.fi> wrote:
  
> > > > b) Then I run crossbuild script, and I need remove paxmark lines
> > > > in gcc's APKBUILD as it returns an unknown error (while
> > > > creat-cross script runs just fine). AFAIK, it is about security
> > > > concerns, not system's functionality, so for now I guess it is
> > > > okay.     
> > >  
> > > Your kernel is probably built without XATTR support. paxmark
> > > requires XATTR enabled kernel.     
> > 
> > I guess my x86_64 machine running grsec kernel (linux-grsec package)
> > is XATTR-enabled. I looked a little bit closer on the build log and
> > see this : http://sprunge.us/EIVE. When I try to run those $ paxctl
> > manually (code from /usr/sbin/paxmark), they just pass alright
> > with no output on stdout nor stderr. I also tried adding
> > --enable-xattr-support to configure script in gcc APKBUILD, but it
> > won't help. What do you think ?
> 
> The kernel should be xattr enabled. What filesystem are you using?
> Perhaps there's some filesystem level knob (kernel config) or
> limitation.

$ df -T | grep sda
/dev/sda3            ext4        20473424   6217272  13196368  32% /
/dev/sda1            ext4           95054     16460     71426  19% /boot
$ cat /proc/fs/ext4/sda1/options | grep xattr
user_xattr
$ cat /proc/fs/ext4/sda3/options | grep xattr
user_xattr

> > Another problem I am having is when cross-compiling linux-vanilla
> > package using aports/scripts/bootstrap.sh : http://sprunge.us/AAcA.
> > I am reading abuild source code to find the cause but still nothing
> > new. It'd be nice if you help me to have a look.
> 
> Bootstrap script was not yet updated for libressl change. I'll take a
> look at this. I think the only change needed is to build libressl
> instead of openssl. I'll push fix for this soon.

Actually this occurred to me before libress was adopted. After libressl
was introduced, I also changed Bootstrap script, libressl APKBUILD,
kernel config to adop libressl too, but still the bug. So I guess it's
more likely coming from abuild. Here are some of my patches, in case
you might want to have a look.
https://github.com/tmh1999/alpine-bootstrap-s390x/tree/master/patches/aports/scripts
https://github.com/tmh1999/alpine-bootstrap-s390x/tree/master/patches/aports/main



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20161119092109.13567a93@vostro>
In-Reply-To
<20161119041548.765e6513.tmhoang@flatglobe.org> (view parent)
Sender timestamp
1479540069
DKIM signature
missing
Download raw message
On Sat, 19 Nov 2016 04:15:48 +0700
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> On Thu, 17 Nov 2016 08:24:40 +0200
> Timo Teras <timo.teras@iki.fi> wrote:
>   
> > > > > b) Then I run crossbuild script, and I need remove paxmark
> > > > > lines in gcc's APKBUILD as it returns an unknown error (while
> > > > > creat-cross script runs just fine). AFAIK, it is about
> > > > > security concerns, not system's functionality, so for now I
> > > > > guess it is okay.       
> > > >  
> > > > Your kernel is probably built without XATTR support. paxmark
> > > > requires XATTR enabled kernel.       
> > > 
> > > I guess my x86_64 machine running grsec kernel (linux-grsec
> > > package) is XATTR-enabled. I looked a little bit closer on the
> > > build log and see this : http://sprunge.us/EIVE. When I try to
> > > run those $ paxctl manually (code from /usr/sbin/paxmark), they
> > > just pass alright with no output on stdout nor stderr. I also
> > > tried adding --enable-xattr-support to configure script in gcc
> > > APKBUILD, but it won't help. What do you think ?  
> > 
> > The kernel should be xattr enabled. What filesystem are you using?
> > Perhaps there's some filesystem level knob (kernel config) or
> > limitation.  
> 
> $ df -T | grep sda
> /dev/sda3            ext4        20473424   6217272  13196368  32% /
> /dev/sda1            ext4           95054     16460     71426
> 19% /boot $ cat /proc/fs/ext4/sda1/options | grep xattr
> user_xattr
> $ cat /proc/fs/ext4/sda3/options | grep xattr
> user_xattr

IIRC, if it's grsec kernel, you don't even need the user_xattr mount
option as grsec kernel treats those xattrs specially.

Of course grsec does not support s390x so the markings are not really
needed for vanilla kernel. However would be good to figure out so we
get the build right from beginning.

You could try strace the attr command and see where it fails.

> > > Another problem I am having is when cross-compiling linux-vanilla
> > > package using aports/scripts/bootstrap.sh :
> > > http://sprunge.us/AAcA. I am reading abuild source code to find
> > > the cause but still nothing new. It'd be nice if you help me to
> > > have a look.  
> > 
> > Bootstrap script was not yet updated for libressl change. I'll take
> > a look at this. I think the only change needed is to build libressl
> > instead of openssl. I'll push fix for this soon.  
> 
> Actually this occurred to me before libress was adopted. After
> libressl was introduced, I also changed Bootstrap script, libressl
> APKBUILD, kernel config to adop libressl too, but still the bug. So I
> guess it's more likely coming from abuild. Here are some of my
> patches, in case you might want to have a look.
> https://github.com/tmh1999/alpine-bootstrap-s390x/tree/master/patches/aports/scripts
> https://github.com/tmh1999/alpine-bootstrap-s390x/tree/master/patches/aports/main
 
I would be happy to pick up some of these patches already now. Would
you be able to give them exported with "git format-patch" so you get
annotated as author and the commit log is preserved.

I could try bootstrap s390 too then and see if the same problem happens
and analyze it a bit more.

In any case I recently fixed the bootstrap script and few aports. I was
able to bootstrap again git master for armv7.

Cheers,
Timo



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<20161121073927.2d0a9e65.tmhoang@flatglobe.org>
In-Reply-To
<20161119092109.13567a93@vostro> (view parent)
Sender timestamp
1479688767
DKIM signature
missing
Download raw message
On Sat, 19 Nov 2016 09:21:09 +0200
Timo Teras <timo.teras@iki.fi> wrote:

> On Sat, 19 Nov 2016 04:15:48 +0700
> "Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:
> 
> > On Thu, 17 Nov 2016 08:24:40 +0200
> > Timo Teras <timo.teras@iki.fi> wrote:
> >     
> > > > > > b) Then I run crossbuild script, and I need remove paxmark
> > > > > > lines in gcc's APKBUILD as it returns an unknown error
> > > > > > (while creat-cross script runs just fine). AFAIK, it is
> > > > > > about security concerns, not system's functionality, so for
> > > > > > now I guess it is okay.         
> > > > >  
> > > > > Your kernel is probably built without XATTR support. paxmark
> > > > > requires XATTR enabled kernel.         
> > > > 
> > > > I guess my x86_64 machine running grsec kernel (linux-grsec
> > > > package) is XATTR-enabled. I looked a little bit closer on the
> > > > build log and see this : http://sprunge.us/EIVE. When I try to
> > > > run those $ paxctl manually (code from /usr/sbin/paxmark), they
> > > > just pass alright with no output on stdout nor stderr. I also
> > > > tried adding --enable-xattr-support to configure script in gcc
> > > > APKBUILD, but it won't help. What do you think ?    
> > > 
> > > The kernel should be xattr enabled. What filesystem are you using?
> > > Perhaps there's some filesystem level knob (kernel config) or
> > > limitation.    
> > 
> > $ df -T | grep sda
> > /dev/sda3            ext4        20473424   6217272  13196368  32% /
> > /dev/sda1            ext4           95054     16460     71426
> > 19% /boot $ cat /proc/fs/ext4/sda1/options | grep xattr
> > user_xattr
> > $ cat /proc/fs/ext4/sda3/options | grep xattr
> > user_xattr  
> 
> IIRC, if it's grsec kernel, you don't even need the user_xattr mount
> option as grsec kernel treats those xattrs specially.
> 
> Of course grsec does not support s390x so the markings are not really
> needed for vanilla kernel. However would be good to figure out so we
> get the build right from beginning.
> 
> You could try strace the attr command and see where it fails.
> 

I see that you didn't provide linux-grsec/config-grsec.aarch64. grsec
does not support aarch64 neither? I have read some grsec patch and it
looks like they do provide some s390/s390x patch like this one :
http://dev.alpinelinux.org/~ncopa/grsec/grsecurity-3.1-4.4.32-201604252206-alpine.patch
I am not sure is this called "supported".

Actually according to the log above, $ paxctl are those failed (ret=1).
I will try to strace them this week.

> > > > Another problem I am having is when cross-compiling
> > > > linux-vanilla package using aports/scripts/bootstrap.sh :
> > > > http://sprunge.us/AAcA. I am reading abuild source code to find
> > > > the cause but still nothing new. It'd be nice if you help me to
> > > > have a look.    
> > > 
> > > Bootstrap script was not yet updated for libressl change. I'll
> > > take a look at this. I think the only change needed is to build
> > > libressl instead of openssl. I'll push fix for this soon.    
> > 
> > Actually this occurred to me before libress was adopted. After
> > libressl was introduced, I also changed Bootstrap script, libressl
> > APKBUILD, kernel config to adop libressl too, but still the bug. So
> > I guess it's more likely coming from abuild. Here are some of my
> > patches, in case you might want to have a look.
> > https://github.com/tmh1999/alpine-bootstrap-s390x/tree/master/patches/aports/scripts
> > https://github.com/tmh1999/alpine-bootstrap-s390x/tree/master/patches/aports/main  
>  
> I would be happy to pick up some of these patches already now. Would
> you be able to give them exported with "git format-patch" so you get
> annotated as author and the commit log is preserved.
> 
> I could try bootstrap s390 too then and see if the same problem
> happens and analyze it a bit more.
> 
> In any case I recently fixed the bootstrap script and few aports. I
> was able to bootstrap again git master for armv7.
> 
> Cheers,
> Timo

Please find my patches in the attachments. There were some small
changes that I did not include in those patches because I think it only
(temporarily) affects my build. So I list here fyi:
- scripts/bootstrap.sh : linux-headers & zlib-dev need to be explicitly
  installed (besides fortify-headers, libc-dev, build-base) even though
  linux-headers is zlib's makedends, zlib-dev is binutils's makedepends.

If there was any problem, please kindly let me know.

Cheers,
Tuan
Timo Teras
Details
Message ID
<20161121105925.685761a2@vostro.util.wtbts.net>
In-Reply-To
<20161121073927.2d0a9e65.tmhoang@flatglobe.org> (view parent)
Sender timestamp
1479718765
DKIM signature
missing
Download raw message
On Mon, 21 Nov 2016 07:39:27 +0700
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> I see that you didn't provide linux-grsec/config-grsec.aarch64. grsec
> does not support aarch64 neither? I have read some grsec patch and it
> looks like they do provide some s390/s390x patch like this one :
> http://dev.alpinelinux.org/~ncopa/grsec/grsecurity-3.1-4.4.32-201604252206-alpine.patch
> I am not sure is this called "supported".

Yes, aarch64 is not supported to my knowledge.

But indeed there seems to be s390 support in grsec. Feel free to look
at it, and enable a grsec config for it.

> Actually according to the log above, $ paxctl are those failed
> (ret=1). I will try to strace them this week.

Ok.

> > I would be happy to pick up some of these patches already now. Would
> > you be able to give them exported with "git format-patch" so you get
> > annotated as author and the commit log is preserved.
> > 
> > I could try bootstrap s390 too then and see if the same problem
> > happens and analyze it a bit more.
> > 
> > In any case I recently fixed the bootstrap script and few aports. I
> > was able to bootstrap again git master for armv7.
> > 
> Please find my patches in the attachments. There were some small
> changes that I did not include in those patches because I think it
> only (temporarily) affects my build. So I list here fyi:
> - scripts/bootstrap.sh : linux-headers & zlib-dev need to be
> explicitly installed (besides fortify-headers, libc-dev, build-base)
> even though linux-headers is zlib's makedends, zlib-dev is binutils's
> makedepends.
> 
> If there was any problem, please kindly let me know.

I applied apk-tools patch to both aports, and apk-tools git. I also
took parts of aports/gcc patch (the s390 specific config).
linux-headers patch was applied as is.

I'm still looking at the rest of the patches. Here's my review thoughts
of them:

- aports/gcc, moving the libgcc_s.so symlink is probably not the right
  solution. We prefer to keep that in the main 'gcc' package. During
  cross-build it should work so that the 'gcc-s390x' cross-build
  package has this in usr/s390x-alpine-linux-musl/lib/libgcc_s.so. Now
  it breaks because on other arch's this is in reality a linker script
  and not a symlink. The linker script enables gcc to search for the
  right file from all of the libpath again. But s390 is having it as
  symlink, and breaks because the actual file is relocated to the
  sysroot. I am not fully sure what to do here. I might lean on just
  overwriting the cross compiler's libgcc_s.so to be a while with
  contents:
  GROUP ( libgcc_s.so.1 -lgcc )

  I'll think about it a bit more, and can push a fix later today for it.

- aports/linux-vanilla looks good. Though you probably just got the
  kernel config from some other linux distro? It would probably be good
  to cross-check that with other Alpine's linux-vanilla configs to see
  that it is more or less in harmony.

- aports/musl looks mostly ok. We hope musl would soon make release,
  and much of the patch is not needed then as it's mostly upstreamed
  now. There's one issue that's causing some of your trouble. The
  following hunk is wrong:

@@ -142,6 +146,7 @@ compat() {
 	mips*) _ld="ld.so.1" ;;
+	s390*) _ld="ld-musl-s390x.so.1" ;;

  The compat package is supposed to create symlinks that mimic glibc
  provided ld and libraries. The _ld var needs to be setup to be the
  glibc ld name, not the musl one. Looking at gcc, the proper thing
  here to do is:
-       mips*) _ld="ld.so.1" ;;
+       mips* | s390*) _ld="ld.so.1" ;;

- In main/libressl you add --disable-asm? Why is this needed?
  It seemed to compile just fine without this flag. If really needed,
  this should be made conditional so that it's only applied for s390.

- The bootstrap script changes should not be needed after the above
  fixes.

Thanks for working on this! It's starting to look pretty good, and with
the few tweaks above and after musl release, I think we are good to
commit all of the required changes.

Cheers,
Timo






---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<20170122184652.338e4344.tmhoang@flatglobe.org>
In-Reply-To
<20161121105925.685761a2@vostro.util.wtbts.net> (view parent)
Sender timestamp
1485128812
DKIM signature
missing
Download raw message
On Mon, 21 Nov 2016 10:59:25 +0200
Timo Teras <timo.teras@iki.fi> wrote:

> I'm still looking at the rest of the patches. Here's my review
> thoughts of them:
> 
> 
> - aports/linux-vanilla looks good. Though you probably just got the
>   kernel config from some other linux distro? It would probably be
> good to cross-check that with other Alpine's linux-vanilla configs to
> see that it is more or less in harmony.

I have done some work on it. You can find for reference here :
http://flatglobe.org/config-vanilla.s390x

> - aports/musl looks mostly ok. We hope musl would soon make release,
>   and much of the patch is not needed then as it's mostly upstreamed
>   now. There's one issue that's causing some of your trouble. The
>   following hunk is wrong:
> 
> @@ -142,6 +146,7 @@ compat() {
>  	mips*) _ld="ld.so.1" ;;
> +	s390*) _ld="ld-musl-s390x.so.1" ;;
> 
>   The compat package is supposed to create symlinks that mimic glibc
>   provided ld and libraries. The _ld var needs to be setup to be the
>   glibc ld name, not the musl one. Looking at gcc, the proper thing
>   here to do is:
> -       mips*) _ld="ld.so.1" ;;
> +       mips* | s390*) _ld="ld.so.1" ;;
> 
Thanks. This helped a lot.

> - In main/libressl you add --disable-asm? Why is this needed?
>   It seemed to compile just fine without this flag. If really needed,
>   this should be made conditional so that it's only applied for s390.
> 

Yep this is old flag. Now it is obsolete.

Btw, fyi, 31 support of s390 architecture has been dropped, now it's
all s390x (64 bit version of s390) for all packages, except in linux
kernel, the name still remains s390.
https://github.com/dbussink/linux/commit/5a79859ae0f35d25c67a03e82bf0c80592f16a39

I am currently having two issues, hope you can take some time to see it.

1. I was running Alpine s390x in Docker with some essential
cross-compiled packages targeting s390x, in order to compile natively
the toolchains. It all boils down to missing perl package as
compile-time dependency. Perl cannot be cross-compiled, yet is needed
for native compile. When I try to compile perl natively (inside either
Docker or a chroot), I had segmentation fault on running fakeroot.

Just wonder if this is the correct way you did and segmentation fault
is only my thing.


2. I was having an error when building linux-vanilla-dev ( with above
kernel config ). When running $ make in dev() in APKBUILD, HOSTCC
compiles some scripts in "$srcdir"/linux-$_kernver/scripts directory
into host arch (x86_64) object/exec files. These files were later get
copied into "$subpkgdir"/usr/src/linux-headers-${_abi_release}
(aports/main/linux-vanilla/pkg/linux-vanilla-dev/usr/src/linux-headers-4.4.34).
Then libc.musl-x86_64.so.1 comes up in as a needed .so file in later
steps (scan_shared_objects() function). This happens for
linux-vanilla-dev package, while bootstraping both aarch64, and s390x
using aports master.

$ scanelf --nobanner --recursive --needed
aports/main/linux-vanilla/pkg/linux-vanilla-dev
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/sortextable
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/asn1_compiler
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/dtc/dtc
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/recordmcount
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/genksyms/genksyms
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/kconfig/conf
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/mod/modpost
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/mod/mk_elfconfig
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/basic/bin2c
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/basic/fixdep
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/conmakehash
ET_DYN
libc.musl-x86_64.so.1 ./usr/src/linux-headers-4.4.38/scripts/kallsyms

Thanks,
Tuan




---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20170127090054.5f6ba6ce@vostro.util.wtbts.net>
In-Reply-To
<20170122184652.338e4344.tmhoang@flatglobe.org> (view parent)
Sender timestamp
1485500454
DKIM signature
missing
Download raw message
On Sun, 22 Jan 2017 18:46:52 -0500
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> I am currently having two issues, hope you can take some time to see
> it.
> 
> 1. I was running Alpine s390x in Docker with some essential
> cross-compiled packages targeting s390x, in order to compile natively
> the toolchains. It all boils down to missing perl package as
> compile-time dependency. Perl cannot be cross-compiled, yet is needed
> for native compile. When I try to compile perl natively (inside either
> Docker or a chroot), I had segmentation fault on running fakeroot.

Yes, we also native build perl. It's a pain to cross-compile.
 
> Just wonder if this is the correct way you did and segmentation fault
> is only my thing.

If fakeroot segfaults you'll end up in trouble sooner or later. It'd be
good to fix that. Do you have strace or gdb backtrace of the crash? 

> 2. I was having an error when building linux-vanilla-dev ( with above
> kernel config ). When running $ make in dev() in APKBUILD, HOSTCC
> compiles some scripts in "$srcdir"/linux-$_kernver/scripts directory
> into host arch (x86_64) object/exec files. These files were later get
> copied into "$subpkgdir"/usr/src/linux-headers-${_abi_release}
> (aports/main/linux-vanilla/pkg/linux-vanilla-dev/usr/src/linux-headers-4.4.34).
> Then libc.musl-x86_64.so.1 comes up in as a needed .so file in later
> steps (scan_shared_objects() function). This happens for
> linux-vanilla-dev package, while bootstraping both aarch64, and s390x
> using aports master.

This is tricky. Should the -dev package be built for the target or for
the build host. When cross-compiling it should probably be built for
the build host. Because it will probably be used to build the modules
too. Maybe the following change in linux-vanilla will fix it:
-subpackages="$pkgname-dev"
+subpackages="$pkgname-dev::$CBUILD_ARCH"

Cheers,
Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Breno Leitao
Details
Message ID
<a6e58a61-f56d-4bf3-e735-8a9352a33030@br.ibm.com>
In-Reply-To
<20170127090054.5f6ba6ce@vostro.util.wtbts.net> (view parent)
Sender timestamp
1485518052
DKIM signature
missing
Download raw message
On 01/27/2017 05:00 AM, Timo Teras wrote:
> On Sun, 22 Jan 2017 18:46:52 -0500
> "Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:
> 
>> I am currently having two issues, hope you can take some time to see
>> it.
>>
>> 1. I was running Alpine s390x in Docker with some essential
>> cross-compiled packages targeting s390x, in order to compile natively
>> the toolchains. It all boils down to missing perl package as
>> compile-time dependency. Perl cannot be cross-compiled, yet is needed
>> for native compile. When I try to compile perl natively (inside either
>> Docker or a chroot), I had segmentation fault on running fakeroot.
> 
> Yes, we also native build perl. It's a pain to cross-compile.

That is what we are doing with ppc64le also. We just cross-compiled the
minimum set of packages to be able to run abuild inside the chroot, and then,
(re)build everything natively.



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<20170128200029.271b0fcf.tmhoang@flatglobe.org>
In-Reply-To
<20170127090054.5f6ba6ce@vostro.util.wtbts.net> (view parent)
Sender timestamp
1485651629
DKIM signature
missing
Download raw message
On Fri, 27 Jan 2017 09:00:54 +0200
Timo Teras <timo.teras@iki.fi> wrote:
> 
> If fakeroot segfaults you'll end up in trouble sooner or later. It'd
> be good to fix that. Do you have strace or gdb backtrace of the
> crash? 

Cross-compiling strace and gdb requires 'linux-headers'
installed in $CBUILDROOT. strace is cool but gdb still fails
for me with following errors :

In file included from gregset.h:26:0,
                 from s390-linux-nat.c:28:
nm.h:23:24: error: 'elf_gregset_t' does not name a type
 #define GDB_GREGSET_T  elf_gregset_t
                        ^
gregset.h:36:9: note: in expansion of macro 'GDB_GREGSET_T'
 typedef GDB_GREGSET_T gdb_gregset_t;
         ^~~~~~~~~~~~~
nm.h:24:24: error: 'elf_fpregset_t' does not name a type
 #define GDB_FPREGSET_T elf_fpregset_t
                        ^
gregset.h:37:9: note: in expansion of macro 'GDB_FPREGSET_T'
 typedef GDB_FPREGSET_T gdb_fpregset_t;
         ^~~~~~~~~~~~~~
In file included from s390-linux-nat.c:28:0:
gregset.h:52:14: error: 'gdb_gregset_t' does not name a type
        const gdb_gregset_t *gregs);
              ^~~~~~~~~~~~~
gregset.h:54:15: error: 'gdb_fpregset_t' does not name a type
         const gdb_fpregset_t *fpregs);
               ^~~~~~~~~~~~~~
gregset.h:61:6: error: 'gdb_gregset_t' has not been declared
      gdb_gregset_t *gregs, int regno);
      ^~~~~~~~~~~~~
gregset.h:63:7: error: 'gdb_fpregset_t' has not been declared
       gdb_fpregset_t *fpregs, int regno);
       ^~~~~~~~~~~~~~

elf_gregset_t and elf_fpregset_t were supposed to be defined in
https://github.com/torvalds/linux/blob/master/arch/s390/include/asm/elf.h
and I think it should be built in 'linux-headers', but no trace of them.

Here is the log of strace fakeroot : http://pastebin.com/Pn7YUWRz

> > 2. I was having an error when building linux-vanilla-dev ( with
> > above kernel config ). When running $ make in dev() in APKBUILD,
> > HOSTCC compiles some scripts in "$srcdir"/linux-$_kernver/scripts
> > directory into host arch (x86_64) object/exec files. These files
> > were later get copied into
> > "$subpkgdir"/usr/src/linux-headers-${_abi_release}
> > (aports/main/linux-vanilla/pkg/linux-vanilla-dev/usr/src/linux-headers-4.4.34).
> > Then libc.musl-x86_64.so.1 comes up in as a needed .so file in
> > later steps (scan_shared_objects() function). This happens for
> > linux-vanilla-dev package, while bootstraping both aarch64, and
> > s390x using aports master.
> 
> This is tricky. Should the -dev package be built for the target or for
> the build host. When cross-compiling it should probably be built for
> the build host. Because it will probably be used to build the modules
> too. Maybe the following change in linux-vanilla will fix it:
> -subpackages="$pkgname-dev"
> +subpackages="$pkgname-dev::$CBUILD_ARCH"
> 
This indeed fixes it and makes linux-vanilla-dev a HOST's package.

Thank you,
Tuan



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20170130132156.1ce001a8@vostro.util.wtbts.net>
In-Reply-To
<20170128200029.271b0fcf.tmhoang@flatglobe.org> (view parent)
Sender timestamp
1485775316
DKIM signature
missing
Download raw message
On Sat, 28 Jan 2017 20:00:29 -0500
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> On Fri, 27 Jan 2017 09:00:54 +0200
> Timo Teras <timo.teras@iki.fi> wrote:
> > 
> > If fakeroot segfaults you'll end up in trouble sooner or later. It'd
> > be good to fix that. Do you have strace or gdb backtrace of the
> > crash?   
> 
> Cross-compiling strace and gdb requires 'linux-headers'
> installed in $CBUILDROOT. strace is cool but gdb still fails

Should fix strace APKBUILD to support cross-compilation using the
makedepends_host as used in other cross-compile enabled packages.

> for me with following errors :
> 
> In file included from gregset.h:26:0,
>                  from s390-linux-nat.c:28:
> nm.h:23:24: error: 'elf_gregset_t' does not name a type
>  #define GDB_GREGSET_T  elf_gregset_t
>                         ^
> gregset.h:36:9: note: in expansion of macro 'GDB_GREGSET_T'
>  typedef GDB_GREGSET_T gdb_gregset_t;
>          ^~~~~~~~~~~~~
> nm.h:24:24: error: 'elf_fpregset_t' does not name a type
>  #define GDB_FPREGSET_T elf_fpregset_t
>                         ^
> gregset.h:37:9: note: in expansion of macro 'GDB_FPREGSET_T'
>  typedef GDB_FPREGSET_T gdb_fpregset_t;
>          ^~~~~~~~~~~~~~
> In file included from s390-linux-nat.c:28:0:
> gregset.h:52:14: error: 'gdb_gregset_t' does not name a type
>         const gdb_gregset_t *gregs);
>               ^~~~~~~~~~~~~
> gregset.h:54:15: error: 'gdb_fpregset_t' does not name a type
>          const gdb_fpregset_t *fpregs);
>                ^~~~~~~~~~~~~~
> gregset.h:61:6: error: 'gdb_gregset_t' has not been declared
>       gdb_gregset_t *gregs, int regno);
>       ^~~~~~~~~~~~~
> gregset.h:63:7: error: 'gdb_fpregset_t' has not been declared
>        gdb_fpregset_t *fpregs, int regno);
>        ^~~~~~~~~~~~~~
> 
> elf_gregset_t and elf_fpregset_t were supposed to be defined in
> https://github.com/torvalds/linux/blob/master/arch/s390/include/asm/elf.h
> and I think it should be built in 'linux-headers', but no trace of
> them.

Strange. Sounds like a bug somewhere.

> Here is the log of strace fakeroot : http://pastebin.com/Pn7YUWRz

Could you retry with "strace -f -s 1024" ?

> > > 2. I was having an error when building linux-vanilla-dev ( with
> > > above kernel config ). When running $ make in dev() in APKBUILD,
> > > HOSTCC compiles some scripts in "$srcdir"/linux-$_kernver/scripts
> > > directory into host arch (x86_64) object/exec files. These files
> > > were later get copied into
> > > "$subpkgdir"/usr/src/linux-headers-${_abi_release}
> > > (aports/main/linux-vanilla/pkg/linux-vanilla-dev/usr/src/linux-headers-4.4.34).
> > > Then libc.musl-x86_64.so.1 comes up in as a needed .so file in
> > > later steps (scan_shared_objects() function). This happens for
> > > linux-vanilla-dev package, while bootstraping both aarch64, and
> > > s390x using aports master.  
> > 
> > This is tricky. Should the -dev package be built for the target or
> > for the build host. When cross-compiling it should probably be
> > built for the build host. Because it will probably be used to build
> > the modules too. Maybe the following change in linux-vanilla will
> > fix it: -subpackages="$pkgname-dev"
> > +subpackages="$pkgname-dev::$CBUILD_ARCH"
> >   
> This indeed fixes it and makes linux-vanilla-dev a HOST's package.

Ok, I'll probably commit that to the relevant kernel packages then.

Thanks.


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<20170201014224.3842294c.tmhoang@flatglobe.org>
In-Reply-To
<20170130132156.1ce001a8@vostro.util.wtbts.net> (view parent)
Sender timestamp
1485931344
DKIM signature
missing
Download raw message
On Mon, 30 Jan 2017 13:21:56 +0200
Timo Teras <timo.teras@iki.fi> wrote:

> > Cross-compiling strace and gdb requires 'linux-headers'
> > installed in $CBUILDROOT. strace is cool but gdb still fails  
> 
> Should fix strace APKBUILD to support cross-compilation using the
> makedepends_host as used in other cross-compile enabled packages.

Thanks, it's very neat.

> > elf_gregset_t and elf_fpregset_t were supposed to be defined in
> > https://github.com/torvalds/linux/blob/master/arch/s390/include/asm/elf.h
> > and I think it should be built in 'linux-headers', but no trace of
> > them.  
> 
> Strange. Sounds like a bug somewhere.

Indeed. It was a musl bug, and with some patches, gdb is good now. Looks
like /usr/lib/libfakeroot.so is the one causes the segfault. Is there
any option in abuild to build package with out being stripped? I would
like to do so then debug fakeroot better. Adding -DDEBUG or -g in
CFLAGS and CXXFLAGS in APKBUILD doesn't help.

> > Here is the log of strace fakeroot : http://pastebin.com/Pn7YUWRz  
> 
> Could you retry with "strace -f -s 1024" ?
>

I am not sure the strace would be informative for you : 
http://pastebin.com/4P0f5sDF

Thanks,
Tuan




---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras
Details
Message ID
<20170201090359.25e84463@vostro.util.wtbts.net>
In-Reply-To
<20170201014224.3842294c.tmhoang@flatglobe.org> (view parent)
Sender timestamp
1485932639
DKIM signature
missing
Download raw message
On Wed, 1 Feb 2017 01:42:24 -0500
"Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:

> On Mon, 30 Jan 2017 13:21:56 +0200
> Timo Teras <timo.teras@iki.fi> wrote:
> 
> > > Cross-compiling strace and gdb requires 'linux-headers'
> > > installed in $CBUILDROOT. strace is cool but gdb still fails    
> > 
> > Should fix strace APKBUILD to support cross-compilation using the
> > makedepends_host as used in other cross-compile enabled packages.  
> 
> Thanks, it's very neat.
> 
> > > elf_gregset_t and elf_fpregset_t were supposed to be defined in
> > > https://github.com/torvalds/linux/blob/master/arch/s390/include/asm/elf.h
> > > and I think it should be built in 'linux-headers', but no trace of
> > > them.    
> > 
> > Strange. Sounds like a bug somewhere.  
> 
> Indeed. It was a musl bug, and with some patches, gdb is good now.

Ok. Good, perhaps you can submit them upstream to musl?

> Looks like /usr/lib/libfakeroot.so is the one causes the segfault. Is
> there any option in abuild to build package with out being stripped?
> I would like to do so then debug fakeroot better. Adding -DDEBUG or
> -g in CFLAGS and CXXFLAGS in APKBUILD doesn't help.

DEBUG=yes abuild

or 

add "$pkgname-dbg" to subpackages, and a separate -dbg package is
created with the debug information.

> > > Here is the log of strace fakeroot :
> > > http://pastebin.com/Pn7YUWRz    
> > 
> > Could you retry with "strace -f -s 1024" ?
> 
> I am not sure the strace would be informative for you : 
> http://pastebin.com/4P0f5sDF

Unfortunately, this does not immediately reveal anything additional.
Getting gdb backtrace should help.

Thanks,
Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Tuan M. Hoang
Details
Message ID
<747ae81f-e49a-7508-a359-75c7d3ee87e7@flatglobe.org>
In-Reply-To
<20170201090359.25e84463@vostro.util.wtbts.net> (view parent)
Sender timestamp
1487352456
DKIM signature
missing
Download raw message
Hi,

Please help me to review these patches.

Thanks,
Tuan

On 02/01/2017 02:03 AM, Timo Teras wrote:
> On Wed, 1 Feb 2017 01:42:24 -0500
> "Tuan M. Hoang" <tmhoang@flatglobe.org> wrote:
>
>> On Mon, 30 Jan 2017 13:21:56 +0200
>> Timo Teras <timo.teras@iki.fi> wrote:
>>
>>>> Cross-compiling strace and gdb requires 'linux-headers'
>>>> installed in $CBUILDROOT. strace is cool but gdb still fails
>>>
>>> Should fix strace APKBUILD to support cross-compilation using the
>>> makedepends_host as used in other cross-compile enabled packages.
>>
>> Thanks, it's very neat.
>>
>>>> elf_gregset_t and elf_fpregset_t were supposed to be defined in
>>>> https://github.com/torvalds/linux/blob/master/arch/s390/include/asm/elf.h
>>>> and I think it should be built in 'linux-headers', but no trace of
>>>> them.
>>>
>>> Strange. Sounds like a bug somewhere.
>>
>> Indeed. It was a musl bug, and with some patches, gdb is good now.
>
> Ok. Good, perhaps you can submit them upstream to musl?
>
>> Looks like /usr/lib/libfakeroot.so is the one causes the segfault. Is
>> there any option in abuild to build package with out being stripped?
>> I would like to do so then debug fakeroot better. Adding -DDEBUG or
>> -g in CFLAGS and CXXFLAGS in APKBUILD doesn't help.
>
> DEBUG=yes abuild
>
> or
>
> add "$pkgname-dbg" to subpackages, and a separate -dbg package is
> created with the debug information.
>
>>>> Here is the log of strace fakeroot :
>>>> http://pastebin.com/Pn7YUWRz
>>>
>>> Could you retry with "strace -f -s 1024" ?
>>
>> I am not sure the strace would be informative for you :
>> http://pastebin.com/4P0f5sDF
>
> Unfortunately, this does not immediately reveal anything additional.
> Getting gdb backtrace should help.
>
> Thanks,
> Timo
>
>
> ---
> Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> Help:         alpine-devel+help@lists.alpinelinux.org
> ---
>