~alpine/devel

5 5

[alpine-devel] Proposal: testsuite support in APKBUILDs

William Pitcock <nenolod@dereferenced.org>
Details
Message ID
<CA+T2pCEB9g=un1jx0x2k+wKNQWeBt8hijEwq1Yy+qLLvb0OVrw@mail.gmail.com>
Sender timestamp
1485401880
DKIM signature
missing
Download raw message
Hello,

A concern I hear occasionally from people when discussing Alpine as a
distribution they could use for their projects is that Alpine does not
typically run testsuites in APKBUILDs.  This is largely because abuild
does not have awareness of testsuites.

As such, I propose we add a testsuite() phase to the build process,
that runs after build(), but prior to package().  APKBUILDs should
override the testsuite phase, but the default_testsuite() function
will be a noop for the Alpine 3.6 release cycle that raises a warning
that the APKBUILD does not define any mechanism for running a
testsuite.  If "!testsuite" is defined in $features, the warning is
ignored.

For cross-builds, the testsuite would be skipped unless the package is noarch.

After Alpine 3.6 (or maybe 3.7), we change the warning to a fatal
error, requiring APKBUILDs to explicitly opt out of running a
testsuite through $features.

Thoughts?


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Timo Teras <timo.teras@iki.fi>
Details
Message ID
<20170126112655.44ad5c0e@vostro.util.wtbts.net>
In-Reply-To
<CA+T2pCEB9g=un1jx0x2k+wKNQWeBt8hijEwq1Yy+qLLvb0OVrw@mail.gmail.com> (view parent)
Sender timestamp
1485422815
DKIM signature
missing
Download raw message
On Wed, 25 Jan 2017 21:38:00 -0600
William Pitcock <nenolod@dereferenced.org> wrote:

> As such, I propose we add a testsuite() phase to the build process,
> that runs after build(), but prior to package().  APKBUILDs should
> override the testsuite phase, but the default_testsuite() function
> will be a noop for the Alpine 3.6 release cycle that raises a warning
> that the APKBUILD does not define any mechanism for running a
> testsuite.  If "!testsuite" is defined in $features, the warning is
> ignored.
> 
> For cross-builds, the testsuite would be skipped unless the package
> is noarch.
> 
> After Alpine 3.6 (or maybe 3.7), we change the warning to a fatal
> error, requiring APKBUILDs to explicitly opt out of running a
> testsuite through $features.
> 
> Thoughts?

+1

We have some packages that run 'make check' or similar, but it would be
good to make it a separate step.

The above plan sounds good to me.

Thanks,
Timo


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Natanael Copa <ncopa@alpinelinux.org>
Details
Message ID
<20170126122237.22ed77fa@ncopa-desktop.copa.dup.pw>
In-Reply-To
<CA+T2pCEB9g=un1jx0x2k+wKNQWeBt8hijEwq1Yy+qLLvb0OVrw@mail.gmail.com> (view parent)
Sender timestamp
1485429757
DKIM signature
missing
Download raw message
On Wed, 25 Jan 2017 21:38:00 -0600
William Pitcock <nenolod@dereferenced.org> wrote:

> Hello,
> 
> A concern I hear occasionally from people when discussing Alpine as a
> distribution they could use for their projects is that Alpine does not
> typically run testsuites in APKBUILDs.  This is largely because abuild
> does not have awareness of testsuites.
> 
> As such, I propose we add a testsuite() phase to the build process,
> that runs after build(), but prior to package().  APKBUILDs should
> override the testsuite phase, but the default_testsuite() function
> will be a noop for the Alpine 3.6 release cycle that raises a warning
> that the APKBUILD does not define any mechanism for running a
> testsuite.  If "!testsuite" is defined in $features, the warning is
> ignored.
> 
> For cross-builds, the testsuite would be skipped unless the package is noarch.
> 
> After Alpine 3.6 (or maybe 3.7), we change the warning to a fatal
> error, requiring APKBUILDs to explicitly opt out of running a
> testsuite through $features.
> 
> Thoughts?

Sounds good to me.

-nc


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Jakub Jirutka <jakub@jirutka.cz>
Details
Message ID
<4EDA0683-3991-44D0-882C-C34FBD4C38B2@jirutka.cz>
In-Reply-To
<20170126112655.44ad5c0e@vostro.util.wtbts.net> (view parent)
Sender timestamp
1485436109
DKIM signature
missing
Download raw message
Hi,

currently we have the following phases: sanitycheck, builddeps, clean, fetch, unpack, prepare, mkusers, rootpkg, cleanup. All of them except rootpkg are verbs. So I’d prefer “check” (as Arch) instead of “testsuite”.

Jakub

> On 26. Jan 2017, at 10:26, Timo Teras <timo.teras@iki.fi> wrote:
> 
> On Wed, 25 Jan 2017 21:38:00 -0600
> William Pitcock <nenolod@dereferenced.org> wrote:
> 
>> As such, I propose we add a testsuite() phase to the build process,
>> that runs after build(), but prior to package().  APKBUILDs should
>> override the testsuite phase, but the default_testsuite() function
>> will be a noop for the Alpine 3.6 release cycle that raises a warning
>> that the APKBUILD does not define any mechanism for running a
>> testsuite.  If "!testsuite" is defined in $features, the warning is
>> ignored.
>> 
>> For cross-builds, the testsuite would be skipped unless the package
>> is noarch.
>> 
>> After Alpine 3.6 (or maybe 3.7), we change the warning to a fatal
>> error, requiring APKBUILDs to explicitly opt out of running a
>> testsuite through $features.
>> 
>> Thoughts?
> 
> +1
> 
> We have some packages that run 'make check' or similar, but it would be
> good to make it a separate step.
> 
> The above plan sounds good to me.
> 
> Thanks,
> Timo
> 
> 
> ---
> Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> Help:         alpine-devel+help@lists.alpinelinux.org
> ---
> 



---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---
Details
Message ID
<v3qvsh.oke9lb.1hge1d6-qmf@gmx.com>
In-Reply-To
<4EDA0683-3991-44D0-882C-C34FBD4C38B2@jirutka.cz> (view parent)
Sender timestamp
1485446156
DKIM signature
missing
Download raw message
Hey,
 
Since we're talking about vocabulary and semantics, I would rather avoid check, since it doesn't reflect what happens. A testsuite is a suite (set of, ordered, predictable) of tests (dynamic procedure to validate an element, that has to be executed). A check is merely an observation with assertion on an existing state. As an example, you can check the door is open, but you have to test if the lock works (you can't just check it in normal conditions and without a specialized apparatus).
 
Long story short, I don't have any problem with making it shorter and simpler, but I would rather have something that matches what is actually does. :)
 
Theo.

On Thu Jan 26 14:08:29 2017 GMT+0100, Jakub Jirutka wrote:
> Hi,
> 
> currently we have the following phases: sanitycheck, builddeps, clean, fetch, unpack, prepare, mkusers, rootpkg, cleanup. All of them except rootpkg are verbs. So I’d prefer “check” (as Arch) instead of “testsuite”.
> 
> Jakub
> 
> > On 26. Jan 2017, at 10:26, Timo Teras <timo.teras@iki.fi> wrote:
> > 
> > On Wed, 25 Jan 2017 21:38:00 -0600
> > William Pitcock <nenolod@dereferenced.org> wrote:
> > 
> >> As such, I propose we add a testsuite() phase to the build process,
> >> that runs after build(), but prior to package().  APKBUILDs should
> >> override the testsuite phase, but the default_testsuite() function
> >> will be a noop for the Alpine 3.6 release cycle that raises a warning
> >> that the APKBUILD does not define any mechanism for running a
> >> testsuite.  If "!testsuite" is defined in $features, the warning is
> >> ignored.
> >> 
> >> For cross-builds, the testsuite would be skipped unless the package
> >> is noarch.
> >> 
> >> After Alpine 3.6 (or maybe 3.7), we change the warning to a fatal
> >> error, requiring APKBUILDs to explicitly opt out of running a
> >> testsuite through $features.
> >> 
> >> Thoughts?
> > 
> > +1
> > 
> > We have some packages that run 'make check' or similar, but it would be
> > good to make it a separate step.
> > 
> > The above plan sounds good to me.
> > 
> > Thanks,
> > Timo
> > 
> > 
> > ---
> > Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> > Help:         alpine-devel+help@lists.alpinelinux.org
> > ---
> > 
> 
> 
> 
> ---
> Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
> Help:         alpine-devel+help@lists.alpinelinux.org
> ---
> 
>
Łukasz Jendrysik <scadu@yandex.com>
Details
Message ID
<9E6F2B08-FE29-48A8-BCC9-7D5B0547B35D@yandex.com>
In-Reply-To
<4EDA0683-3991-44D0-882C-C34FBD4C38B2@jirutka.cz> (view parent)
Sender timestamp
1485445854
DKIM signature
missing
Download raw message
+1

On January 26, 2017 2:08:29 PM GMT+01:00, Jakub Jirutka <jakub@jirutka.cz> wrote:
>Hi,
>
>currently we have the following phases: sanitycheck, builddeps, clean,
>fetch, unpack, prepare, mkusers, rootpkg, cleanup. All of them except
>rootpkg are verbs. So I’d prefer “check” (as Arch) instead of
>“testsuite”.
>
>Jakub
>
>> On 26. Jan 2017, at 10:26, Timo Teras <timo.teras@iki.fi> wrote:
>> 
>> On Wed, 25 Jan 2017 21:38:00 -0600
>> William Pitcock <nenolod@dereferenced.org> wrote:
>> 
>>> As such, I propose we add a testsuite() phase to the build process,
>>> that runs after build(), but prior to package().  APKBUILDs should
>>> override the testsuite phase, but the default_testsuite() function
>>> will be a noop for the Alpine 3.6 release cycle that raises a
>warning
>>> that the APKBUILD does not define any mechanism for running a
>>> testsuite.  If "!testsuite" is defined in $features, the warning is
>>> ignored.
>>> 
>>> For cross-builds, the testsuite would be skipped unless the package
>>> is noarch.
>>> 
>>> After Alpine 3.6 (or maybe 3.7), we change the warning to a fatal
>>> error, requiring APKBUILDs to explicitly opt out of running a
>>> testsuite through $features.
>>> 
>>> Thoughts?
>> 
>> +1
>> 
>> We have some packages that run 'make check' or similar, but it would
>be
>> good to make it a separate step.
>> 
>> The above plan sounds good to me.
>> 
>> Thanks,
>> Timo
>> 
>> 
>> ---
>> Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
>> Help:         alpine-devel+help@lists.alpinelinux.org
>> ---
>> 
>
>
>
>---
>Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
>Help:         alpine-devel+help@lists.alpinelinux.org
>---

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
Reply to thread Export thread (mbox)