Precedence: list
MIME-Version: 1.0
In-Reply-To: <20170305050829.GA28810@riseup.net>
References: <20170304153449.GA24485@angua.1042.ch> <20170305050829.GA28810@riseup.net>
From: William Pitcock <nenolod@dereferenced.org>
Date: Tue, 7 Mar 2017 00:34:27 -0600
Message-ID: <CA+T2pCHF2PZE3HS6fEhoA1xJSniqdXpMjh9Uum0PpPkiUUD3ow@mail.gmail.com>
Subject: Re: [alpine-devel] Calling build() check() package() might be a bad order
To: Ivan Tham <pickfire@riseup.net>
Cc: alpine-dev <alpine-devel@lists.alpinelinux.org>
Content-Type: text/plain; charset=UTF-8

Hello,

On Sat, Mar 4, 2017 at 11:08 PM, Ivan Tham <pickfire@riseup.net> wrote:
> Yes, I think build() -> package() -> check() would be a nice idea. I
> suggest that check() can be optional and not be included as well if we
> are building for abuild since there is not much reason to do so.

The proposal which brought us check made it explicit that packages are
required to run testsuites if possible prior to generating a package.
This was on purpose: we want to ensure that what we are shipping (the
binaries generated by build()) properly passes upstream tests -- this
gives us a significant advantage in catching bugs before they affect
end users.
If a package does not wish to use check(), it can declare that it does
not want to use check() by declaring "!check" in $options.
In the future (2018), it will be considered a hard failure if the
check() function is missing and "!check" is not in $options.

Now on to the other thing (using check() for testing a debug build or similar):

> On Sat, Mar 04, 2017 at 04:34:49PM +0100, Jean-Louis Fuchs wrote:
>>
>> I wanted to point out that there might be a problem with the order in
>> which abuild calls build(), check() and package().
>>
>> My project has to be built different for testing and releasing. Since
>> abuild will call build() first, then check() and then package(), it is
>> built the wrong way either in check() or in package(). I am in the
>> fortunate position that my project can do out-of-source builds, but if
>> it couldn't I would have to do the following:
>>
>> build()
>>    Build for release
>>
>> check()
>>    Clean, build for test, test, clean and build for release

This is not what check() is for.  The point of check() is that what
you are putting in the package itself should be tested.
If a release build cannot be tested then you should opt-out of testing
by declaring "!check" in $options.
However, as an aside, I would strongly suggest that you make your
release builds formally testable, because that will allow you to make
sure that what you are actually releasing is in good shape.

>>
>> package()
>>    Package
>>
>> I know that any project that does coverage has to be rebuilt for
>> testing, so my project shouldn't be the only case. Also tests often
>> only work if assert is active, which it isn't in release mode.

Coverage tests and checking for assert() hits is not really what the
scope of check() is.
check() is for running formal testsuites against a "release" build,
using a set of tests and a test runner such as kyua (see main/pkgconf
package for an example of what a kyua-based testsuite looks like), or
the autotools test runner, etc.
If this does not apply to a package, then they should just skip tests
as passing/failing on binaries other than the ones we are going to
ship isn't directly relevant to us.

A way to look at it is that check() is for checking that the binaries
actually going into the package are kosher.

>>
>> Here is the out-of-source version of the package of my project:
>>
>> https://github.com/ganwell/chirp/blob/build_cleanup/dist/alpine/chirp/APKBUILD
>>
>> If check() would run at the end:
>>
>> build()
>>    Build for release
>>
>> package()
>>    Package
>>
>> check()
>>    Clean, build for test and test
>>
>> May be there is a much better way that I am not aware of, in that case
>> we should document this in the wiki.

There is no other way to do it: the point of check() coming after
build() and before package() is so that we can fail the build before
we package it.  This is an intentional design decision.
In your case, I would simply not bother running the testsuite at all,
because you're not testing the actual binaries going into the package,
making the testsuite ineffective (from a distribution point of view).

William


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---