Hi
I wanted to point out that there might be a problem with the order in
which abuild calls build(), check() and package().
My project has to be built different for testing and releasing. Since
abuild will call build() first, then check() and then package(), it is
built the wrong way either in check() or in package(). I am in the
fortunate position that my project can do out-of-source builds, but if
it couldn't I would have to do the following:
build()
Build for release
check()
Clean, build for test, test, clean and build for release
package()
Package
I know that any project that does coverage has to be rebuilt for
testing, so my project shouldn't be the only case. Also tests often
only work if assert is active, which it isn't in release mode.
Here is the out-of-source version of the package of my project:
https://github.com/ganwell/chirp/blob/build_cleanup/dist/alpine/chirp/APKBUILD
If check() would run at the end:
build()
Build for release
package()
Package
check()
Clean, build for test and test
May be there is a much better way that I am not aware of, in that case
we should document this in the wiki.
Best,
Jean-Louis
Yes, I think build() -> package() -> check() would be a nice idea. I
suggest that check() can be optional and not be included as well if we
are building for abuild since there is not much reason to do so.
On Sat, Mar 04, 2017 at 04:34:49PM +0100, Jean-Louis Fuchs wrote:
>I wanted to point out that there might be a problem with the order in>which abuild calls build(), check() and package().>>My project has to be built different for testing and releasing. Since>abuild will call build() first, then check() and then package(), it is>built the wrong way either in check() or in package(). I am in the>fortunate position that my project can do out-of-source builds, but if>it couldn't I would have to do the following:>>build()> Build for release>>check()> Clean, build for test, test, clean and build for release>>package()> Package>>I know that any project that does coverage has to be rebuilt for>testing, so my project shouldn't be the only case. Also tests often>only work if assert is active, which it isn't in release mode.>>Here is the out-of-source version of the package of my project:>https://github.com/ganwell/chirp/blob/build_cleanup/dist/alpine/chirp/APKBUILD>>If check() would run at the end:>>build()> Build for release>>package()> Package>>check()> Clean, build for test and test>>May be there is a much better way that I am not aware of, in that case>we should document this in the wiki.
--
Do what you like, like what you do. -- Pickfire
Hello,
On Sat, Mar 4, 2017 at 11:08 PM, Ivan Tham <pickfire@riseup.net> wrote:
> Yes, I think build() -> package() -> check() would be a nice idea. I> suggest that check() can be optional and not be included as well if we> are building for abuild since there is not much reason to do so.
The proposal which brought us check made it explicit that packages are
required to run testsuites if possible prior to generating a package.
This was on purpose: we want to ensure that what we are shipping (the
binaries generated by build()) properly passes upstream tests -- this
gives us a significant advantage in catching bugs before they affect
end users.
If a package does not wish to use check(), it can declare that it does
not want to use check() by declaring "!check" in $options.
In the future (2018), it will be considered a hard failure if the
check() function is missing and "!check" is not in $options.
Now on to the other thing (using check() for testing a debug build or similar):
> On Sat, Mar 04, 2017 at 04:34:49PM +0100, Jean-Louis Fuchs wrote:>>>> I wanted to point out that there might be a problem with the order in>> which abuild calls build(), check() and package().>>>> My project has to be built different for testing and releasing. Since>> abuild will call build() first, then check() and then package(), it is>> built the wrong way either in check() or in package(). I am in the>> fortunate position that my project can do out-of-source builds, but if>> it couldn't I would have to do the following:>>>> build()>> Build for release>>>> check()>> Clean, build for test, test, clean and build for release
This is not what check() is for. The point of check() is that what
you are putting in the package itself should be tested.
If a release build cannot be tested then you should opt-out of testing
by declaring "!check" in $options.
However, as an aside, I would strongly suggest that you make your
release builds formally testable, because that will allow you to make
sure that what you are actually releasing is in good shape.
>>>> package()>> Package>>>> I know that any project that does coverage has to be rebuilt for>> testing, so my project shouldn't be the only case. Also tests often>> only work if assert is active, which it isn't in release mode.
Coverage tests and checking for assert() hits is not really what the
scope of check() is.
check() is for running formal testsuites against a "release" build,
using a set of tests and a test runner such as kyua (see main/pkgconf
package for an example of what a kyua-based testsuite looks like), or
the autotools test runner, etc.
If this does not apply to a package, then they should just skip tests
as passing/failing on binaries other than the ones we are going to
ship isn't directly relevant to us.
A way to look at it is that check() is for checking that the binaries
actually going into the package are kosher.
>>>> Here is the out-of-source version of the package of my project:>>>> https://github.com/ganwell/chirp/blob/build_cleanup/dist/alpine/chirp/APKBUILD>>>> If check() would run at the end:>>>> build()>> Build for release>>>> package()>> Package>>>> check()>> Clean, build for test and test>>>> May be there is a much better way that I am not aware of, in that case>> we should document this in the wiki.
There is no other way to do it: the point of check() coming after
build() and before package() is so that we can fail the build before
we package it. This is an intentional design decision.
In your case, I would simply not bother running the testsuite at all,
because you're not testing the actual binaries going into the package,
making the testsuite ineffective (from a distribution point of view).
William
---
Unsubscribe: alpine-devel+unsubscribe@lists.alpinelinux.org
Help: alpine-devel+help@lists.alpinelinux.org
---
Hi William
On Tue, Mar 07, 2017 at 12:34:27AM -0600, William Pitcock wrote:
> I would simply not bother running the testsuite at all,> because you're not testing the actual binaries going into the package,> making the testsuite ineffective (from a distribution point of view).
I think I got it, I will change my build system the following way:
make check
Not instrumented (release mode), goal: checking compatibility
make test
Instrumented (dev mode), goal: helping developers to find bugs
Of course most of tests are shared between both test-suites, but some
may not make sense in either test-suite.
Best,
Jean-Louis
Hi,
On Wed, Mar 8, 2017 at 7:19 AM, Jean-Louis Fuchs <ganwell@fangorn.ch> wrote:
> Hi William>> On Tue, Mar 07, 2017 at 12:34:27AM -0600, William Pitcock wrote:>>> I would simply not bother running the testsuite at all,>> because you're not testing the actual binaries going into the package,>> making the testsuite ineffective (from a distribution point of view).>> I think I got it, I will change my build system the following way:>> make check> Not instrumented (release mode), goal: checking compatibility>> make test> Instrumented (dev mode), goal: helping developers to find bugs>> Of course most of tests are shared between both test-suites, but some> may not make sense in either test-suite.
Absolutely this is the way to go. Then run "make check" from the APKBUILD.
William
---
Unsubscribe: alpine-devel+unsubscribe@lists.alpinelinux.org
Help: alpine-devel+help@lists.alpinelinux.org
---