Date: Mon, 4 Aug 2014 12:59:07 -0700
From: Isaac Dunham <ibid.ag@gmail.com>
To: alpine-devel@lists.alpinelinux.org
Subject: [alpine-devel] numpy, scipy, openmp, and openblas
Message-ID: <20140804195906.GB1760@newbook>
References: <20140803002548.GA2919@newbook>
 <20140804051931.GA26972@newbook>
Precedence: list
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20140804051931.GA26972@newbook>
User-Agent: Mutt/1.5.23 (2014-03-12)

On Sun, Aug 03, 2014 at 10:19:31PM -0700, Isaac Dunham wrote:
> I haven't built scipy yet, but I've built the prerequisite that isn't
> packaged yet: a BLAS/LAPACK library.
> 
> For this, I built OpenBLAS with the following flags:
> USE_OPENMP=1 NO_LAPACK=0 NUM_THREADS=2 MAJOR_VERSION=3 NO_AFFINITY=1 TARGET=ATOM
> Justification:
> USE_OPENMP=1: I'm not sure this is sensible for me.
> NO_LAPACK=0:  Needed to get LAPACK

Having read up more, I've found that:
-To build scipy, numpy must be built with BLAS/LAPACK; OpenBLAS will work
 for both (or so say the numpy docs...)
-GNU OpenMP (GOMP) has an issue that makes it impossible to use safely
in libraries that may be used in forking processes; this means that anything
based on current GNU OpenMP should not be used with Python < 3.4.
There's a patch from the OpenBLAS developers for this, but I'm not sure what
its status is; the first two versions got rejected and the bugs closed as
invalid.
-The Intel OpenMP runtime should work. It's ABI compatible with GOMP.
-NUM_THREADS is an alternative to OpenMP for threading.
It works with forking processes.

> NUM_THREADS=2: I have a hyperthreading Atom N270.
> The recommended value is actually 1 thread per hardware core/FPU
> (hyperthreaded cores or Bulldozer cores divided by 2).
> NO_AFFINITY=1: Needed for some software to work right; this may be needed
> to build with some versions of musl as well.
> TARGET=ATOM: This is what I have.
> 
> As NUM_THREADS increases, so does the size of internal structures.
> 
> For packaging, I suppose it would be sensible to use something like this:
> USE_OPENMP=1 NO_LAPACK=0 NUM_THREADS=16 NO_AFFINITY=1 TARGET=???
> NUM_THREADS=16 is half the 32 cores that our kernel supports, because most
> systems that have that many cores are Bulldozer or Intel (hyperthreading),
> and setting it high will bloat the library.
> I'm not sure what to say for TARGET; sensible values might be P2,
> SSE_GENERIC, or possibly using DYNAMIC_CORE instead.
> 
> This package required a trivial patch, which I've sent them:
> it used get_nprocs(), which is the same as sysconf(_SC_NPROCESSORS_ONLN).
Was accepted upstream the same afternoon.

Thanks,
Isaac Dunham


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---