X-Original-To: alpine-user@lists.alpinelinux.org Received: from mail-qt0-f179.google.com (mail-qt0-f179.google.com [209.85.216.179]) by lists.alpinelinux.org (Postfix) with ESMTP id 86E625C4E55 for ; Thu, 8 Feb 2018 22:38:44 +0000 (GMT) Received: by mail-qt0-f179.google.com with SMTP id u6so7875962qtg.13 for ; Thu, 08 Feb 2018 14:38:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=2cEisSclQ2ra6MRCijiesVPZ5regltYIYyj8+jlxzO0=; b=MuIc4PolivXLI1oMDQzbV0D8ALNXM+Cg9BRoe1zmhBczHxvgJWQQA/ix5uIFpGeXnm nxwFQL9Gq9WHwvhLvcX4CnCKQGKzKEfi9jb1/gzytKVV85/IQxfhOWAxjFadYDQDWpxx yDX1WOH8jqLRX+EV/yY2dZvleAOoSRneZ9wYumN1L2slteqJ48ZKe4kzECritN4fbple R5WiPuonnH+4srnPBHjPoiedUcXefPrVrbGKCoD9gA9IJ3O4S3W3RXFYrmM7iU/ACwyy MXiSIFzq1iQXxGz4kofildRnN7gXSChmgYS0K6xrOsdMKdnPUHZ2iDT431ht37QagML1 J0Ug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=2cEisSclQ2ra6MRCijiesVPZ5regltYIYyj8+jlxzO0=; b=qt69lFE0e0E2go1KwlfoT7eToGh7LoLmx7TmRPn8+vROAAycvP7Ae8nGgOqWJXqdnu FGBt/xWvdxG6iZtDSkNghxkPU2Z3gyBj+3hlvbssmK75HH/fUH7mB8hAAcs3Pxo8E5pL anrwB0qp6g3MGR3JeGRFsyAD1Z3iKkId+/n5UE7/bSM64rts6X0xQfRCcqy9NcicpPeL toW1HZ/Ld38/yx+O58E98h3MUiaVXVW9uVcg17A6k/vJ+eI53twokCViImVe5Fawwefe Ba0xpIl5G/54ZVVFzm19n/ZN6aRDg+S1ukJNviWI2aZyR089jJKYHL7DPzg7vIYnScPy /mIQ== X-Gm-Message-State: APf1xPC1dNsjAkPtmRUZxWA8PRr/ksIJ2damYTRECd/77bD+yx+ytZ2k ymAfI2qpQmQWElpPg7VshpgQj2gMf/O/60MSfd6moQ== X-Google-Smtp-Source: AH8x227Cwh1GPZcjvfYlXyCIYPF0zGIIBw/hNTZVQ7sEP4//e5q4ig8jQsfU8NfIu89veyrkTbi6SLX8ZdfAFfVgEeE= X-Received: by 10.237.37.178 with SMTP id x47mr1278613qtc.290.1518129523599; Thu, 08 Feb 2018 14:38:43 -0800 (PST) X-Mailinglist: alpine-user Precedence: list List-Id: Alpine Development List-Unsubscribe: List-Post: List-Help: List-Subscribe: MIME-Version: 1.0 Received: by 10.12.178.203 with HTTP; Thu, 8 Feb 2018 14:38:43 -0800 (PST) In-Reply-To: References: From: Andrew Bell Date: Thu, 8 Feb 2018 17:38:43 -0500 Message-ID: Subject: [alpine-user] Re: ld-musl not resolving symbols To: alpine-user@lists.alpinelinux.org Content-Type: multipart/alternative; boundary="001a11412a70fa2f6b0564bb12da" --001a11412a70fa2f6b0564bb12da Content-Type: text/plain; charset="UTF-8" Replying to my own note here in case it helps someone else. On Wed, Feb 7, 2018 at 1:49 PM, Andrew Bell wrote: > Hi, > > I'm trying to run a program I've written that opens a library with > dlopen("libname", RTLD_NOW). dlopen() succeeds without error and I can > use dlsym() to get a good address of an entry point in the .so, but global > symbols in the dlopen'ed library seem not to be resolved (they're null). > The symbols weren't null, but they weren't resolved to the previously resolved global symbols of the same name - they were given addresses known only to the dlopen'ed library and dependencies. > > The really strange thing is that this only happens if I run my code as a > python extension through the python2.7 binary. If I invoke my code through > my own driver, things work fine. > In my own driver, the library wasn't dlopen'ed as a python extension is, it was simply linked. > Is there anything that python could be doing that would tell ld-musl not > to resolve all the symbols when dlopen'ing a library? The documentation > I've seen says that ld-musl always resolves all the symbols in a library > when dlopen() is called. Are there exceptions to this? Is there some way > I can trace the behavior of ld-musl? > When python dlopen's an extension, by default it uses the flag RTLD_NOW, but not RTLD_GLOBAL. This means that the symbols defined in the library that were linked to the extension module weren't available to be resolved by subsequent dlopen() calls that were made by code invoked by the extension module. In python, this can be changed by setting the dlopen flags prior to importing the extension module: import sys import ctypes flags = sys.getdlopenflags() sys.setdlopenflags(flags | ctypes.RTLD_GLOBAL) from whatever import extentionmodule sys.setdlopenflags(flags) Alternatively, you can force the symbols you need exposed to be made global by calling from C/C++ after the library is loaded: dlopen("library.so", RTLD_NOLOAD | RTLD_GLOBAL); The bit I don't understand is why this behavior was different on Ubuntu. I read that ld-musl doesn't do lazy symbol resolution, but the documentation says that only applies to functions, not objects/data anyway. Is this perhaps a bug in GNU ld.so that I didn't notice because it worked as I hoped rather than as documented? -- Andrew Bell andrew.bell.ia@gmail.com --001a11412a70fa2f6b0564bb12da Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Replying to my own note here in case it helps someone else= .

On Wed, Feb 7, 2= 018 at 1:49 PM, Andrew Bell <andrew.bell.ia@gmail.com> wrote:
Hi,

I'm trying to run a program I've written that opens a libra= ry with dlopen("libname", RTLD_NOW).=C2=A0 =C2=A0dlopen() succeed= s without error and I can use dlsym() to=C2=A0get a good address of an entr= y point in the .so, but=C2=A0global symbols in the dlopen'ed library se= em not to be resolved (they're null).

=
The symbols weren't null, but they weren't resolved to t= he previously resolved global symbols of the same name - they were given ad= dresses known only to the dlopen'ed library and dependencies.
=C2=A0

The really strange thing is that this only happens if I run my code a= s a python extension through the python2.7 binary.=C2=A0 If I invoke my cod= e through my own driver, things work fine.
In my own driver, the library wasn't dlopen'ed as a pyt= hon extension is, it was simply linked.
=C2=A0
Is there anything that python could= be doing that would tell ld-musl=C2=A0not to resolve all the symbols when = dlopen'ing a library?=C2=A0 The documentation I've seen says that l= d-musl=C2=A0always resolves all the symbols in a library when dlopen() is c= alled.=C2=A0 Are there exceptions to this?=C2=A0 Is there some way I can tr= ace the behavior of ld-musl?

When python dlopen's an extension, by default it uses the flag RTLD_N= OW, but not RTLD_GLOBAL.=C2=A0 This means that the symbols defined in the l= ibrary that were linked to the extension module weren't available to be= resolved by subsequent dlopen() calls that were made by code invoked by th= e extension module.=C2=A0 In python, this can be changed by setting the dlo= pen flags prior to importing the extension module:

import sys
import ctypes
flags =3D sys.getdlopenflags(= )
sys.setdlopenflags(flags | ctypes.RTLD_GLOBAL)
from w= hatever import extentionmodule
sys.setdlopenflags(flags)
<= /div>

Alternatively, you can force the symbols you need = exposed to be made global by calling from C/C++ after the library is loaded= :

dlopen("library.so", RTLD_NOLOAD | RTL= D_GLOBAL);

The bit I don't understand is why t= his behavior was different on Ubuntu.=C2=A0 I read that ld-musl doesn't= do lazy symbol resolution, but the documentation says that only applies to= functions, not objects/data anyway.=C2=A0 Is this perhaps a bug in GNU ld.= so that I didn't notice because it worked as I hoped rather than as doc= umented?

--=C2=A0
Andrew Bell
andrew.bell.ia@gmail.com=
--001a11412a70fa2f6b0564bb12da-- --- Unsubscribe: alpine-user+unsubscribe@lists.alpinelinux.org Help: alpine-user+help@lists.alpinelinux.org ---