Date: Mon, 30 Nov 2015 08:21:37 +0100
From: Natanael Copa <ncopa@alpinelinux.org>
To: =?ISO-8859-1?B?U/ZyZW4=?= Tempel <soeren+git@soeren-tempel.net>
Cc: alpine-devel@lists.alpinelinux.org
Subject: Re: [alpine-devel] [PATCH 3/5] abuild: rewrite hardlink handling
 when compressing man pages
Message-ID: <20151130082137.00517e0e@ncopa-desktop.alpinelinux.org>
In-Reply-To: <1448581847-30376-3-git-send-email-soeren+git@soeren-tempel.net>
References: <1448581847-30376-1-git-send-email-soeren+git@soeren-tempel.net>
	<1448581847-30376-3-git-send-email-soeren+git@soeren-tempel.net>
Precedence: list
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Fri, 27 Nov 2015 00:50:45 +0100
S=F6ren Tempel <soeren+git@soeren-tempel.net> wrote:

> The problem is that gzip refuses to run if it detects that a file has
> more than 1 link. Our existing solution (removing hardlinks, compressing
> the man page and recreating the hardlinks) made certain assumptions
> about inode order that are only given on Unix v7 like filesystems
> meaning it didn't work properly on 'tree-based' filesystems like BTRFS
> or ZFS.
>=20
> This patch has a different more bulletproof approach: It simply replaces
> all hardlinks with symlinks. This is way easier because symlinks (unlike
> hardlinks) can point to a file that doesn't exist, therefore we can
> update all links before compressing the file in an easy way.

How well it this tested?

The other approach has build entire world at this point.

I am not very happy about rebuilding world again for this change.

-nc

> ---
>  abuild.in | 40 ++++++++++++++++++++++------------------
>  1 file changed, 22 insertions(+), 18 deletions(-)
>=20
> diff --git a/abuild.in b/abuild.in
> index 4c45504..26b7982 100644
> --- a/abuild.in
> +++ b/abuild.in
> @@ -1403,29 +1403,33 @@ default_doc() {
>  	done
> =20
>  	# compress man pages
> -	local previnode=3D prevname=3D  mandir=3D"$subpkgdir"/usr/share/man
> -	[ -d "$mandir" ] && find "$subpkgdir"/usr/share/man \
> -		-type f \( -name \*.[0-8n] -o -name \*.[0-8][a-z]* \) \
> -		-exec stat -c "%i %n" {} \; | sort -n \
> -		| while read inode name; do
> -
> -		if [ "$inode" =3D "$previnode" ]; then
> -			# update hard link
> -			rm "$name"
> -			ln "$prevname".gz "$name".gz
> -		else
> -			gzip -9 "$name"
> -		fi
> +	local mandir=3D"$subpkgdir"/usr/share/man
> +	[ -d "$mandir" ] && find "$mandir" -type f \
> +		-a \( -name \*.[0-8n] -o -name \*.[0-8][a-z]* \) \
> +		-exec stat -c "%i %n" \{\} \; | while read inode name; do
> +
> +		# Skip hardlinks removed in last iteration.
> +		[ -f "$name" ] || continue
> +
> +		local islink=3D0
> +		find "$mandir" -type f -links +1 \
> +			-a \( -name \*.[0-8n] -o -name \*.[0-8][a-z]* \) \
> +			-exec stat -c "%i %n" \{\} \; | while read linode lname; do
> +			if [ "$linode" =3D "$inode" -a "$lname" !=3D "$name" ]; then
> +				islink=3D1
> +				rm -f "$lname"
> +				ln -s "${name##*/}".gz "$lname".gz
> +			fi
> +		done

How is performance on this? It looks like this can be horribly slow if
$mandir is huge.

Could you try locate the biggest -doc package and run this on armhf and
post a specific number?


> =20
> -		previnode=3D"$inode"
> -		prevname=3D"$name"
> +		[ $islink -eq 0 ] && gzip -9 "$name"
>  	done
> -	[ -d "$mandir" ] && find "$subpkgdir"/usr/share/man \
> -		-type l \( -name \*.[0-8n] -o -name \*.[0-8][a-z]*
> \) \
> +	[ -d "$mandir" ] && find "$mandir" -type l \
> +		-a \( -name \*.[0-8n] -o -name \*.[0-8][a-z]* \) \
>  		| while read symlink; do
> =20
>  		ln -s $(readlink $symlink).gz "$symlink".gz
> -		rm "$symlink"
> +		rm -f "$symlink"
>  	done
> =20
>  	rm -f "$subpkgdir/usr/share/info/dir"

-nc


---
Unsubscribe:  alpine-devel+unsubscribe@lists.alpinelinux.org
Help:         alpine-devel+help@lists.alpinelinux.org
---