Received: from mail-lf1-f48.google.com (mail-lf1-f48.google.com [209.85.167.48]) by nld3-dev1.alpinelinux.org (Postfix) with ESMTPS id F25FA782BDA for <~alpine/users@lists.alpinelinux.org>; Fri, 22 May 2020 04:51:08 +0000 (UTC) Received: by mail-lf1-f48.google.com with SMTP id z206so2137836lfc.6 for <~alpine/users@lists.alpinelinux.org>; Thu, 21 May 2020 21:51:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=f9q9wVOoPbFM+/4nPnpMGiBcCa2NB/2LpmIZWNBlohY=; b=UZoSovI1wS50k1Ak4DkjCDAaDebJ4/RL56dCUfnA/LSWQYbQdYHlRXzalll71Xg3r7 PfuoMNSBTNsRdpB4ZjbkPhUNja7xuj0PZKpATZ5NQemz0+m11UJZ08WucvkCrBzHiIsM bNUB7t5tyWLw0fdNgw0LeysTU27AqU+q1Af6SZVpwESgw1VogRCDTqsoL2Vj+siw+1uw vcB0BH/nnFkclq+y5eiUyXWph8+wEmkk4pwfKJDOogWyDPWh0q+IfCCa/hZcyV/Wa0UG Z+GgnETF6SwIgmxu+WtO/2PfHB7SNb8Tffw3pg2Lt+UmYxUyCmfIAzjiwgFodx1SooWE A8zA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=f9q9wVOoPbFM+/4nPnpMGiBcCa2NB/2LpmIZWNBlohY=; b=q1g17O8DO4f5EzbMPQJzVAgbU6/Z9OL2N71OFA2R9RR8+U3ayQWzT0kKGZ/5vw2gbF cqDcXa2OaScEgftUyH9H8oJeXr0O8o893gtAUmqhpNsJ0WSB3cPrcTBgybiSIhR4PUB/ E9C3ebsfo9J9nYNT+qRmWbsQ0aeF/yeEmFvWir8DodBm9G1qPKUjGEZuO+oe+Zykmp08 G6K4LG+MbU5OCXPTPVNPr5yD2hh6EKhEgCHkCAC4yAjS8VvyWKikGbs+PEcXH1D9tc6c ZOt4p+u0LxpVS57egLZ16J04cjkO5/d7k/mqTu6rdZ1YAmBUaa+xppgDEq76Pq7YUIhe Lcdw== X-Gm-Message-State: AOAM532vqalPPWWabOagY3Hz66HZ63GoBmMOvdsmhRcD1aF6dirOnN+M qAqLv8Ii4cWy2j+5JHvM/Nvz/l2bYtI3GPCiCEw= X-Google-Smtp-Source: ABdhPJzielTwS/tvNndEjFW0CZ8O+ENAD/llFr6GosX/iuTrSS6xrPjpI38/1Ml6MmI2CoNtqaoOobteeuf7iNv/plo= X-Received: by 2002:a19:2250:: with SMTP id i77mr6752468lfi.133.1590123067595; Thu, 21 May 2020 21:51:07 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: PICCORO McKAY Lenz Date: Fri, 22 May 2020 00:50:55 -0400 Message-ID: Subject: Re: curl large header issue https To: Axel U Cc: ~alpine/users@lists.alpinelinux.org Content-Type: multipart/alternative; boundary="00000000000097df0605a635603b" --00000000000097df0605a635603b Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable https://github.com/curl/curl/issues/659 ???? Lenz McKAY Gerardo (PICCORO) http://qgqlochekone.blogspot.com El jue., 21 de may. de 2020 a la(s) 16:11, Axel U (ulrich.axel@gmail.com) escribi=C3=B3: > If someone could follow steps below to confirm what I am seeing, I > would appreciate it, to exclude something wrong with my setup. > > Environment Alpine 3.11 > $ curl --version > curl 7.67.0 (x86_64-alpine-linux-musl) libcurl/7.67.0 OpenSSL/1.1.1g > zlib/1.2.11 nghttp2/1.40.0 > .. > > This is not a discussion that servers should reject large headers for > security reasons or similar. > I found the issue originally when running an existing set of test > scripts that check server behavior for http1.1 requests with large > headers. When running those test scripts on an Alpine host, curl > reports an empty server response error (52), but the request with curl > succeeds on another distro against the same server. Also, wget on > Alpine has no issue with essentially the same request against the same > server. So the test scripts in question would have to be adapted when > running them on an Alpine host to use wget instead of curl, but before > doing so, it smells like a curl issue on Alpine, so if someone could > confirm, I would appreciate it, to exclude something wrong with my > setup. I tried it on two Alpine hosts I have access to with the same > result. I tried the same on another distro and don=E2=80=99t have the iss= ue. > > Note: Below steps are only set to http 0.9 because of ncat, as I tried > to come up with the easiest setup to reproduce using standard tools > and one line commands. So if you find or already run or want to setup > a http/https server that accepts large header sizes to test this > against and echos it back, then run this for http1.1 (to get error > 52). This occurs on https only. > > Steps to reproduce > > on the host we need curl, wget (to replace BusyBox=E2=80=99s wget > implementation), findutils (to replace BusyBox=E2=80=99s xargs > implementation), nmap-ncat and perl > > In Step 2 below change the multiplier (example has it as 2000) to vary > the header size; for me up to 1600 success, 1700+ error. > > We need two terminals open: terminal one and terminal two > > Step 1 in terminal one (server side): > $ ncat -lvp 48443 --ssl -c 'xargs -n1 echo' > > Step 2 in terminal two (client side): > $ perl -e '$b=3D`curl --insecure --http0.9 --silent --show-error -H > foo:@{["0123456789"x2000]} -H zzz:here https://127.0.0.1:48443/`;print > > $b;' > > Step 3 in terminal one: > Press Ctrl-C > > Step 4 in terminal two: > Observe the output - reports an error. > > > Repeat above using wget instead of curl in Step 2 in terminal two =3D no > issue: > $ perl -e '$b=3D`wget --no-check-certificate -O - > --header=3D"foo:@{["0123456789"x2000]}" --header=3D"zzz:here" > https://127.0.0.1:48443/`;print $b;' > > > Repeat above with curl against http server =3D no issue: > Step 1 in terminal one (server side): > $ ncat -lvp 48080 -c 'xargs -n1 echo' > > Step 2 in terminal two (client side): > $ perl -e '$b=3D`curl --silent --http0.9 --show-error -H > foo:@{["0123456789"x2000]} -H zzz:here http://127.0.0.1:48080/`;print > > $b;' > > Step 3 and 4 as above. > > > Repeat original steps unchanged (curl/https) on another distro that > has same tools installed =3D no issue. > --00000000000097df0605a635603b Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
= El jue., 21 de may. de 2020 a la(s) 16:11, Axel U (ulrich.axel@gmail.com) escribi=C3=B3:
If someone could follow steps be= low to confirm what I am seeing, I
would appreciate it, to exclude something wrong with my setup.

Environment Alpine 3.11
$ curl --version
curl 7.67.0 (x86_64-alpine-linux-musl) libcurl/7.67.0 OpenSSL/1.1.1g
zlib/1.2.11 nghttp2/1.40.0
..

This is not a discussion that servers should reject large headers for
security reasons or similar.
I found the issue originally when running an existing set of test
scripts that check server behavior for http1.1 requests with large
headers. When running those test scripts on an Alpine host, curl
reports an empty server response error (52), but the request with curl
succeeds on another distro against the same server.=C2=A0 Also, wget on
Alpine has no issue with essentially the same request against the same
server. So the test scripts in question would have to be adapted when
running them on an Alpine host to use wget instead of curl, but before
doing so, it smells like a curl issue on Alpine, so if someone could
confirm, I would appreciate it, to exclude something wrong with my
setup. I tried it on two Alpine hosts I have access to with the same
result. I tried the same on another distro and don=E2=80=99t have the issue= .

Note: Below steps are only set to http 0.9 because of ncat, as I tried
to come up with the easiest setup to reproduce using standard tools
and one line commands.=C2=A0 So if you find or already run or want to setup=
a http/https server that accepts large header sizes to test this
against and echos it back, then run this for http1.1 (to get error
52). This occurs on https only.

Steps to reproduce

on the host we need curl, wget (to replace BusyBox=E2=80=99s wget
implementation), findutils (to replace BusyBox=E2=80=99s xargs
implementation), nmap-ncat and perl

In Step 2 below change the multiplier (example has it as 2000) to vary
the header size; for me up to 1600 success, 1700+ error.

We need two terminals open: terminal one and terminal two

Step 1 in terminal one (server side):
$ ncat -lvp 48443 --ssl -c 'xargs -n1 echo'

Step 2 in terminal two (client side):
$ perl -e '$b=3D`curl --insecure --http0.9 --silent --show-error -H
foo:@{["0123456789"x2000]} -H zzz:here https://127.0.0.1:484= 43/`;print
$b;'

Step 3 in terminal one:
Press Ctrl-C

Step 4 in terminal two:
Observe the output - reports an error.


Repeat above using wget instead of curl in Step 2 in terminal two =3D no is= sue:
$ perl -e '$b=3D`wget --no-check-certificate -O -
--header=3D"foo:@{["0123456789"x2000]}" --header=3D&quo= t;zzz:here"
https://127.0.0.1:48443/`;print $b;'


Repeat above with curl against http server =3D no issue:
Step 1 in terminal one (server side):
$ ncat -lvp 48080 -c 'xargs -n1 echo'

Step 2 in terminal two (client side):
$ perl -e '$b=3D`curl --silent --http0.9 --show-error -H
foo:@{["0123456789"x2000]} -H zzz:here http://127.0.0.1:48080= /`;print
$b;'

Step 3 and 4 as above.


Repeat original steps unchanged (curl/https) on another distro that
has same tools installed =3D no issue.
--00000000000097df0605a635603b--