Delivery-date: Thu, 27 Jun 2024 05:11:18 -0700
Received-SPF: pass (google.com: domain of darosior@protonmail.com designates 185.70.40.138 as permitted sender) client-ip=185.70.40.138;
Date: Thu, 27 Jun 2024 09:35:59 +0000
To: Eric Voskuil <eric@voskuil.org>
From: "'Antoine Poinsot' via Bitcoin Development Mailing List" <bitcoindev@googlegroups.com>
Cc: Bitcoin Development Mailing List <bitcoindev@googlegroups.com>
Subject: Re: [bitcoindev] Re: Great Consensus Cleanup Revival
Message-ID: <jJLDrYTXvTgoslhl1n7Fk9-pL1mMC-0k6gtoniQINmioJpzgtqrJ_WqyFZkLltsCUusnQ4jZ6HbvRC-mGuaUlDi3kcqcFHALd10-JQl-FMY=@protonmail.com>
In-Reply-To: <be78e733-6e9f-4f4e-8dc2-67b79ddbf677n@googlegroups.com>
References: <gnM89sIQ7MhDgI62JciQEGy63DassEv7YZAMhj0IEuIo0EdnafykF6RH4OqjTTHIHsIoZvC2MnTUzJI7EfET4o-UQoD-XAQRDcct994VarE=@protonmail.com> <72e83c31-408f-4c13-bff5-bf0789302e23n@googlegroups.com> <heKH68GFJr4Zuf6lBozPJrb-StyBJPMNvmZL0xvKFBnBGVA3fVSgTLdWc-_8igYWX8z3zCGvzflH-CsRv0QCJQcfwizNyYXlBJa_Kteb2zg=@protonmail.com> <5b0331a5-4e94-465d-a51d-02166e2c1937n@googlegroups.com> <yt1O1F7NiVj-WkmnYeta1fSqCYNFx8h6OiJaTBmwhmJ2MWAZkmmjPlUST6FM7t6_-2NwWKdglWh77vcnEKA8swiAnQCZJY2SSCAh4DOKt2I=@protonmail.com> <be78e733-6e9f-4f4e-8dc2-67b79ddbf677n@googlegroups.com>
Feedback-ID: 7060259:user:proton
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="b1_1CgWDOX9rJbJHO7L15t9f9KK006luTO7qO3U6fzgbI"
Reply-To: Antoine Poinsot <darosior@protonmail.com>
Precedence: list
Mailing-list: list bitcoindev@googlegroups.com; contact bitcoindev+owners@googlegroups.com

This is a multi-part message in MIME format.

--b1_1CgWDOX9rJbJHO7L15t9f9KK006luTO7qO3U6fzgbI
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

> It is not clear to me how determining the coinbase size can be done at an=
 earlier stage of validation than detection of the non-null coinbase.

My point wasn't about checking the coinbase size, it was about being able t=
o cache the hash of a (non-malleated) invalid block as permanently invalid =
to avoid re-downloading and re-validating it.

> It seems to me that introducing an arbitrary tx size validity may create =
more potential implementation bugs than it resolves.

The potential for implementation bugs is a fair point to raise, but in this=
 case i don't think it's a big concern. Verifying no transaction in a block=
 is 64 bytes is as simple a check as you can get.

> And certainly anyone implementing such a verifier must know many intricac=
ies of the protocol.

They need to know some, but i don't think it's reasonable to expect them to=
 realize the merkle tree construction is such that an inner node may be con=
fused with a 64 bytes transaction.

> I do not see this. I see a very ugly perpetual seam which will likely res=
ult in unexpected complexities over time.

What makes you think making 64 bytes transactions invalid could result in u=
nexpected complexities? And why do you think it's likely?

> This does not produce unmalleable block hashes. Duplicate tx hash malleat=
ion remains in either case, to the same effect. Without a resolution to bot=
h issues this is an empty promise.

Duplicate txids have been invalid since 2012 (CVE-2012-2459). If 64 bytes t=
ransactions are also made invalid, this would make it impossible for two va=
lid blocks to have the same hash.

Best,
Antoine
On Monday, June 24th, 2024 at 2:35 AM, Eric Voskuil <eric@voskuil.org> wrot=
e:

> Thanks for the responses Antoine.
>
>> As discussed here it would let node implementations cache block failures=
 at an earlier stage of validation. Not a large gain, but still nice to hav=
e.
>
> It is not clear to me how determining the coinbase size can be done at an=
 earlier stage of validation than detection of the non-null coinbase. The f=
ormer requires parsing the coinbase to determine its size, the latter requi=
res parsing it to know if the point is null. Both of these can be performed=
 as early as immediately following the socket read.
>
> size check
>
> (1) requires new consensus rule: 64 byte transactions (or coinbases?) are=
 invalid.
> (2) creates a consensus "seam" (complexity) in txs, where < 64 bytes and =
> 64 bytes are potentially valid.
> (3) can be limited to reading/skipping header (80 bytes) plus parsing 0 -=
 65 coinbase bytes.
>
> point check
>
> (1) requires no change.
> (2) creates no consensus seam.
> (3) can be limited to reading/skipping header (80 bytes) plus parsing 6 -=
 43 coinbase bytes.
>
> Not only is this not a large (performance) gain, it's not one at all.
>
>> It would also avoid a large footgun for anyone implementing a software v=
erifying an SPV proof verifier and not knowing the intricacies of the proto=
col...
>
> It seems to me that introducing an arbitrary tx size validity may create =
more potential implementation bugs than it resolves. And certainly anyone i=
mplementing such a verifier must know many intricacies of the protocol. Thi=
s does not remove one, it introduces another - as there is not only a bifur=
cation around tx size but one around the question of whether this rule is a=
ctive.
>
>> Finally, it would get rid of a large footgun in general.
>
> I do not see this. I see a very ugly perpetual seam which will likely res=
ult in unexpected complexities over time.
>
>> Certainly, unique block hashes would be a useful property for Bitcoin to=
 have. It's not far-fetched to expect current or future Bitcoin-related sof=
tware to rely on this.
>
> This does not produce unmalleable block hashes. Duplicate tx hash malleat=
ion remains in either case, to the same effect. Without a resolution to bot=
h issues this is an empty promise.
>
> The only possible benefit that I can see here is the possible very small =
bandwidth savings pertaining to SPV proofs. I would have a very hard time j=
ustifying adding any consensus rule to achieve only that result.
>
> Best,
> Eric
>
> --
> You received this message because you are subscribed to the Google Groups=
 "Bitcoin Development Mailing List" group.
> To unsubscribe from this group and stop receiving emails from it, send an=
 email to bitcoindev+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgi=
d/bitcoindev/be78e733-6e9f-4f4e-8dc2-67b79ddbf677n%40googlegroups.com.

--=20
You received this message because you are subscribed to the Google Groups "=
Bitcoin Development Mailing List" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to bitcoindev+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/=
bitcoindev/jJLDrYTXvTgoslhl1n7Fk9-pL1mMC-0k6gtoniQINmioJpzgtqrJ_WqyFZkLltsC=
UusnQ4jZ6HbvRC-mGuaUlDi3kcqcFHALd10-JQl-FMY%3D%40protonmail.com.

--b1_1CgWDOX9rJbJHO7L15t9f9KK006luTO7qO3U6fzgbI
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<blockquote style=3D"border-color: rgb(200, 200, 200); border-left: 3px sol=
id rgb(200, 200, 200); padding-left: 10px; color: rgb(102, 102, 102);"><div=
 style=3D"font-family: Arial, sans-serif; font-size: 14px; color: rgb(0, 0,=
 0); background-color: rgb(255, 255, 255);"><span style=3D"font-family: Ari=
al, sans-serif;">It is not clear to me how determining the coinbase size ca=
n be done at
an earlier stage of validation than detection of the non-null coinbase.</sp=
an><br></div></blockquote><div style=3D"font-family: Arial, sans-serif; fon=
t-size: 14px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"><=
br></div><div style=3D"font-family: Arial, sans-serif; font-size: 14px; col=
or: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"><span style=3D"fon=
t-family: Arial, sans-serif;">My point wasn't about checking the coinbase s=
ize, it was about being able to cache the hash of a (non-malleated) invalid=
 block as permanently invalid to avoid re-downloading and re-validating it.=
</span><br></div><br><blockquote style=3D"border-color: rgb(200, 200, 200);=
 border-left: 3px solid rgb(200, 200, 200); padding-left: 10px; color: rgb(=
102, 102, 102);"><div style=3D"font-family: Arial, sans-serif; font-size: 1=
4px; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"><span styl=
e=3D"font-family: Arial, sans-serif;">It seems to me that introducing an ar=
bitrary tx size validity may create more potential implementation bugs than=
 it resolves.</span></div></blockquote><div style=3D""><br></div><div style=
=3D""><span style=3D"font-family: Arial, sans-serif; font-size: 14px; font-=
weight: 400; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">Th=
e potential for implementation bugs is a fair point to raise, but in this c=
ase i don't think it's a big con</span><span style=3D"font-family: Arial, s=
ans-serif; font-size: 14px; font-weight: 400; color: rgb(0, 0, 0); backgrou=
nd-color: rgb(255, 255, 255);">cern. Verifying no transaction in a block is=
 </span><span style=3D"font-family: Arial, sans-serif; font-size: 14px; fon=
t-weight: 400; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">=
64 bytes is as simple a check as you can get.</span></div><div style=3D""><=
span style=3D"font-family: Arial, sans-serif; font-size: 14px; font-weight:=
 400; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"><br></spa=
n></div><blockquote style=3D"border-color: rgb(200, 200, 200); border-left:=
 3px solid rgb(200, 200, 200); padding-left: 10px; color: rgb(102, 102, 102=
);"><div style=3D""><span style=3D"font-family: Arial, sans-serif; font-siz=
e: 14px; font-weight: 400; color: rgb(0, 0, 0); background-color: rgb(255, =
255, 255);"><span style=3D"font-family: Arial, sans-serif;">And certainly a=
nyone implementing such a verifier must know many intricacies of the protoc=
ol.</span><span><br></span></span></div></blockquote><div style=3D""><span =
style=3D"font-family: Arial, sans-serif; font-size: 14px; font-weight: 400;=
 color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"><br></span></d=
iv><div style=3D""><span style=3D"font-family: Arial, sans-serif; font-size=
: 14px; font-weight: 400; color: rgb(0, 0, 0); background-color: rgb(255, 2=
55, 255);">They need to know some, but i don't think it's reasonable to exp=
ect them to realize the merkle tree construction is such that an inner node=
 may be confused with a 64 bytes transaction.</span></div><div style=3D""><=
span style=3D"font-family: Arial, sans-serif; font-size: 14px; font-weight:=
 400; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);"><br></spa=
n></div><blockquote style=3D"border-color: rgb(200, 200, 200); border-left:=
 3px solid rgb(200, 200, 200); padding-left: 10px; color: rgb(102, 102, 102=
);"><div style=3D""><span style=3D"font-family: Arial, sans-serif; font-siz=
e: 14px; font-weight: 400; color: rgb(0, 0, 0); background-color: rgb(255, =
255, 255);">I do not see this. I see a very ugly perpetual seam which will =
likely result in unexpected complexities over time.</span><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 14px; font-weight: 400; color: rgb=
(0, 0, 0); background-color: rgb(255, 255, 255);"><br></span></div></blockq=
uote><div style=3D""><br></div><div style=3D""><span style=3D""><span style=
=3D"font-family: Arial, sans-serif; font-size: 14px; font-weight: 400; colo=
r: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">What makes you thin=
k making 64 bytes transactions invalid could result in unexpected complexit=
ies? And why do you think it's likely?</span></span></div><div style=3D""><=
span style=3D""><span style=3D"font-family: Arial, sans-serif; font-size: 1=
4px; font-weight: 400; color: rgb(0, 0, 0); background-color: rgb(255, 255,=
 255);"><br></span></span></div><blockquote style=3D"border-color: rgb(200,=
 200, 200); border-left: 3px solid rgb(200, 200, 200); padding-left: 10px; =
color: rgb(102, 102, 102);"><div style=3D""><span style=3D""><span style=3D=
"font-family: Arial, sans-serif; font-size: 14px; font-weight: 400; color: =
rgb(0, 0, 0); background-color: rgb(255, 255, 255);">This does not produce =
unmalleable block hashes. Duplicate tx hash
malleation remains in either case, to the same effect. Without a
resolution to both issues this is an empty promise.</span><span style=3D"fo=
nt-family: Arial, sans-serif; font-size: 14px; font-weight: 400; color: rgb=
(0, 0, 0); background-color: rgb(255, 255, 255);"><br></span></span></div><=
/blockquote><div style=3D""><br></div><div style=3D""><span style=3D"font-f=
amily: Arial, sans-serif;">Duplicate txids have been invalid since 2012 (</=
span><span style=3D"font-family: Arial, sans-serif;">CVE-2012-2459</span><s=
pan style=3D"font-family: Arial, sans-serif;">).</span><span style=3D""><sp=
an style=3D"font-family: Arial, sans-serif;"> If 64 bytes transactions are =
also made invalid, this would make it impossible for two valid blocks to ha=
ve the same hash.</span></span></div><div style=3D""><span style=3D""><span=
 style=3D"font-family: Arial, sans-serif;"><br></span></span></div><div sty=
le=3D""><span style=3D""><span style=3D"font-family: Arial, sans-serif;">Be=
st,<br>Antoine<br></span><span style=3D""></span></span></div><div class=3D=
"protonmail_quote">
        On Monday, June 24th, 2024 at 2:35 AM, Eric Voskuil &lt;eric@voskui=
l.org&gt; wrote:<br>
        <blockquote type=3D"cite" class=3D"protonmail_quote">
            Thanks for the responses Antoine.<br><br>&gt;  As discussed her=
e it would let node implementations cache block failures at an earlier stag=
e of validation. Not a large gain, but still nice to have.<br><br>It is not=
 clear to me how determining the coinbase size can be done at an earlier st=
age of validation than detection of the non-null coinbase. The former requi=
res parsing the coinbase to determine its size, the latter requires parsing=
 it to know if the point is null. Both of these can be performed as early a=
s immediately following the socket read.<br><br>size check<br><br>(1) requi=
res new consensus rule: 64 byte transactions (or coinbases?) are invalid.<b=
r>(2) creates a consensus "seam"  (complexity) in txs, where &lt; 64 bytes =
and &gt; 64 bytes are potentially valid.<br>(3) can be limited to reading/s=
kipping header (80 bytes) plus parsing 0 - 65 coinbase bytes.<br><br>point =
check<br><br>(1) requires no change.<br>(2) creates no consensus seam.<br>(=
3) can be limited to reading/skipping header (80 bytes) plus parsing 6 - 43=
 coinbase bytes.<br><br>Not only is this not a large (performance) gain, it=
's not one at all.<br><br>&gt; It would also avoid a large footgun for anyo=
ne implementing a software verifying an SPV proof verifier and not knowing =
the intricacies of the protocol...<br><br>It seems to me that introducing a=
n arbitrary tx size validity may create more potential implementation bugs =
than it resolves. And certainly anyone implementing such a verifier must kn=
ow many intricacies of the protocol. This does not remove one, it introduce=
s another - as there is not only a bifurcation around tx size but one aroun=
d the question of whether this rule is active.<br> <br>&gt; Finally, it wou=
ld get rid of a large footgun in general. <br><br>I do not see this. I see =
a very ugly perpetual seam which will likely result in unexpected complexit=
ies over time.<br><br>&gt; Certainly, unique block hashes would be a useful=
 property for Bitcoin to have. It's not far-fetched to expect current or fu=
ture Bitcoin-related software to rely on this.<br><br>This does not produce=
 unmalleable block hashes. Duplicate tx hash malleation remains in either c=
ase, to the same effect. Without a resolution to both issues this is an emp=
ty promise.<br><br>The only possible benefit that I can see here is the pos=
sible very small bandwidth savings pertaining to SPV proofs. I would have a=
 very hard time justifying adding any consensus rule to achieve only that r=
esult.<br><br>Best,<br>Eric<br><span style=3D"font-family: Arial, sans-seri=
f;"><br></span>

<p></p>

-- <br>
You received this message because you are subscribed to the Google Groups "=
Bitcoin Development Mailing List" group.<br>
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a target=3D"_blank" rel=3D"noreferrer nofollow noopener" href=3D"m=
ailto:bitcoindev+unsubscribe@googlegroups.com">bitcoindev+unsubscribe@googl=
egroups.com</a>.<br>
To view this discussion on the web visit <a target=3D"_blank" rel=3D"norefe=
rrer nofollow noopener" href=3D"https://groups.google.com/d/msgid/bitcoinde=
v/be78e733-6e9f-4f4e-8dc2-67b79ddbf677n%40googlegroups.com">https://groups.=
google.com/d/msgid/bitcoindev/be78e733-6e9f-4f4e-8dc2-67b79ddbf677n%40googl=
egroups.com</a>.<br>

        </blockquote><br>
    </div>

<p></p>

-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;Bitcoin Development Mailing List&quot; group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:bitcoindev+unsubscribe@googlegroups.com">bitcoind=
ev+unsubscribe@googlegroups.com</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/d/msgid/bitcoindev/jJLDrYTXvTgoslhl1n7Fk9-pL1mMC-0k6gtoniQINmioJpzgtqrJ_=
WqyFZkLltsCUusnQ4jZ6HbvRC-mGuaUlDi3kcqcFHALd10-JQl-FMY%3D%40protonmail.com?=
utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.com/d/msgid/b=
itcoindev/jJLDrYTXvTgoslhl1n7Fk9-pL1mMC-0k6gtoniQINmioJpzgtqrJ_WqyFZkLltsCU=
usnQ4jZ6HbvRC-mGuaUlDi3kcqcFHALd10-JQl-FMY%3D%40protonmail.com</a>.<br />

--b1_1CgWDOX9rJbJHO7L15t9f9KK006luTO7qO3U6fzgbI--