MIME-Version: 1.0
In-Reply-To: <20180607222028.zbva4vrv64dzrmxy@petertodd.org>
References: <20180607171311.6qdjohfuuy3ufriv@petertodd.org>
	<CAHUJnBB7UL3mH6SixP_M4yooMVP3DgZa+5hiQOmF=AiqfdpfOg@mail.gmail.com>
	<20180607222028.zbva4vrv64dzrmxy@petertodd.org>
From: Bram Cohen <bram@chia.net>
Date: Fri, 8 Jun 2018 20:29:30 -0700
Message-ID: <CAHUJnBCj8wnjP1=jobfpg7jkfjkX9iSBLeeAOyQCpobh6-AhUA@mail.gmail.com>
To: Peter Todd <pete@petertodd.org>
Content-Type: multipart/alternative; boundary="000000000000e85efc056e2d1f66"
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Trusted merkle tree depth for safe tx inclusion
 proofs without a soft fork
Precedence: list

--000000000000e85efc056e2d1f66
Content-Type: text/plain; charset="UTF-8"

So are you saying that if fully validating nodes wish to prune they can
maintain the ability to validate old transactions by cacheing the number of
transactions in each previous block?

On Thu, Jun 7, 2018 at 3:20 PM, Peter Todd <pete@petertodd.org> wrote:

> On Thu, Jun 07, 2018 at 02:15:35PM -0700, Bram Cohen wrote:
> > Are you proposing a soft fork to include the number of transactions in a
> > block in the block headers to compensate for the broken Merkle format?
> That
> > sounds like a good idea.
> >
> > On Thu, Jun 7, 2018 at 10:13 AM, Peter Todd via bitcoin-dev <
> > bitcoin-dev@lists.linuxfoundation.org> wrote:
> >
> > > It's well known that the Bitcoin merkle tree algorithm fails to
> distinguish
> > > between inner nodes and 64 byte transactions, as both txs and inner
> nodes
> > > are
> > > hashed the same way. This potentially poses a problem for tx inclusion
> > > proofs,
> > > as a miner could (with ~60 bits of brute forcing) create a transaction
> that
> > > committed to a transaction that was not in fact in the blockchain.
> > >
> > > Since odd-numbered inner/leaf nodes are concatenated with themselves
> and
> > > hashed
> > > twice, the depth of all leaves (txs) in the tree is fixed.
> > >
> > > It occured to me that if the depth of the merkle tree is known, this
> > > vulnerability can be trivially avoided by simply comparing the length
> of
> > > the
> > > merkle path to that known depth. For pruned nodes, if the depth is
> saved
> > > prior
> > > to pruning the block contents itself, this would allow for completely
> safe
> > > verification of tx inclusion proofs, without a soft-fork; storing this
>                                          ^^^^^^^^^^^^^^^^^^^
>
> Re-read my post: I specifically said you do not need a soft-fork to
> implement
> this. In fact, I think you can argue that this is an accidental feature,
> not a
> bug, as it further encourages the use of safe full verifiaction rather than
> unsafe lite clients.
>
> --
> https://petertodd.org 'peter'[:-1]@petertodd.org
>

--000000000000e85efc056e2d1f66
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">So are you saying that if fully validating nodes wish to p=
rune they can maintain the ability to validate old transactions by cacheing=
 the number of transactions in each previous block?</div><div class=3D"gmai=
l_extra"><br><div class=3D"gmail_quote">On Thu, Jun 7, 2018 at 3:20 PM, Pet=
er Todd <span dir=3D"ltr">&lt;<a href=3D"mailto:pete@petertodd.org" target=
=3D"_blank">pete@petertodd.org</a>&gt;</span> wrote:<br><blockquote class=
=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padd=
ing-left:1ex"><span class=3D"">On Thu, Jun 07, 2018 at 02:15:35PM -0700, Br=
am Cohen wrote:<br>
&gt; Are you proposing a soft fork to include the number of transactions in=
 a<br>
&gt; block in the block headers to compensate for the broken Merkle format?=
 That<br>
&gt; sounds like a good idea.<br>
&gt; <br>
&gt; On Thu, Jun 7, 2018 at 10:13 AM, Peter Todd via bitcoin-dev &lt;<br>
&gt; <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@l=
ists.<wbr>linuxfoundation.org</a>&gt; wrote:<br>
&gt; <br>
&gt; &gt; It&#39;s well known that the Bitcoin merkle tree algorithm fails =
to distinguish<br>
&gt; &gt; between inner nodes and 64 byte transactions, as both txs and inn=
er nodes<br>
&gt; &gt; are<br>
&gt; &gt; hashed the same way. This potentially poses a problem for tx incl=
usion<br>
&gt; &gt; proofs,<br>
&gt; &gt; as a miner could (with ~60 bits of brute forcing) create a transa=
ction that<br>
&gt; &gt; committed to a transaction that was not in fact in the blockchain=
.<br>
&gt; &gt;<br>
&gt; &gt; Since odd-numbered inner/leaf nodes are concatenated with themsel=
ves and<br>
&gt; &gt; hashed<br>
&gt; &gt; twice, the depth of all leaves (txs) in the tree is fixed.<br>
&gt; &gt;<br>
&gt; &gt; It occured to me that if the depth of the merkle tree is known, t=
his<br>
&gt; &gt; vulnerability can be trivially avoided by simply comparing the le=
ngth of<br>
&gt; &gt; the<br>
&gt; &gt; merkle path to that known depth. For pruned nodes, if the depth i=
s saved<br>
&gt; &gt; prior<br>
&gt; &gt; to pruning the block contents itself, this would allow for comple=
tely safe<br>
&gt; &gt; verification of tx inclusion proofs, without a soft-fork; storing=
 this<br>
</span>=C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0^^^^^^^^^^^^^^^^^^^<br>
<br>
Re-read my post: I specifically said you do not need a soft-fork to impleme=
nt<br>
this. In fact, I think you can argue that this is an accidental feature, no=
t a<br>
bug, as it further encourages the use of safe full verifiaction rather than=
<br>
unsafe lite clients.<br>
<div class=3D"HOEnZb"><div class=3D"h5"><br>
-- <br>
<a href=3D"https://petertodd.org" rel=3D"noreferrer" target=3D"_blank">http=
s://petertodd.org</a> &#39;peter&#39;[:-1]@<a href=3D"http://petertodd.org"=
 rel=3D"noreferrer" target=3D"_blank">petertodd.org</a><br>
</div></div></blockquote></div><br></div>

--000000000000e85efc056e2d1f66--