path: root/b4/ad0da079ff6033dca72695325829a20d5ee9d0

Return-Path: <dscotese@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id C39B0136F
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Wed,  3 Apr 2019 21:39:47 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-io1-f41.google.com (mail-io1-f41.google.com
	[209.85.166.41])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B23F5E3
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Wed,  3 Apr 2019 21:39:45 +0000 (UTC)
Received: by mail-io1-f41.google.com with SMTP id e13so200222ioq.6
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Wed, 03 Apr 2019 14:39:45 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:references:in-reply-to:from:date
	:message-id:subject:to:cc;
	bh=9WvO8PmUjx1fk7Ux5SUzqnDmLAD574gTg3fk89gr2HQ=;
	b=NnebnaJJo3aSuegOfTGP0AvV6GzX0m65CrsbS06XHAZ2+RBgHyv38fVss3Nm4yOANL
	/4jFVtk6kF2LWyCQsBvrR5uf+UbuLG77QoJsy3e04vaAB52LAAx/QfjHUgZ2OEmW3wCY
	j+4aXO/89kNgdoqOoe2DjmTva9C/UY/WcgJ3JGzGS8Q1Tzet7DpHVmsdgyyl4ApLT11e
	GQ/8oNuUPfwytjpbthLA2z2HSdbX+wiz3HJ6kdIug80snOGFqwf7QqAU5VN5DZVLfLVs
	gyyz+ETXjKkszSttujSvH9eqWtS52HcOqQDz9sr2VSm2tgNQ+xh1VF59cOZQ4fD/HrBc
	O8+g==
X-Gm-Message-State: APjAAAU3eA1QTFJOgTvzfPM8kmfrEGbKJnxojO5EO8rtCH4qaMEWTR5u
	TGDBHGQ8i8uOmbtQH6leuBNYwfk0WS7rPNJGqfc=
X-Google-Smtp-Source: APXvYqwMhHkV+x0MB6B1E4DgWTjkDPVXwuOYmhH5J+XoYqM7vRNueX32uvsmCE/cgNr1oJesjyktBFIx4q/7M7i1d1U=
X-Received: by 2002:a6b:6f0a:: with SMTP id k10mr1782883ioc.185.1554327584849; 
	Wed, 03 Apr 2019 14:39:44 -0700 (PDT)
MIME-Version: 1.0
References: <CAPfvXf+JS6ZhXUieWVxiaNa4uhhWwafCk3odMKy5F_yi=XwngA@mail.gmail.com>
	<816FFA03-B4D9-4ECE-AF15-85ACBFA4BA8F@jonasschnelli.ch>
	<CACiOHGxxqm5Qn8J9u5oDE5Ek5smqB4E4iz4PJOZHpJO5kwP=-A@mail.gmail.com>
In-Reply-To: <CACiOHGxxqm5Qn8J9u5oDE5Ek5smqB4E4iz4PJOZHpJO5kwP=-A@mail.gmail.com>
From: Dave Scotese <dscotese@litmocracy.com>
Date: Wed, 3 Apr 2019 14:39:32 -0700
Message-ID: <CAGLBAhf1NZfT9TunhHAb==mFTfaAacjekQh6Pqn4yBS+90Zw6A@mail.gmail.com>
To: Ethan Scruples <ethan.scruples@gmail.com>, 
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="0000000000008f45780585a71791"
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,FREEMAIL_FROM,
	HTML_MESSAGE,RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
X-Mailman-Approved-At: Wed, 03 Apr 2019 22:47:39 +0000
Subject: Re: [bitcoin-dev] assumeutxo and UTXO snapshots
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Wed, 03 Apr 2019 21:39:47 -0000

--0000000000008f45780585a71791
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Every block's hash is smaller than the difficulty at that time.  Block
569927's hash was VERY small (started with 21 zeros).  The ratio of block
hash to difficulty requirement (0xffffffff - difficulty, I think) could be
used to identify blocks as "special," thus providing the opportunity to
popularize unimportant but memorable-and-therefore-useful details.  How can
they be useful if they are unimportant?  They are useful for sanity
checking.  For example, if the drunken bishop walk (or some other popular
randomart) produced by block 569927's hash looked like a face, that would
be memorable: "The block with the smallest hash in 2019 (maybe ever?) looks
like a face after the drunken bishop walk."

If a few of these showed up each year, then Bob and/or Alice would have a
good chance of seeing that something was wrong if and when they checked.
It would not be surprising, given Ethan's assumption that the invalid block
Bob found contributed to Alice's UTXOs, that at some point, the history one
of them has would be missing the memorable things beginning at some block
height because, clearly, one of them has been forked.

Luke's comment that it could "lead to users trusting third parties (like
developers) way too much" is pertinent too, but I think an honest abatement
of that concern is impossible without teaching everyone C++.  "Developers"
as an open group (anyone can fork the github repo, find a problem, and make
an issue) deserve the trust we put in them, and that's because they're
accountable (any such error found in the repo will have been put there by
someone).  The same thing goes for making it possible to download (*not
just the compiled software*, but) the entire UTXO Set if a commitment of it
is hardcoded into the software, as James suggests.  We all trust
"developers" like that, and it's okay.  No one holds the "ring of power."

On Wed, Apr 3, 2019 at 8:39 AM Ethan Scruples via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Jonas,
>
> If we can get mandatory UTXO commitments soft forked into Bitcoin, we get
> the advantage of a non-growing IBD, which I think everyone would agree is=
 a
> benefit that, uh, grows over time. The thing I do not see people noticing
> is that we actually pay little to no security price for this benefit.
>
> To see this, consider Alice, who starts from a UTXO snapshot made at
> current height - 50,000 and Bob who validates from genesis.
>
> After her partial validation, Alice is satisfied that she is in possessio=
n
> of the UTXO set-- she is in consensus with the rest of the network peers.
>
> However, Bob realizes that there is actually an invalid block at current
> height - 50,001.
>
> Three things to notice:
>
> 1. This scenario essentially cannot happen. There is no way that the
> miners are going to stack 50,000 blocks on top of an invalid block withou=
t
> the economic majority abandoning the invalid chain.
>
> 2. If this scenario DOES happen, Bob has learned about it too late for it
> to matter to Bob. The blockchain Bob wants to be on is the one that
> everyone has been using for the last year, whether or not it is besmirche=
d
> by an invalid block.
>
> 3. If this scenario DOES happen, and Bob DOES want to reject the last
> 50,000 mined blocks as invalid, he may discover to his dismay that in the=
 1
> year since the invalid block, mischievous entities have enough time to mi=
ne
> equally weighted alternative histories from the Genesis block forward to
> the invalid block, meaning that Bob has no way to use POW to come to
> consensus with other Bobs out there.
>
> On Wed, Apr 3, 2019 at 3:33 AM Jonas Schnelli via bitcoin-dev <
> bitcoin-dev@lists.linuxfoundation.org> wrote:
>
>> Thanks James for the post.
>>
>> I proposed a similar idea [1] back in 2016 with the difference of signin=
g
>> the UTXO-set hash in a gitian-ish way.
>>
>> While the idea of UTXO-set-syncs are attractive, there are probably stil=
l
>> significant downsides in usability (compared to models with less securit=
y),
>> mainly:
>> * Assume the UTXO set is 6 weeks old (which seems a reasonable age for
>> providing enough security) a peer using that snapshot would still requir=
e
>> to download and verify ~6048 blocks (~7.9GB at 1.3MB blocks,=E2=80=A6 pr=
obably
>> CPU-days on a phone)
>> * Do we semi-trust the peer that servers the UTXO set (compared to a
>> block or tx which we can validate)? What channel to we use to serve the
>> snapshot?
>>
>> If the goal is to run a full node on a consumer device that is also been
>> used for other CPU intense operations (like a phone, etc.), I=E2=80=99m =
not sure if
>> this proposal will lead to a satisfactory user experience.
>>
>> The longer I think around this problem, the more I lean towards acceptin=
g
>> the fact that one need to use dedicated hardware in his own environment =
to
>> perform a painless full validation.
>>
>> /jonas
>>
>> [1]
>> https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-February/01=
2478.html
>>
>> > Am 02.04.2019 um 22:43 schrieb James O'Beirne via bitcoin-dev <
>> bitcoin-dev@lists.linuxfoundation.org>:
>> >
>> > Hi,
>> >
>> > I'd like to discuss assumeutxo, which is an appealing and simple
>> > optimization in the spirit of assumevalid[0].
>> >
>> > # Motivation
>> >
>> > To start a fully validating bitcoin client from scratch, that client
>> currently
>> > needs to perform an initial block download. To the surprise of no one,
>> IBD
>> > takes a linear amount time based on the length of the chain's history.
>> For
>> > clients running on modest hardware under limited bandwidth constraints=
,
>> > say a mobile device, completing IBD takes a considerable amount of tim=
e
>> > and thus poses serious usability challenges.
>> >
>> > As a result, having fully validating clients run on such hardware is
>> rare and
>> > basically unrealistic. Clients with even moderate resource constraints
>> > are encouraged to rely on the SPV trust model. Though we have promisin=
g
>> > improvements to existing SPV modes pending deployment[1], it's worth
>> > thinking about a mechanism that would allow such clients to use trust
>> > models closer to full validation.
>> >
>> > The subject of this mail is a proposal for a complementary alternative
>> to SPV
>> > modes, and which is in the spirit of an existing default,
>> `assumevalid`. It may
>> > help modest clients transact under a security model that closely
>> resembles
>> > full validation within minutes instead of hours or days.
>> >
>> > # assumeutxo
>> >
>> > The basic idea is to allow nodes to initialize using a serialized
>> version of the
>> > UTXO set rendered by another node at some predetermined height. The
>> > initializing node syncs the headers chain from the network, then
>> obtains and
>> > loads one of these UTXO snapshots (i.e. a serialized version of the
>> UTXO set
>> > bundled with the block header indicating its "base" and some other
>> metadata).
>> >
>> > Based upon the snapshot, the node is able to quickly reconstruct its
>> chainstate,
>> > and compares a hash of the resulting UTXO set to a preordained hash
>> hard-coded
>> > in the software a la assumevalid. This all takes ~23 minutes, not
>> accounting for
>> > download of the 3.2GB snapshot[2].
>> >
>> > The node then syncs to the network tip and afterwards begins a
>> simultaneous
>> > background validation (i.e., a conventional IBD) up to the base height
>> of the
>> > snapshot in order to achieve full validation. Crucially, even while th=
e
>> > background validation is happening the node can validate incoming
>> blocks and
>> > transact with the benefit of the full (assumed-valid) UTXO set.
>> >
>> > Snapshots could be obtained from multiple separate peers in the same
>> manner as
>> > block download, but I haven't put much thought into this. In concept i=
t
>> doesn't
>> > matter too much where the snapshots come from since their validity is
>> > determined via content hash.
>> >
>> > # Security
>> >
>> > Obviously there are some security implications due consideration. Whil=
e
>> this
>> > proposal is in the spirit of assumevalid, practical attacks may become
>> easier.
>> > Under assumevalid, a user can be tricked into transacting under a fals=
e
>> history
>> > if an attacker convinces them to start bitcoind with a malicious
>> `-assumevalid`
>> > parameter, sybils their node, and then feeds them a bogus chain
>> encompassing
>> > all of the hard-coded checkpoints[3].
>> >
>> > The same attack is made easier in assumeutxo because, unlike in
>> assumevalid,
>> > the attacker need not construct a valid PoW chain to get the victim's
>> node into
>> > a false state; they simply need to get the user to accept a bad
>> `-assumeutxo`
>> > parameter and then supply them an easily made UTXO snapshot containing=
,
>> say, a
>> > false coin assignment.
>> >
>> > For this reason, I recommend that if we were to implement assumeutxo,
>> we not
>> > allow its specification via commandline argument[4].
>> >
>> > Beyond this risk, I can't think of material differences in security
>> relative to
>> > assumevalid, though I appeal to the list for help with this.
>> >
>> > # More fully validating clients
>> >
>> > A particularly exciting use-case for assumeutxo is the possibility of
>> mobile
>> > devices functioning as fully validating nodes with access to the
>> complete UTXO
>> > set (as an alternative to SPV models). The total resource burden neede=
d
>> to start a node
>> > from scratch based on a snapshot is, at time of writing, a ~(3.2GB
>> > + blocks_to_tip * 4MB) download and a few minutes of processing time,
>> which sounds
>> > manageable for many mobile devices currently in use.
>> >
>> > A mobile user could initialize an assumed-valid bitcoin node within an
>> hour,
>> > transact immediately, and complete a pruned full validation of their
>> > assumed-valid chain over the next few days, perhaps only doing the
>> background
>> > IBD when their device has access to suitable high-bandwidth connection=
s.
>> >
>> > If we end up implementing an accumulator-based UTXO scaling
>> design[5][6] down
>> > the road, it's easy to imagine an analogous process that would allow
>> very fast
>> > startup using an accumulator of a few kilobytes in lieu of a multi-GB
>> snapshot.
>> >
>> > ---
>> >
>> > I've created a related issue at our Github repository here:
>> >   https://github.com/bitcoin/bitcoin/issues/15605
>> >
>> > and have submitted a draft implementation of snapshot usage via RPC
>> here:
>> >   https://github.com/bitcoin/bitcoin/pull/15606
>> >
>> > I'd like to discuss here whether this is a good fit for Bitcoin
>> conceptually. Concrete
>> > plans for deployment steps should be discussed in the Github issue, an=
d
>> after all
>> > that my implementation may be reviewed as a sketch of the specific
>> software
>> > changes necessary.
>> >
>> > Regards,
>> > James
>> >
>> >
>> > [0]:
>> https://bitcoincore.org/en/2017/03/08/release-0.14.0/#assumed-valid-bloc=
ks
>> > [1]: https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki
>> > [2]: as tested at height 569895, on a 12 core Intel Xeon Silver 4116
>> CPU @ 2.10GHz
>> > [3]:
>> https://github.com/bitcoin/bitcoin/blob/84d0fdc/src/chainparams.cpp#L145=
-L161
>> > [4]: Marco Falke is due credit for this point
>> > [5]: utreexo: https://www.youtube.com/watch?v=3DedRun-6ubCc
>> > [6]: Boneh, Bunz, Fisch on accumulators:
>> https://eprint.iacr.org/2018/1188
>> >
>> > _______________________________________________
>> > bitcoin-dev mailing list
>> > bitcoin-dev@lists.linuxfoundation.org
>> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
>> _______________________________________________
>> bitcoin-dev mailing list
>> bitcoin-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>


--=20
I like to provide some work at no charge to prove my value. Do you need a
techie?
I own Litmocracy <http://www.litmocracy.com> and Meme Racing
<http://www.memeracing.net> (in alpha).
I'm the webmaster for The Voluntaryist <http://www.voluntaryist.com> which
now accepts Bitcoin.
I also code for The Dollar Vigilante <http://dollarvigilante.com/>.
"He ought to find it more profitable to play by the rules" - Satoshi
Nakamoto

--0000000000008f45780585a71791
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div></div><div>Every block&#39;s hash is smaller than the=
 difficulty at that time.=C2=A0 Block 569927&#39;s hash was VERY small (sta=
rted with 21 zeros).=C2=A0 The ratio of block hash to difficulty requiremen=
t (0xffffffff - difficulty, I think) could be used to identify blocks as &q=
uot;special,&quot; thus providing the opportunity to popularize unimportant=
 but memorable-and-therefore-useful details.=C2=A0 How can they be useful i=
f they are unimportant?=C2=A0 They are useful for sanity checking.=C2=A0 Fo=
r example, if the drunken bishop walk (or some other popular randomart) pro=
duced by=20
block 569927&#39;s hash

looked like a face, that would be memorable: &quot;The block with the small=
est hash in 2019 (maybe ever?) looks like a face after the drunken bishop w=
alk.&quot;<br></div><div><br></div><div>If a few of these showed up each ye=
ar, then Bob and/or Alice would have a good chance of seeing that something=
 was wrong if and when they checked.=C2=A0 It would not be surprising, give=
n Ethan&#39;s assumption that the invalid block Bob found contributed to Al=
ice&#39;s UTXOs, that at some point, the history one of them has would be m=
issing the memorable things beginning at some block height because, clearly=
, one of them has been forked.<br></div><div><br></div><div>Luke&#39;s comm=
ent that it could &quot;lead to users trusting third parties (like develope=
rs) way too much&quot; is pertinent too, but I think an honest abatement of=
 that concern is impossible without teaching everyone C++.=C2=A0 &quot;Deve=
lopers&quot; as an open group (anyone can fork the github repo, find a prob=
lem, and make an issue) deserve the trust we put in them, and that&#39;s be=
cause they&#39;re accountable (any such error found in the repo will have b=
een put there by someone).=C2=A0 The same thing goes for making it possible=
 to download (<b>not just the compiled software</b>, but) the entire UTXO S=
et if a commitment of it is hardcoded into the software, as James suggests.=
=C2=A0 We all trust &quot;developers&quot; like that, and it&#39;s okay.=C2=
=A0 No one holds the &quot;ring of power.&quot;<br></div></div><br><div cla=
ss=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Wed, Apr 3, 201=
9 at 8:39 AM Ethan Scruples via bitcoin-dev &lt;<a href=3D"mailto:bitcoin-d=
ev@lists.linuxfoundation.org">bitcoin-dev@lists.linuxfoundation.org</a>&gt;=
 wrote:<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px =
0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=
=3D"ltr">Jonas,<div><br></div><div>If we can get mandatory UTXO commitments=
 soft forked into Bitcoin, we get the advantage of a non-growing IBD, which=
 I think everyone would agree is a benefit that, uh, grows over time. The t=
hing I do not see people noticing is that we actually pay little to no secu=
rity price for this benefit.</div><div><br></div><div>To see this, consider=
 Alice, who starts from a UTXO snapshot made at current height - 50,000 and=
 Bob who validates from genesis.</div><div><br></div><div>After her partial=
 validation, Alice is satisfied that she is in possession of the UTXO set--=
 she is in consensus with the rest of the network peers.</div><div><br></di=
v><div>However, Bob realizes that there is actually an invalid block at cur=
rent height - 50,001.</div><div><br></div><div>Three things to notice:</div=
><div><br></div><div>1. This scenario essentially cannot happen. There is n=
o way that the miners are going to stack 50,000 blocks on top of an invalid=
 block without the economic majority abandoning the invalid chain.</div><di=
v><br></div><div>2. If this scenario DOES happen, Bob has learned about it =
too late for it to matter to Bob. The blockchain Bob wants to be on is the =
one that everyone has been using for the last year, whether or not it is be=
smirched by an invalid block.</div><div><br></div><div>3. If this scenario =
DOES happen, and Bob DOES want to reject the last 50,000 mined blocks as in=
valid, he may discover to his dismay that in the 1 year since the invalid b=
lock, mischievous entities have enough time to mine equally weighted altern=
ative histories from the Genesis block forward to the invalid block, meanin=
g that Bob has no way to use POW to come to consensus with other Bobs out t=
here.<br></div></div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=
=3D"gmail_attr">On Wed, Apr 3, 2019 at 3:33 AM Jonas Schnelli via bitcoin-d=
ev &lt;<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_=
blank">bitcoin-dev@lists.linuxfoundation.org</a>&gt; wrote:<br></div><block=
quote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1=
px solid rgb(204,204,204);padding-left:1ex">Thanks James for the post.<br>
<br>
I proposed a similar idea [1] back in 2016 with the difference of signing t=
he UTXO-set hash in a gitian-ish way.<br>
<br>
While the idea of UTXO-set-syncs are attractive, there are probably still s=
ignificant downsides in usability (compared to models with less security), =
mainly:<br>
* Assume the UTXO set is 6 weeks old (which seems a reasonable age for prov=
iding enough security) a peer using that snapshot would still require to do=
wnload and verify ~6048 blocks (~7.9GB at 1.3MB blocks,=E2=80=A6 probably C=
PU-days on a phone)<br>
* Do we semi-trust the peer that servers the UTXO set (compared to a block =
or tx which we can validate)? What channel to we use to serve the snapshot?=
<br>
<br>
If the goal is to run a full node on a consumer device that is also been us=
ed for other CPU intense operations (like a phone, etc.), I=E2=80=99m not s=
ure if this proposal will lead to a satisfactory user experience.<br>
<br>
The longer I think around this problem, the more I lean towards accepting t=
he fact that one need to use dedicated hardware in his own environment to p=
erform a painless full validation.<br>
<br>
/jonas<br>
<br>
[1] <a href=3D"https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016=
-February/012478.html" rel=3D"noreferrer" target=3D"_blank">https://lists.l=
inuxfoundation.org/pipermail/bitcoin-dev/2016-February/012478.html</a><br>
<br>
&gt; Am 02.04.2019 um 22:43 schrieb James O&#39;Beirne via bitcoin-dev &lt;=
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">=
bitcoin-dev@lists.linuxfoundation.org</a>&gt;:<br>
&gt; <br>
&gt; Hi,<br>
&gt; <br>
&gt; I&#39;d like to discuss assumeutxo, which is an appealing and simple<b=
r>
&gt; optimization in the spirit of assumevalid[0].<br>
&gt; <br>
&gt; # Motivation<br>
&gt; <br>
&gt; To start a fully validating bitcoin client from scratch, that client c=
urrently<br>
&gt; needs to perform an initial block download. To the surprise of no one,=
 IBD<br>
&gt; takes a linear amount time based on the length of the chain&#39;s hist=
ory. For<br>
&gt; clients running on modest hardware under limited bandwidth constraints=
,<br>
&gt; say a mobile device, completing IBD takes a considerable amount of tim=
e<br>
&gt; and thus poses serious usability challenges.<br>
&gt; <br>
&gt; As a result, having fully validating clients run on such hardware is r=
are and<br>
&gt; basically unrealistic. Clients with even moderate resource constraints=
<br>
&gt; are encouraged to rely on the SPV trust model. Though we have promisin=
g<br>
&gt; improvements to existing SPV modes pending deployment[1], it&#39;s wor=
th<br>
&gt; thinking about a mechanism that would allow such clients to use trust<=
br>
&gt; models closer to full validation.<br>
&gt; <br>
&gt; The subject of this mail is a proposal for a complementary alternative=
 to SPV<br>
&gt; modes, and which is in the spirit of an existing default, `assumevalid=
`. It may<br>
&gt; help modest clients transact under a security model that closely resem=
bles<br>
&gt; full validation within minutes instead of hours or days.<br>
&gt; <br>
&gt; # assumeutxo<br>
&gt; <br>
&gt; The basic idea is to allow nodes to initialize using a serialized vers=
ion of the<br>
&gt; UTXO set rendered by another node at some predetermined height. The<br=
>
&gt; initializing node syncs the headers chain from the network, then obtai=
ns and<br>
&gt; loads one of these UTXO snapshots (i.e. a serialized version of the UT=
XO set<br>
&gt; bundled with the block header indicating its &quot;base&quot; and some=
 other metadata).<br>
&gt; <br>
&gt; Based upon the snapshot, the node is able to quickly reconstruct its c=
hainstate,<br>
&gt; and compares a hash of the resulting UTXO set to a preordained hash ha=
rd-coded<br>
&gt; in the software a la assumevalid. This all takes ~23 minutes, not acco=
unting for<br>
&gt; download of the 3.2GB snapshot[2].<br>
&gt; <br>
&gt; The node then syncs to the network tip and afterwards begins a simulta=
neous<br>
&gt; background validation (i.e., a conventional IBD) up to the base height=
 of the<br>
&gt; snapshot in order to achieve full validation. Crucially, even while th=
e<br>
&gt; background validation is happening the node can validate incoming bloc=
ks and<br>
&gt; transact with the benefit of the full (assumed-valid) UTXO set.<br>
&gt; <br>
&gt; Snapshots could be obtained from multiple separate peers in the same m=
anner as<br>
&gt; block download, but I haven&#39;t put much thought into this. In conce=
pt it doesn&#39;t<br>
&gt; matter too much where the snapshots come from since their validity is<=
br>
&gt; determined via content hash.<br>
&gt; <br>
&gt; # Security<br>
&gt; <br>
&gt; Obviously there are some security implications due consideration. Whil=
e this<br>
&gt; proposal is in the spirit of assumevalid, practical attacks may become=
 easier.<br>
&gt; Under assumevalid, a user can be tricked into transacting under a fals=
e history<br>
&gt; if an attacker convinces them to start bitcoind with a malicious `-ass=
umevalid`<br>
&gt; parameter, sybils their node, and then feeds them a bogus chain encomp=
assing<br>
&gt; all of the hard-coded checkpoints[3].<br>
&gt; <br>
&gt; The same attack is made easier in assumeutxo because, unlike in assume=
valid,<br>
&gt; the attacker need not construct a valid PoW chain to get the victim&#3=
9;s node into<br>
&gt; a false state; they simply need to get the user to accept a bad `-assu=
meutxo`<br>
&gt; parameter and then supply them an easily made UTXO snapshot containing=
, say, a<br>
&gt; false coin assignment.<br>
&gt; <br>
&gt; For this reason, I recommend that if we were to implement assumeutxo, =
we not<br>
&gt; allow its specification via commandline argument[4].<br>
&gt; <br>
&gt; Beyond this risk, I can&#39;t think of material differences in securit=
y relative to<br>
&gt; assumevalid, though I appeal to the list for help with this.<br>
&gt; <br>
&gt; # More fully validating clients<br>
&gt; <br>
&gt; A particularly exciting use-case for assumeutxo is the possibility of =
mobile<br>
&gt; devices functioning as fully validating nodes with access to the compl=
ete UTXO<br>
&gt; set (as an alternative to SPV models). The total resource burden neede=
d to start a node<br>
&gt; from scratch based on a snapshot is, at time of writing, a ~(3.2GB<br>
&gt; + blocks_to_tip * 4MB) download and a few minutes of processing time, =
which sounds<br>
&gt; manageable for many mobile devices currently in use.<br>
&gt; <br>
&gt; A mobile user could initialize an assumed-valid bitcoin node within an=
 hour,<br>
&gt; transact immediately, and complete a pruned full validation of their<b=
r>
&gt; assumed-valid chain over the next few days, perhaps only doing the bac=
kground<br>
&gt; IBD when their device has access to suitable high-bandwidth connection=
s.<br>
&gt; <br>
&gt; If we end up implementing an accumulator-based UTXO scaling design[5][=
6] down<br>
&gt; the road, it&#39;s easy to imagine an analogous process that would all=
ow very fast<br>
&gt; startup using an accumulator of a few kilobytes in lieu of a multi-GB =
snapshot.<br>
&gt; <br>
&gt; ---<br>
&gt; <br>
&gt; I&#39;ve created a related issue at our Github repository here:<br>
&gt;=C2=A0 =C2=A0<a href=3D"https://github.com/bitcoin/bitcoin/issues/15605=
" rel=3D"noreferrer" target=3D"_blank">https://github.com/bitcoin/bitcoin/i=
ssues/15605</a><br>
&gt; <br>
&gt; and have submitted a draft implementation of snapshot usage via RPC he=
re:<br>
&gt;=C2=A0 =C2=A0<a href=3D"https://github.com/bitcoin/bitcoin/pull/15606" =
rel=3D"noreferrer" target=3D"_blank">https://github.com/bitcoin/bitcoin/pul=
l/15606</a><br>
&gt; <br>
&gt; I&#39;d like to discuss here whether this is a good fit for Bitcoin co=
nceptually. Concrete<br>
&gt; plans for deployment steps should be discussed in the Github issue, an=
d after all<br>
&gt; that my implementation may be reviewed as a sketch of the specific sof=
tware<br>
&gt; changes necessary.<br>
&gt; <br>
&gt; Regards,<br>
&gt; James<br>
&gt; <br>
&gt; <br>
&gt; [0]: <a href=3D"https://bitcoincore.org/en/2017/03/08/release-0.14.0/#=
assumed-valid-blocks" rel=3D"noreferrer" target=3D"_blank">https://bitcoinc=
ore.org/en/2017/03/08/release-0.14.0/#assumed-valid-blocks</a><br>
&gt; [1]: <a href=3D"https://github.com/bitcoin/bips/blob/master/bip-0157.m=
ediawiki" rel=3D"noreferrer" target=3D"_blank">https://github.com/bitcoin/b=
ips/blob/master/bip-0157.mediawiki</a><br>
&gt; [2]: as tested at height 569895, on a 12 core Intel Xeon Silver 4116 C=
PU @ 2.10GHz<br>
&gt; [3]: <a href=3D"https://github.com/bitcoin/bitcoin/blob/84d0fdc/src/ch=
ainparams.cpp#L145-L161" rel=3D"noreferrer" target=3D"_blank">https://githu=
b.com/bitcoin/bitcoin/blob/84d0fdc/src/chainparams.cpp#L145-L161</a><br>
&gt; [4]: Marco Falke is due credit for this point<br>
&gt; [5]: utreexo: <a href=3D"https://www.youtube.com/watch?v=3DedRun-6ubCc=
" rel=3D"noreferrer" target=3D"_blank">https://www.youtube.com/watch?v=3Ded=
Run-6ubCc</a><br>
&gt; [6]: Boneh, Bunz, Fisch on accumulators: <a href=3D"https://eprint.iac=
r.org/2018/1188" rel=3D"noreferrer" target=3D"_blank">https://eprint.iacr.o=
rg/2018/1188</a><br>
&gt; <br>
&gt; _______________________________________________<br>
&gt; bitcoin-dev mailing list<br>
&gt; <a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_bl=
ank">bitcoin-dev@lists.linuxfoundation.org</a><br>
&gt; <a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-=
dev" rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org=
/mailman/listinfo/bitcoin-dev</a><br>
<br>
_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">=
bitcoin-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
</blockquote></div>
_______________________________________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank">=
bitcoin-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/mail=
man/listinfo/bitcoin-dev</a><br>
</blockquote></div><br clear=3D"all"><br>-- <br><div dir=3D"ltr" class=3D"g=
mail_signature"><div dir=3D"ltr">I like to provide some work at no charge t=
o prove my value. Do you need a techie?=C2=A0 <br>I own <a href=3D"http://w=
ww.litmocracy.com" target=3D"_blank">Litmocracy</a> and <a href=3D"http://w=
ww.memeracing.net" target=3D"_blank">Meme Racing</a> (in alpha). <br>I&#39;=
m the webmaster for <a href=3D"http://www.voluntaryist.com" target=3D"_blan=
k">The Voluntaryist</a> which now accepts Bitcoin.<br>I also code for <a hr=
ef=3D"http://dollarvigilante.com/" target=3D"_blank">The Dollar Vigilante</=
a>.<br>&quot;He ought to find it more profitable to play by the rules&quot;=
 - Satoshi Nakamoto</div></div>

--0000000000008f45780585a71791--