path: root/bc/1836ed46123eb5cd060c9c91febdc6cd52bc56

Return-Path: <tier.nolan@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id 1E455ACC
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Sat, 11 Jul 2015 13:17:56 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-qk0-f180.google.com (mail-qk0-f180.google.com
	[209.85.220.180])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id DFBBA17A
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Sat, 11 Jul 2015 13:17:54 +0000 (UTC)
Received: by qkbp125 with SMTP id p125so224498546qkb.2
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Sat, 11 Jul 2015 06:17:54 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
	h=mime-version:in-reply-to:references:date:message-id:subject:from:to
	:content-type; bh=0s0mmDq0Slx5McQ92L6FExafDDLv+2XCneWTTqk/Y1w=;
	b=AgFDmOSJ/fst/XJ6zAxcWfH+iywqeQPTCLLw7xb6KjiLAk4B7yWZVclBaljVE9YSP+
	Tqu3j7Le+8i61+1OeOCliQluMvAVfbuBPQsMzkBJcxqb3V5ODRJBuRZqKfQ2+wTtWGrX
	RxWtP3Mfiuad+L+gASPp+5b2IZdJzbPS276leYYIrspV8vFpLmS4JWw6xJarXD7cXj37
	1cwNIo8VaDULblax10aOLIshrVQ2ZakkzSOBjiUnA1b5Yj5tg4NG6zwaE/7l2BOTJ7DA
	sONwXKnpDjZwWXweZ+H5SnV4PqBPPVid73jHmCl/LJtAE333csFtxjPcrIeZvpgJb8hM
	hK5w==
MIME-Version: 1.0
X-Received: by 10.141.23.136 with SMTP id z130mr43498136qhd.36.1436620674048; 
	Sat, 11 Jul 2015 06:17:54 -0700 (PDT)
Received: by 10.140.93.162 with HTTP; Sat, 11 Jul 2015 06:17:53 -0700 (PDT)
In-Reply-To: <CAFdHNGj8BXAazAtHZsPe0qwxjRaxn6fQ4G-==bLCqwp+QH09Kg@mail.gmail.com>
References: <CAFdHNGg2dezj4V-i-E6dRLp99nZMQ_ErKdBo0OgQJ=9WPm90jQ@mail.gmail.com>
	<CABm2gDoAa5F5crO4enKO-Qqb+Zd3=9b8ohBDYmrygsPSWdevoQ@mail.gmail.com>
	<CAE-z3OWOoHfMaEN04CQ-j8tzmAr1+Evjh+tfHRDbF6F1jxykHA@mail.gmail.com>
	<CAFdHNGj8BXAazAtHZsPe0qwxjRaxn6fQ4G-==bLCqwp+QH09Kg@mail.gmail.com>
Date: Sat, 11 Jul 2015 14:17:53 +0100
Message-ID: <CAE-z3OW8rOmnBOpK=mFdxK_sATNL-2TRhqzv=r0in3EzF4cGoA@mail.gmail.com>
From: Tier Nolan <tier.nolan@gmail.com>
To: bitcoin-dev@lists.linuxfoundation.org
Content-Type: multipart/alternative; boundary=001a114232b6f429c0051a995037
X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW
	autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
Subject: Re: [bitcoin-dev] SPV Mining reveals a problematic incentive issue.
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Sat, 11 Jul 2015 13:17:56 -0000

--001a114232b6f429c0051a995037
Content-Type: text/plain; charset=UTF-8

On Sat, Jul 11, 2015 at 1:09 PM, Nathan Wilcox <nathan@leastauthority.com>
wrote:

>
> If there's any cost to non-SPV mining, and the chance that an incoming
> block contains only valid transactions is very high, isn't there still an
> incentive to make this timeout longer and longer?
>

The benefit drops off pretty quickly as the timeout increases (and
eventually goes negative).

You could look at it that headers having 4 states.

1) Valid
2) Probably Valid
3) Probably Invalid
4) Invalid

SPV mining puts newly received headers into the "probably valid" category.

It builds empty (coinbase only) blocks on top of probably valid headers and
build empty blocks on the header.

Once it receives the full block, it can change the state to Valid.  At that
point, it can build full blocks on top of the header.

As time passes without the full block being received/validated, it becomes
less and less likely that the block is actually valid.

The timeout is to recognize that fact.  Making the timeout 24 hours is not
likely to give the miner much benefit over making it 1-2 minutes.

Setting the timeout to low means that the miner sometimes switches away
from a header that turns out to be valid.

Setting it to high means that the miner ends up staying to long on a header
that turns out to be invalid.

At some point, the second effect is stronger than the first effect.  The
timeout needs to be high enough so switching away from valid headers is
rare but low enough so that it doesn't stay on invalid headers for ages.

If 99% of full blocks are received (and validated) within 30 seconds of the
header arriving, then it is reasonable for the miner to assume that if the
full block hasn't arrived within 60 seconds of the header arriving, then
the header is for an invalid block.

Yes. If it's rare enough, then skipping transaction validation saves more
> cost than the expected cost of mining atop an invalid block.  ... but if
> everyone follows that logic, the chance is no longer rare.
>

SPV miners don't actually produce invalid blocks though (other than
building on invalid blocks).  The full blocks they produce are still fully
checked blocks.


> SPV-mining is to prevent hashing hardware from having to waste power when
>> it isn't needed.
>>
>>
> It may be less of a problem if (when?) electricity costs dominate hardware
>> capital costs.
>>
>
> Oh.  This is a different explanation than Peter Todd's I linked to above,
> which claimed it was network latency in receiving transactions.
>

SPV mining is mainly to protect against latency.  The reason that matters
is that latency means that hashers end up building on blocks even though a
new block has been found.

You can look at it as wasting hashing power due to latency.

In the world where minting fees are very low, there is no point in SPV
mining.

I assume at the point, the memory pool/queue is a few blocks deep.  This
means that the pool can create a full block without having to wait for new
transactions to be sent in.

It still needs to wait for the new full block before it knows which
transactions to remove from its memory pool.

Pools have to pay their hashers for hashing power.  When minting fees are
tiny, pools only get income only from tx fees.

There is no point in creating empty blocks, since they don't pay anything.

Between when the block is found and the pool has a new block ready to mine,
there is no incentive for the pool to give out new work.  The stratum
protocol could be modified so pools can say.  (It might already support
this)

<Send work to hashers>

-- block is found by some other pool --

<Cancel work for miners>

-- pool builds new block --

<Send new work to hashers>

The cancel command says that the pool will not accept any of the work that
has been made obsolete.

This gives a window of 20-30 seconds after each block where the pool has
invalidated the old work, but does not send new work.  Hashers' hardware
would be stalled.

On the other hand, the pool that found the block could create a new block
straight away.  There is an incentive for hashers to have a connection to
multiple pools.

Pools might go pure pay per share.  The protocol would say how much they
are willing to pay for a share and the local mining proxy would pick the
most profitable pool.  This eliminates pools having lots of ways of saying
how they charge fees, you just connect to lots of pools and go with the one
that pays the most.

More interestingly, the average fee per byte for transactions in the memory
pool is likely to increase as time passes since the last block.

When two blocks are found very fast one after another, the second block is
likely to have lower average fees.  This is because the first block would
have included most of the high value transactions and there wasn't enough
time for new ones to arrive before the second block was found.

Hashers would end up getting variable payouts based on how long since the
last block was received.  If some miners increase/decrease their output
based on the fees the pools offer, then as time passes since the last
block, the hashing rate would increase.  This reduces the variation in
block to block times.

For example, if there was 1.5MB of transactions in the memory pool and they
all paid the same fee per byte, then the block would be a full block at
that rate.  However, there would only be 0.5MB of transactions left.  This
means that the next block would be half full and only be able to pay out
50% of the fee, but the difficulty would be the same.  The pay per hash
would be 50% lower.  Once 0.5MB of new transactions arrive, the fee would
be back up to the same as the previous block.

If there are major SHA256 altcoins (or side chains), then miners might end
up switching between coins.  The longer a coin went without a new block
being found, the more tx fees available in the memory pool, so the more
hashing power would decide to switch to that coin.

There could be a situation where adding a few more transactions to the
memory pool of a coin would cause a 100X increasing in hashing until the
block was found and then it stalling again as the hashing power switches
away.  This is similar to alt coins getting blasted by coin switching pools
and then dropping to almost no power.

--001a114232b6f429c0051a995037
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_quote">On S=
at, Jul 11, 2015 at 1:09 PM, Nathan Wilcox <span dir=3D"ltr">&lt;<a href=3D=
"mailto:nathan@leastauthority.com" target=3D"_blank">nathan@leastauthority.=
com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"mar=
gin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><br><div dir=3D=
"ltr"><span class=3D""></span><span class=3D""></span><div>If there&#39;s a=
ny cost to non-SPV mining, and the chance that an incoming block contains o=
nly valid transactions is very high, isn&#39;t there still an incentive to =
make this timeout longer and longer?<br></div></div></blockquote><div><br><=
/div><div>The benefit drops off pretty quickly as the timeout increases (an=
d eventually goes negative).<br><br></div><div>You could look at it that he=
aders having 4 states.<br><br></div><div>1) Valid<br></div><div>2) Probably=
 Valid<br></div><div>3) Probably Invalid<br></div><div>4) Invalid<br><br></=
div><div>SPV mining puts newly received headers into the &quot;probably val=
id&quot; category.<br><br></div><div>It builds empty (coinbase only) blocks=
 on top of probably valid headers and build empty blocks on the header.<br>=
<br></div><div>Once it receives the full block, it can change the state to =
Valid.=C2=A0 At that point, it can build full blocks on top of the header.<=
br></div><div><br></div><div>As time passes without the full block being re=
ceived/validated, it becomes less and less likely that the block is actuall=
y valid.<br><br></div><div>The timeout is to recognize that fact.=C2=A0 Mak=
ing the timeout 24 hours is not likely to give the miner much benefit over =
making it 1-2 minutes.<br><br></div><div>Setting the timeout to low means t=
hat the miner sometimes switches away from a header that turns out to be va=
lid.=C2=A0 <br><br></div><div>Setting it to high means that the miner ends =
up staying to long on a header that turns out to be invalid.<br><br></div><=
div>At some point, the second effect is stronger than the first effect.=C2=
=A0 The timeout needs to be high enough so switching away from valid header=
s is rare but low enough so that it doesn&#39;t stay on invalid headers for=
 ages.<br></div><div><br></div><div>If 99% of full blocks are received (and=
 validated) within 30 seconds of the header arriving, then it is reasonable=
 for the miner to assume that if the full block hasn&#39;t arrived within 6=
0 seconds of the header arriving, then the header is for an invalid block.<=
br></div><br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;b=
order-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><span class=3D=
""></span>Yes. If it&#39;s rare enough, then skipping transaction validatio=
n saves more cost than the expected cost of mining atop an invalid block.=
=C2=A0 ... but if everyone follows that logic, the chance is no longer rare=
.<br></div></blockquote><div><br></div><div>SPV miners don&#39;t actually p=
roduce invalid blocks though (other than building on invalid blocks).=C2=A0=
 The full blocks they produce are still fully checked blocks.<br></div><div=
>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8=
ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><span clas=
s=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;=
border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><=
div>SPV-mining is to prevent hashing hardware from having to waste power wh=
en it isn&#39;t needed.<br>=C2=A0</div></div></blockquote><div><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex"><div dir=3D"ltr"><div class=3D"gmail_=
extra">It may be less of a problem if (when?) electricity costs dominate ha=
rdware capital costs.<br></div></div></blockquote><div>=C2=A0</div></div></=
span><div>Oh.=C2=A0 This is a different explanation than Peter Todd&#39;s I=
 linked to above, which claimed it was network latency in receiving transac=
tions.<br></div></div></blockquote><div><br></div><div>SPV mining is mainly=
 to protect against latency.=C2=A0 The reason that matters is that latency =
means that hashers end up building on blocks even though a new block has be=
en found.<br><br></div><div>You can look at it as wasting hashing power due=
 to latency.<br></div><div><br></div><div>In the world where minting fees a=
re very low, there is no point in SPV mining.<br><br></div><div>I assume at=
 the point, the memory pool/queue is a few blocks deep.=C2=A0 This means th=
at the pool can create a full block without having to wait for new transact=
ions to be sent in.<br><br></div><div>It still needs to wait for the new fu=
ll block before it knows which transactions to remove from its memory pool.=
<br></div><div><br></div><div>Pools have to pay their hashers for hashing p=
ower.=C2=A0 When minting fees are tiny, pools only get income only from tx =
fees.=C2=A0 <br><br>There is no point in creating empty blocks, since they =
don&#39;t pay anything.<br><br></div><div>Between when the block is found a=
nd the pool has a new block ready to mine, there is no incentive for the po=
ol to give out new work.=C2=A0 The stratum protocol could be modified so po=
ols can say.=C2=A0 (It might already support this)<br><br></div><div>&lt;Se=
nd work to hashers&gt;<br><br></div><div>-- block is found by some other po=
ol --<br><br></div><div>&lt;Cancel work for miners&gt;<br><br></div><div>--=
 pool builds new block --<br><br></div><div>&lt;Send new work to hashers&gt=
;<br><br></div><div>The cancel command says that the pool will not accept a=
ny of the work that has been made obsolete.<br></div><div><br></div><div>Th=
is gives a window of 20-30 seconds after each block where the pool has inva=
lidated the old work, but does not send new work.=C2=A0 Hashers&#39; hardwa=
re would be stalled.<br><br></div><div>On the other hand, the pool that fou=
nd the block could create a new block straight away.=C2=A0 There is an ince=
ntive for hashers to have a connection to multiple pools.<br><br></div><div=
>Pools might go pure pay per share.=C2=A0 The protocol would say how much t=
hey are willing to pay for a share and the local mining proxy would pick th=
e most profitable pool.=C2=A0 This eliminates pools having lots of ways of =
saying how they charge fees, you just connect to lots of pools and go with =
the one that pays the most.<br></div></div><br></div><div class=3D"gmail_ex=
tra">More interestingly, the average fee per byte for transactions in the m=
emory pool is likely to increase as time passes since the last block.=C2=A0=
 <br><br>When two blocks are found very fast one after another, the second =
block is likely to have lower average fees.=C2=A0 This is because the first=
 block would have included most of the high value transactions and there wa=
sn&#39;t enough time for new ones to arrive before the second block was fou=
nd.<br><br></div><div class=3D"gmail_extra">Hashers would end up getting va=
riable payouts based on how long since the last block was received.=C2=A0 I=
f some miners increase/decrease their output based on the fees the pools of=
fer, then as time passes since the last block, the hashing rate would incre=
ase.=C2=A0 This reduces the variation in block to block times.<br><br></div=
><div class=3D"gmail_extra">For example, if there was 1.5MB of transactions=
 in the memory pool and they all paid the same fee per byte, then the block=
 would be a full block at that rate.=C2=A0 However, there would only be 0.5=
MB of transactions left.=C2=A0 This means that the next block would be half=
 full and only be able to pay out 50% of the fee, but the difficulty would =
be the same.=C2=A0 The pay per hash would be 50% lower.=C2=A0 Once 0.5MB of=
 new transactions arrive, the fee would be back up to the same as the previ=
ous block.<br><br></div><div class=3D"gmail_extra">If there are major SHA25=
6 altcoins (or side chains), then miners might end up switching between coi=
ns.=C2=A0 The longer a coin went without a new block being found, the more =
tx fees available in the memory pool, so the more hashing power would decid=
e to switch to that coin.<br><br></div><div class=3D"gmail_extra">There cou=
ld be a situation where adding a few more transactions to the memory pool o=
f a coin would cause a 100X increasing in hashing until the block was found=
 and then it stalling again as the hashing power switches away.=C2=A0 This =
is similar to alt coins getting blasted by coin switching pools and then dr=
opping to almost no power.<br></div></div>

--001a114232b6f429c0051a995037--