Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 359448EE for ; Wed, 11 Oct 2017 04:08:59 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-pf0-f169.google.com (mail-pf0-f169.google.com [209.85.192.169]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 2461944D for ; Wed, 11 Oct 2017 04:08:57 +0000 (UTC) Received: by mail-pf0-f169.google.com with SMTP id m28so465485pfi.11 for ; Tue, 10 Oct 2017 21:08:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=friedenbach-org.20150623.gappssmtp.com; s=20150623; h=from:content-transfer-encoding:mime-version:date:subject:message-id :references:in-reply-to:to; bh=vK4bJvC0MAtBkgD/aErxPc1NyQGTWK2XnBDbMDuvSKg=; b=kGFeEa+XxkilcGeU9bN/seTABOS0jZa/vxlQh0ab4+92k2xmniqIhEUAlaMsPbt6B7 1x9XVspEkp6wBG53xKUMyGSCc92jGYM90lJtSE1xWEEMMKlSZfyBXYiS02DeXGMEQ2oq oIv8DCNu9bqGkeaA3NAxLWwFNeabsjb20nT1rAD9IDQk7JcT2YVq00ZHYGMDY9NJSkCV YBnCQj0sqNsPDgJt4LrVt/qrfBCdGzroRv3LjzyF/Bu+tXp4w7wlJMgAj33PPiV71BB4 /xgB/InMC8siRtvxgyd0HkaTs5ZpLWAyDN7GfR2r66CP9lfA+qudIohAyy+LlYUzHBhb dMBQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version:date :subject:message-id:references:in-reply-to:to; bh=vK4bJvC0MAtBkgD/aErxPc1NyQGTWK2XnBDbMDuvSKg=; b=XSqU56mrs3rEDX0ir6pczBT2i9SJ5FBdGCqCHW6tNkijq9dkQXr4J64QudVe40SpFb HLFSQu2whTigvmIwMt9rNDZ7BlfymkhpstGN3r5MSLnZVYAEVEbK9WincYhXwthq2dLr bGq003Hmf2XQya2XQJxxogCooNSyMl1P17vfnM4gN9dJsoEXDDw1K4AaQ5w1zPDeObO+ Ys3/2wNUwukUQJWKxGbVKFGpSee7pAiKhajuWjn4PZjY5oadAH6sNZZhrwUN7wHBYDLV XTRqLB3BD62kqYAc595jbn3aRWKgjS2dXocynQJ7IgC2+F6NOqohfp/x2EfhNGtQXZ6R /xIg== X-Gm-Message-State: AMCzsaUFtgGlw3tRjcsEgsCxO8gF0bnwoDjf14SHwLnEI6+cnxqnHFAU NFGkO+40oinJv6W/LXZol2Zqw8tOIaTvWg== X-Google-Smtp-Source: AOwi7QAHpSs3bBlPfGPJjy1vWIxpLznnKW1G7+o5YNcqe3urnmWIVmfnbWTsI0xdHPNmNWFUmUlyvw== X-Received: by 10.98.223.72 with SMTP id u69mr10410294pfg.305.1507694936389; Tue, 10 Oct 2017 21:08:56 -0700 (PDT) Received: from ?IPv6:2601:646:8080:1291:999c:3433:2250:b597? ([2601:646:8080:1291:999c:3433:2250:b597]) by smtp.gmail.com with ESMTPSA id e84sm16069089pfd.1.2017.10.10.21.08.55 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 10 Oct 2017 21:08:55 -0700 (PDT) From: Mark Friedenbach Content-Type: multipart/alternative; boundary=Apple-Mail-69F5F7A5-40A1-423D-832C-7D76E4A60CA5 Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (1.0) Date: Tue, 10 Oct 2017 21:08:52 -0700 Message-Id: References: <1213518291.4328204.1507589852818.ref@mail.yahoo.com> <1213518291.4328204.1507589852818@mail.yahoo.com> In-Reply-To: To: Ben Kloester , bitcoin-dev@lists.linuxfoundation.org X-Mailer: iPhone Mail (15A402) X-Spam-Status: No, score=0.5 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_NONE,RCVD_IN_SORBS_SPAM autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] New difficulty algorithm needed for SegWit2x fork? (reformatted text) X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 11 Oct 2017 04:08:59 -0000 --Apple-Mail-69F5F7A5-40A1-423D-832C-7D76E4A60CA5 Content-Type: text/plain; charset=cp932 Content-Transfer-Encoding: quoted-printable You phrase the question as if =81gdeploying a hard fork to bitcoin=81h would= protect the bitcoin chain from the attack. But that=81fs not what happens. I= f you are hard forking from the perspective of deployed nodes, you are an di= fferent ledger, regardless of circumstance or who did it. Instead of there b= eing one altcoin fighting to take hashpower from bitcoin, there=81fd now be 2= . It is not at all obvious to me that this would be a better outcome. If that isn=81ft reason enough, changing the difficulty adjustment algorithm= doesn=81ft solve the underlying issue=81\hashpower not being aligned with u= sers=81f (or even its owners=81f) interests. Propose a fix to the underlying= cause and that might be worth considering, if it passes peer review. But wi= thout that you=81fd just be making the state of affairs arguably worse. And so yes, *if* this incentive problem can=81ft be solved, and the unaltere= d bitcoin chain dies from disuse after suffering a hashpower attack, especia= lly a centrally and/or purposefully instigated one, then bitcoin would be fa= iled a failed project. The thesis (and value proposition) of bitcoin is that a particular category o= f economic incentives can be used to solve the problem of creating a secure t= rustess ledger. If those incentives failed, then he thesis of bitcoin would h= ave been experimentally falsified, yes. Maybe the incentives can be made bet= ter to save the project, but we=81fd have to fix the source of the problem n= ot the symptoms. > On Oct 10, 2017, at 6:44 PM, Ben Kloester wrote: >=20 > Mark, this seems an awful lot like an answer of "no", to my question "Is t= here a contingency plan in the case that the incumbent chain following the B= itcoin Core consensus rules comes under 51% attack?" - is this a correct int= erpretation? >=20 > In fact, beyond a no, it seems like a "no, and I disagree with the idea of= creating one". >=20 > So if Bitcoin comes under successful 51%, the project, in your vision, has= simply failed? >=20 > Ben Kloester >=20 >=20 >> On 10 October 2017 at 13:19, Mark Friedenbach via bitcoin-dev wrote: >> The problem of fast acting but non vulnerable difficulty adjustment algor= ithms is interesting. I would certainly like to see this space further explo= red, and even have some ideas myself. >>=20 >> However without commenting on the technical merits of this specific propo= sal, I think it must be said upfront that the stated goal is not good. The l= argest technical concern (ignoring governance) over B2X is that it is a rush= ed, poorly reviewed hard fork. Hard forks should not be rushed, and they sho= uld receive more than the usual level of expert and community review. >>=20 >> I=81fm that light, doing an even more rushed hard fork on an even newer i= dea with even less review would be hypocritical at best. I would suggest ref= raming as a hardfork wishlist research problem for the next properly planned= hard fork, if one occurs. You might also find the hardfork research group a= more accommodating venue for this discussion: >>=20 >> https://bitcoinhardforkresearch.github.io/ >>=20 >>> On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev wrote: >>>=20 >>> Sorry, my previous email did not have the plain text I intended. >>>=20 >>> Background:=20 >>>=20 >>> The bitcoin difficulty algorithm does not seem to be a good one. If ther= e=20 >>> is a fork due to miners seeking maximum profit without due regard to=20 >>> security, users, and nodes, the "better" coin could end up being the=20 >>> minority chain. If 90% of hashrate is really going to at least initially= go=20 >>> towards using SegWit2x, BTC would face 10x delays in confirmations=20 >>> until the next difficulty adjustment, negatively affecting its price rel= ative=20 >>> to BTC1, causing further delays from even more miner abandonment=20 >>> (until the next adjustment). The 10% miners remaining on BTC do not=20 >>> inevitably lose by staying to endure 10x delays because they have 10x=20= >>> less competition, and the same situation applies to BTC1 miners. If the=20= >>> prices are the same and stable, all seems well for everyone, other thing= s=20 >>> aside. But if the BTC price does not fall to reflect the decreased hashr= ate,=20 >>> he situation seems to be a big problem for both coins: BTC1 miners will=20= >>> jump back to BTC when the difficulty adjustment occurs, initiating a=20 >>> potentially never-ending oscillation between the two coins, potentially=20= >>> worse than what BCH is experiencing. They will not issue coins too fast= =20 >>> like BCH because that is a side effect of the asymmetry in BCH's rise an= d=20 >>> fall algorithm.=20 >>>=20 >>> Solution:=20 >>>=20 >>> Hard fork to implement a new difficulty algorithm that uses a simple rol= ling=20 >>> average with a much smaller window. Many small coins have done this as=20= >>> a way to stop big miners from coming on and then suddenly leaving, leavi= ng=20 >>> constant miners stuck with a high difficulty for the rest of a (long) av= eraging=20 >>> window. Even better, adjust the reward based on recent solvetimes to=20= >>> motivate more mining (or less) if the solvetimes are too slow (or too fa= st).=20 >>> This will keep keep coin issuance rate perfectly on schedule with real t= ime.=20 >>>=20 >>> I recommend the following for Bitcoin, as fast, simple, and better than a= ny=20 >>> other difficulty algorithm I'm aware of. This is the result of a lot of= work the=20 >>> past year.=20 >>>=20 >>> =3D=3D=3D Begin difficulty algorithm =3D=3D=3D=20 >>> # Zawy v6 difficulty algorithm (modified for bitcoin)=20 >>> # Unmodified Zawy v6 for alt coins:=20 >>> # http://zawy1.blogspot.com/2017/07/best-difficulty-algorithm-zawy-v1b.h= tml=20 >>> # All my failed attempts at something better:=20 >>> # https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a180= 0be910ce345668a=20 >>> #=20 >>> # Keep negative solvetimes to correct bad timestamps.=20 >>> # Do not be tempted to use:=20 >>> # next_D =3D sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];=20= >>> # ST=3D Solvetime, TS =3D timestamp=20 >>>=20 >>> # set constants until next hard fork:=20 >>>=20 >>> T=3D600; # coin's TargetSolvetime=20 >>> N=3D30; # Averaging window. Smoother than N=3D15, faster response than N= =3D60.=20 >>> X=3D5;=20 >>> limit =3D X^(2/N); # limit rise and fall in case of timestamp manipulati= on=20 >>> adjust =3D 1/(1+0.67/N); # keeps avg solvetime on track=20 >>>=20 >>> # begin difficulty algorithm=20 >>>=20 >>> avg_ST=3D0; avg_D=3D0;=20 >>> for ( i=3Dheight; i > height-N; i--) { # go through N most recent blo= cks=20 >>> avg_ST +=3D (TS[i] - TS[i-1]) / N;=20 >>> avg_D +=3D D[i]/N;=20 >>> }=20 >>> avg_ST =3D T*limit if avg_ST > T*limit;=20 >>> avg_ST =3D T/limit if avg_ST < T/limit;=20 >>>=20 >>> next_D =3D avg_D * T / avg_ST * adjust;=20 >>>=20 >>> # Tim Olsen suggested changing reward to protect against hash attacks.=20= >>> # Karbowanek coin suggested something similar.=20 >>> # I could not find anything better than the simplest idea below.=20 >>> # It was a great surprise that coin issuance rate came out perfect.=20 >>> # BaseReward =3D coins per block=20 >>>=20 >>> next_reward =3D BaseReward * avg_ST / T;=20 >>>=20 >>> =3D=3D=3D=3D=3D=3D=3D end algo =3D=3D=3D=3D=20 >>>=20 >>> Due to the limit and keeping negative solvetimes in a true average,=20 >>> timestamp errors resulting in negative solvetimes are corrected in the n= ext=20 >>> block. Otherwise, one would need to do like Zcash and cause a 5-block=20= >>> delay in the response by resorting to the median of past 11 blocks (MPT)= =20 >>> as the most recent timestamp, offsetting the timestamps from their=20 >>> corresponding difficulties by 5 blocks. (it does not cause an averaging=20= >>> problem, but it does cause a 5-block delay in the response.) >>> _______________________________________________ >>> bitcoin-dev mailing list >>> bitcoin-dev@lists.linuxfoundation.org >>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>=20 >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>=20 >=20 --Apple-Mail-69F5F7A5-40A1-423D-832C-7D76E4A60CA5 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable You phrase the question as if =E2=80=9Cdepl= oying a hard fork to bitcoin=E2=80=9D would protect the bitcoin chain from t= he attack. But that=E2=80=99s not what happens. If you are hard forking from= the perspective of deployed nodes, you are an different ledger, regardless o= f circumstance or who did it. Instead of there being one altcoin fighting to= take hashpower from bitcoin, there=E2=80=99d now be 2. It is not at all obv= ious to me that this would be a better outcome.

If that i= sn=E2=80=99t reason enough, changing the difficulty adjustment algorithm doe= sn=E2=80=99t solve the underlying issue=E2=80=94hashpower not being aligned w= ith users=E2=80=99 (or even its owners=E2=80=99) interests. Propose a fix to= the underlying cause and that might be worth considering, if it passes peer= review. But without that you=E2=80=99d just be making the state of affairs a= rguably worse.

And so yes, *if* this incentive prob= lem can=E2=80=99t be solved, and the unaltered bitcoin chain dies from disus= e after suffering a hashpower attack, especially a centrally and/or purposef= ully instigated one, then bitcoin would be failed a failed project.

The thesis (and value proposition) of bitcoin is that a par= ticular category of economic incentives can be used to solve the problem of c= reating a secure trustess ledger. If those incentives failed, then he thesis= of bitcoin would have been experimentally falsified, yes. Maybe the incenti= ves can be made better to save the project, but we=E2=80=99d have to fix the= source of the problem not the symptoms.

On Oct 10, 2017,= at 6:44 PM, Ben Kloester <benkl= oester@gmail.com> wrote:

=
Mark, this seems an awful lot like an answer of "no", to my= question "Is there a contingency plan in t= he case that the incumbent chain following the Bitcoin Core consensus rules c= omes under 51% attack?" - is this a correct interpretation?

In fact, beyond a no, it seems like a "no, and I disagree with the id= ea of creating one".

<= /span>
So if Bitcoin comes under s= uccessful 51%, the project, in your vision, has simply failed?
<= /div>

Ben Kloester


On 10 October 2017 at 13:19, Mark Friedenbach= via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org<= /a>> wrote:
The= problem of fast acting but non vulnerable difficulty adjustment algorithms i= s interesting. I would certainly like to see this space further explored, an= d even have some ideas myself.

However without commenting= on the technical merits of this specific proposal, I think it must be said u= pfront that the stated goal is not good. The largest technical concern (igno= ring governance) over B2X is that it is a rushed, poorly reviewed hard fork.= Hard forks should not be rushed, and they should receive more than the usua= l level of expert and community review.

I=E2=80=99m= that light, doing an even more rushed hard fork on an even newer idea with e= ven less review would be hypocritical at best. I would suggest reframing as a= hardfork wishlist research problem for the next properly planned hard fork,= if one occurs. You might also find the hardfork research group a more accom= modating venue for this discussion:


On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev <bitcoin-dev= @lists.linuxfoundation.org> wrote:

Sorry, my previous email did not have the plain text I i= ntended.

Background:

The bitcoin difficulty algorithm does not seem to be a good one.= If there
is a fork due to miners seeking maximum profit wi= thout due regard to
security, users, and nodes, the "better= " coin could end up being the
minority chain. If 90% of has= hrate is really going to at least initially go
towards usin= g SegWit2x, BTC would face 10x delays in confirmations
unti= l the next difficulty adjustment, negatively affecting its price relative
to BTC1, causing further delays from even more miner abandonm= ent
(until the next adjustment). The 10% miners remaining o= n BTC do not
inevitably lose by staying to endure 10x delay= s because they have 10x
less competition, and the same situ= ation applies to BTC1 miners. If the
prices are the same an= d stable, all seems well for everyone, other things
aside. B= ut if the BTC price does not fall to reflect the decreased hashrate, =
he situation seems to be a big problem for both coins: BTC1 miners= will
jump back to BTC when the difficulty adjustment occur= s, initiating a
potentially never-ending oscillation betwee= n the two coins, potentially
worse than what BCH is experie= ncing.  They will not issue coins too fast
like BCH be= cause that is a side effect of the asymmetry in BCH's rise and
fall algorithm.

Solution:
<= span>
Hard fork to implement a new difficulty algorithm that= uses a simple rolling
average with a much smaller window.&= nbsp; Many small coins have done this as
a way to stop big m= iners from coming on and then suddenly leaving, leaving
con= stant miners stuck with a high difficulty for the rest of a (long) averaging=
window.  Even better, adjust the reward based on rece= nt solvetimes to
motivate more mining (or less) if the solv= etimes are too slow (or too fast).
This will keep keep coin= issuance rate perfectly on schedule with real time.

I recommend the following for Bitcoin, as fast, simple, and bette= r than any
other difficulty algorithm I'm aware of.  T= his is the result of a lot of work the
past year.
=3D=3D=3D Begin difficulty algorithm =3D=3D=3D
# Zawy v6 difficulty algorithm (modified for bitcoin) # Unmodified Zawy v6 for alt coins:
# http://zawy1.blogspot.com/2017/07/best-difficulty-a= lgorithm-zawy-v1b.html
# All my failed attempts at some= thing better:
# htt= ps://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a18= 00be910ce345668a
#
# Keep negativ= e solvetimes to correct bad timestamps.
# Do not be tempted= to use:
# next_D =3D sum(last N Ds) * T / [max(last N TSs)= - min(last N TSs];
# ST=3D Solvetime, TS =3D timestamp

# set constants until next hard fork: =

T=3D600; # coin's TargetSolvetime
N=3D30; # Averaging window. Smoother than N=3D15, faster response than N=3D= 60.
X=3D5;
limit =3D X^(2/N); # limit rise= and fall in case of timestamp manipulation
adjust =3D 1/(1= +0.67/N);  # keeps avg solvetime on track

<= span># begin difficulty algorithm

avg_ST=3D= 0; avg_D=3D0;
for ( i=3Dheight;  i > height-N; &nbs= p;i--) {  # go through N most recent blocks
avg_ST +=3D= (TS[i] - TS[i-1]) / N;
avg_D +=3D D[i]/N;
}
avg_ST =3D T*limit if avg_ST > T*limit;
avg_ST =3D T/limit if avg_ST < T/limit;

<= span>next_D =3D avg_D * T / avg_ST * adjust;

# Tim Olsen suggested changing reward to protect against hash attacks.
# Karbowanek coin suggested something similar.
# I could not find anything better than the simplest idea below. <= br># It was a great surprise that coin issuance rate came out perfect.=
# BaseReward =3D coins per block
<= br>next_reward =3D BaseReward * avg_ST / T;
=3D=3D=3D=3D=3D=3D=3D end algo =3D=3D=3D=3D
=
Due to the limit and keeping negative solvetimes in a true average= ,
timestamp errors resulting in negative solvetimes are cor= rected in the next
block. Otherwise, one would need to do l= ike Zcash and cause a 5-block
delay in the response by reso= rting to the median of past 11 blocks (MPT)
as the most rec= ent timestamp, offsetting the timestamps from their
corresp= onding difficulties by 5 blocks. (it does not cause an averaging
= problem, but it does cause a 5-block delay in the response.)_______________________________________________
= bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundati= on.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

_____________________________________________= __
bitcoin-dev mailing list
bitcoin-dev@lists.<= wbr>linuxfoundation.org
https://lists.linuxfoundation.org/m= ailman/listinfo/bitcoin-dev


= --Apple-Mail-69F5F7A5-40A1-423D-832C-7D76E4A60CA5--