Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 62B225B1 for ; Tue, 10 Oct 2017 10:34:38 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from NAM03-BY2-obe.outbound.protection.outlook.com (mail-oln040092006056.outbound.protection.outlook.com [40.92.6.56]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 17EBEF8 for ; Tue, 10 Oct 2017 10:34:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=YmOp+kYo6BtqDhgk7zum39d2fFeMd8Xan6/vrj4IQyA=; b=ZsQ9F/RgaZowRJTp04jZavFii8eTS8sTl157uZBz8JsMpGUt6B3X6Cz9g7iscQp+qRFmMskvBSIRRTLp44k8HyQOHoYx29UkcmQv6kOMHyaG1X7zP6M6DxCgmZEgSag23+9cjGGay/xSlB1tLpOrJDS0Ie/JYLGZKf45Y2ruPta1U55vdfLqxtKB1YI3UVZ6QYOZ//uoXvOBo5Tp6QYFFfdUYbLDIwwN+CYZSYeEdr23yhYt1sYii1e9B/D9TOioCTX0lYzQd5qCSqqd0l4aVxl7jQ5KaSa3YiriIv14+6FcHh8+ynyHs1Hl0XhALrQPEZQ89bVcD3CJiRaPv6tVOQ== Received: from BY2NAM03FT023.eop-NAM03.prod.protection.outlook.com (10.152.84.51) by BY2NAM03HT181.eop-NAM03.prod.protection.outlook.com (10.152.85.171) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.20.77.10; Tue, 10 Oct 2017 10:34:35 +0000 Received: from BN6PR15MB1761.namprd15.prod.outlook.com (10.152.84.52) by BY2NAM03FT023.mail.protection.outlook.com (10.152.84.226) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.20.77.10 via Frontend Transport; Tue, 10 Oct 2017 10:34:35 +0000 Received: from BN6PR15MB1761.namprd15.prod.outlook.com ([10.174.238.139]) by BN6PR15MB1761.namprd15.prod.outlook.com ([10.174.238.139]) with mapi id 15.20.0077.020; Tue, 10 Oct 2017 10:34:35 +0000 From: greg misiorek To: Mark Friedenbach , Bitcoin Protocol Discussion Thread-Topic: [bitcoin-dev] New difficulty algorithm needed for SegWit2x fork? (reformatted text) Thread-Index: AQHTQW5CruXuX0qbfUqcHokoBKEkraLc4wmM Date: Tue, 10 Oct 2017 10:34:35 +0000 Message-ID: References: <1213518291.4328204.1507589852818.ref@mail.yahoo.com> <1213518291.4328204.1507589852818@mail.yahoo.com>, In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-incomingtopheadermarker: OriginalChecksum:CDB2CC39DC1AD08922274C629360EFCF3FE4BD56087B44DA6263C60FA5958E8D; UpperCasedChecksum:B88227088F0E9E49A4398959E33FC089D05B938C06C094A1E1F0E092DE02F4C8; SizeAsReceived:7265; Count:46 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [zAayYn5iqXMr8JBcUKTC8BS14VuCCuFt] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1; BY2NAM03HT181; 6:TYMk9Uj2CfJhiT88HUEdWVpTuoo3xeBI1EGWhpxNfssgGR6crJW0h9arpJyCFItwDY/Plhlt+27VZKHuQ8zuxN/sKZbWzp0Yq7ZxfRh2dslYrbPlD8rV3WY57pNrRGg7QQM5ukBQpmHZkw9ayBgotIpYhtLv2RnQunmq5wZBtjtyFOUft1mPPQj6hKiMsFNmMqGsWDx2enKj+MkL6oBtB7LmfRb/8iL8mH6IV3dwv7It0/cwdCU415/FQORfYCDY/Sr4gLvaCGhkJ2lu8AzLW6vIeyi+OHXLAoVQtdyxxTfEkj/CXAWCWDJK+h1dytcvEp3UVFMxu6Yk61l9BUABhg==; 5:7JR86GdE6Ag20bjFVG7iw2djLHz+CLTQcmNbgX+d0kZBHNWV4PrDLvnKgpq7Rl7RAvnn8juU82PA5XTsXD7R9rHVkXYUBLf4aGYLXD1eRywiCRg6SMN7/kY4xVdjnGamqEjcw5DZsLSvLfB9OyDINg==; 24:jJKz5LUc9HwTWT1HaF/Aw99PGRQU9rZr/KOIrqfMTpK4IitIzQVqTSsXsYFyz+8CrAiYog6O1169Hc+GN8u5AZ8quw1H7LzOFx++ofagBDo=; 7:D0erkB3Z7qGdQcaZvwR4N9Yh7mmS2n8imWZfiCgb7SGg6d/HPXlEcsmBlam3cglJrvdsuigeiahyD7JzMpfDDgaRaKsRrkIkvWlrf8Y/in3TJqP76LzE2KPuvOYo8f7tkDe8POVQ96NUBCL2jp+S4ryK0gyKIC/AFnUTxhpY0X0IgPUIDRMidgtdi3EBwXuXE1NenRtOkHjIVjZ93QkvuPhKqM7qddU6ck5odX3vWo0= x-incomingheadercount: 46 x-eopattributedmessage: 0 x-ms-office365-filtering-correlation-id: ab95229a-ff7f-4ca9-1421-08d50fca8094 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031324274)(2017031323274)(2017031322404)(1601125374)(1603101448)(1701031045); SRVR:BY2NAM03HT181; x-ms-traffictypediagnostic: BY2NAM03HT181: authentication-results: outbound.protection.outlook.com; spf=skipped (originating message); dkim=none (message not signed) header.d=none; dmarc=none action=none header.from=hotmail.com; x-exchange-antispam-report-test: UriScan:(166708455590820)(192374486261705); x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(444000031); SRVR:BY2NAM03HT181; BCL:0; PCL:0; RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095); SRVR:BY2NAM03HT181; x-forefront-prvs: 04569283F9 x-forefront-antispam-report: SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT; SFP:1901; SCL:1; SRVR:BY2NAM03HT181; H:BN6PR15MB1761.namprd15.prod.outlook.com; FPR:; SPF:None; LANG:; spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: multipart/alternative; boundary="_000_BN6PR15MB17610E1A315F50BDD54D336CB1750BN6PR15MB1761namp_" MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-originalarrivaltime: 10 Oct 2017 10:34:35.5604 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY2NAM03HT181 X-Spam-Status: No, score=-0.1 required=5.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE autolearn=disabled version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Tue, 10 Oct 2017 11:37:58 +0000 Subject: Re: [bitcoin-dev] New difficulty algorithm needed for SegWit2x fork? (reformatted text) X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Oct 2017 10:34:38 -0000 --_000_BN6PR15MB17610E1A315F50BDD54D336CB1750BN6PR15MB1761namp_ Content-Type: text/plain; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable Yes, I agree. Hard forks should be as much scrutinized by fellow bitcoiners= , i.e. developers and holders and not only rushed by miners or some other i= nvestment gurus, whose incentives are not entirely clear, to remain as dece= ntralized as economically possible. ________________________________ From: bitcoin-dev-bounces@lists.linuxfoundation.org on behalf of Mark Friedenbach via bitcoin-dev Sent: Monday, October 9, 2017 10:19 PM To: Scott Roberts; Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] New difficulty algorithm needed for SegWit2x for= k? (reformatted text) The problem of fast acting but non vulnerable difficulty adjustment algorit= hms is interesting. I would certainly like to see this space further explor= ed, and even have some ideas myself. However without commenting on the technical merits of this specific proposa= l, I think it must be said upfront that the stated goal is not good. The la= rgest technical concern (ignoring governance) over B2X is that it is a rush= ed, poorly reviewed hard fork. Hard forks should not be rushed, and they sh= ould receive more than the usual level of expert and community review. I=92m that light, doing an even more rushed hard fork on an even newer idea= with even less review would be hypocritical at best. I would suggest refra= ming as a hardfork wishlist research problem for the next properly planned = hard fork, if one occurs. You might also find the hardfork research group a= more accommodating venue for this discussion: https://bitcoinhardforkresearch.github.io/ Welcome - Bitcoin Hard Fork Research bitcoinhardforkresearch.github.io Bitcoin Hard Fork Research This website will be updated with relevant ongoi= ng information about Bitcoin hard fork research. Resources: BIP-MMHF, draft= patch last ... On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev > wrote: Sorry, my previous email did not have the plain text I intended. Background: The bitcoin difficulty algorithm does not seem to be a good one. If there is a fork due to miners seeking maximum profit without due regard to security, users, and nodes, the "better" coin could end up being the minority chain. If 90% of hashrate is really going to at least initially go towards using SegWit2x, BTC would face 10x delays in confirmations until the next difficulty adjustment, negatively affecting its price relati= ve to BTC1, causing further delays from even more miner abandonment (until the next adjustment). The 10% miners remaining on BTC do not inevitably lose by staying to endure 10x delays because they have 10x less competition, and the same situation applies to BTC1 miners. If the prices are the same and stable, all seems well for everyone, other things aside. But if the BTC price does not fall to reflect the decreased hashrate= , he situation seems to be a big problem for both coins: BTC1 miners will jump back to BTC when the difficulty adjustment occurs, initiating a potentially never-ending oscillation between the two coins, potentially worse than what BCH is experiencing. They will not issue coins too fast like BCH because that is a side effect of the asymmetry in BCH's rise and fall algorithm. Solution: Hard fork to implement a new difficulty algorithm that uses a simple rollin= g average with a much smaller window. Many small coins have done this as a way to stop big miners from coming on and then suddenly leaving, leaving constant miners stuck with a high difficulty for the rest of a (long) avera= ging window. Even better, adjust the reward based on recent solvetimes to motivate more mining (or less) if the solvetimes are too slow (or too fast)= . This will keep keep coin issuance rate perfectly on schedule with real time= . I recommend the following for Bitcoin, as fast, simple, and better than any other difficulty algorithm I'm aware of. This is the result of a lot of wo= rk the past year. =3D=3D=3D Begin difficulty algorithm =3D=3D=3D # Zawy v6 difficulty algorithm (modified for bitcoin) # Unmodified Zawy v6 for alt coins: # http://zawy1.blogspot.com/2017/07/best-difficulty-algorithm-zawy-v1b.html # All my failed attempts at something better: # https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be= 910ce345668a # # Keep negative solvetimes to correct bad timestamps. # Do not be tempted to use: # next_D =3D sum(last N Ds) * T / [max(last N TSs) - min(last N TSs]; # ST=3D Solvetime, TS =3D timestamp # set constants until next hard fork: T=3D600; # coin's TargetSolvetime N=3D30; # Averaging window. Smoother than N=3D15, faster response than N=3D= 60. X=3D5; limit =3D X^(2/N); # limit rise and fall in case of timestamp manipulation adjust =3D 1/(1+0.67/N); # keeps avg solvetime on track # begin difficulty algorithm avg_ST=3D0; avg_D=3D0; for ( i=3Dheight; i > height-N; i--) { # go through N most recent blocks avg_ST +=3D (TS[i] - TS[i-1]) / N; avg_D +=3D D[i]/N; } avg_ST =3D T*limit if avg_ST > T*limit; avg_ST =3D T/limit if avg_ST < T/limit; next_D =3D avg_D * T / avg_ST * adjust; # Tim Olsen suggested changing reward to protect against hash attacks. # Karbowanek coin suggested something similar. # I could not find anything better than the simplest idea below. # It was a great surprise that coin issuance rate came out perfect. # BaseReward =3D coins per block next_reward =3D BaseReward * avg_ST / T; =3D=3D=3D=3D=3D=3D=3D end algo =3D=3D=3D=3D Due to the limit and keeping negative solvetimes in a true average, timestamp errors resulting in negative solvetimes are corrected in the next block. Otherwise, one would need to do like Zcash and cause a 5-block delay in the response by resorting to the median of past 11 blocks (MPT) as the most recent timestamp, offsetting the timestamps from their corresponding difficulties by 5 blocks. (it does not cause an averaging problem, but it does cause a 5-block delay in the response.) _______________________________________________ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev --_000_BN6PR15MB17610E1A315F50BDD54D336CB1750BN6PR15MB1761namp_ Content-Type: text/html; charset="Windows-1252" Content-Transfer-Encoding: quoted-printable

Yes, I agree. Hard forks should be as much scrutinized by fellow bitcoin= ers, i.e. developers and holders and not only rushed by miners or some= other investment gurus, whose incentives are not entirely clear, to r= emain as decentralized as economically possible.


From: bitcoin-dev-bounces@l= ists.linuxfoundation.org <bitcoin-dev-bounces@lists.linuxfoundation.org&= gt; on behalf of Mark Friedenbach via bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Monday, October 9, 2017 10:19 PM
To: Scott Roberts; Bitcoin Protocol Discussion
Subject: Re: [bitcoin-dev] New difficulty algorithm needed for SegWi= t2x fork? (reformatted text)
 
The problem of fast acting but non vulnerable difficulty adjustment al= gorithms is interesting. I would certainly like to see this space further e= xplored, and even have some ideas myself.

However without commenting on the technical merits of this specific pr= oposal, I think it must be said upfront that the stated goal is not good. T= he largest technical concern (ignoring governance) over B2X is that it is a= rushed, poorly reviewed hard fork. Hard forks should not be rushed, and they should receive more than the usu= al level of expert and community review.

I=92m that light, doing an even more rushed hard fork on an even newer= idea with even less review would be hypocritical at best. I would suggest = reframing as a hardfork wishlist research problem for the next properly pla= nned hard fork, if one occurs. You might also find the hardfork research group a more accommodating venue for= this discussion:

https://bitcoinhardforkresearch.github.io/<= /div>
bitcoinhardforkresearch.github.io
Bitcoin Hard Fork Research This website will be updated with relevant ongoi= ng information about Bitcoin hard fork research. Resources: BIP-MMHF, draft= patch last ...


On Oct 9, 2017, at 3:57 PM, Scott Roberts via bitcoin-dev <bitcoin-dev@lists.linuxfoundati= on.org> wrote:

Sorry, my previous email did not have the plain text I intended.=

Background:

The bitcoin difficulty algorithm does not seem to be a good one. If t= here
is a fork due to miners seeking maximum profit without due regard to =
security, users, and nodes, the "better" coin could end up = being the
minority chain. If 90% of hashrate is really going to at least initia= lly go
towards using SegWit2x, BTC would face 10x delays in confirmations
until the next difficulty adjustment, negatively affecting its price = relative
to BTC1, causing further delays from even more miner abandonment
(until the next adjustment). The 10% miners remaining on BTC do not <= /span>
inevitably lose by staying to endure 10x delays because they have 10x=
less competition, and the same situation applies to BTC1 miners. If t= he
prices are the same and stable, all seems well for everyone, other th= ings
aside. But if the BTC price does not fall to reflect the decreased ha= shrate,
he situation seems to be a big problem for both coins: BTC1 miners wi= ll
jump back to BTC when the difficulty adjustment occurs, initiating a =
potentially never-ending oscillation between the two coins, potential= ly
worse than what BCH is experiencing.  They will not issue coins = too fast
like BCH because that is a side effect of the asymmetry in BCH's rise= and
fall algorithm.

Solution:

Hard fork to implement a new difficulty algorithm that uses a simple = rolling
average with a much smaller window.  Many small coins have done = this as
a way to stop big miners from coming on and then suddenly leaving, le= aving
constant miners stuck with a high difficulty for the rest of a (long)= averaging
window.  Even better, adjust the reward based on recent solvetim= es to
motivate more mining (or less) if the solvetimes are too slow (or too= fast).
This will keep keep coin issuance rate perfectly on schedule with rea= l time.

I recommend the following for Bitcoin, as fast, simple, and better th= an any
other difficulty algorithm I'm aware of.  This is the result of = a lot of work the
past year.

=3D=3D=3D Begin difficulty algorithm =3D=3D=3D
# Zawy v6 difficulty algorithm (modified for bitcoin)
# Unmodified Zawy v6 for alt coins:
# http://zawy1.blogspot.com/2017/07/best-difficulty-algorithm-zawy-v1b.html
# All my failed attempts at something better:
# https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be91= 0ce345668a
#
# Keep negative solvetimes to correct bad timestamps.
# Do not be tempted to use:
# next_D =3D sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];=
# ST=3D Solvetime, TS =3D timestamp

# set constants until next hard fork:

T=3D600; # coin's TargetSolvetime
N=3D30; # Averaging window. Smoother than N=3D15, faster response tha= n N=3D60.
X=3D5;
limit =3D X^(2/N); # limit rise and fall in case of timestamp manipul= ation
adjust =3D 1/(1+0.67/N);  # keeps avg solvetime on track

# begin difficulty algorithm

avg_ST=3D0; avg_D=3D0;
for ( i=3Dheight;  i > height-N;  i--) {  # go thro= ugh N most recent blocks
avg_ST +=3D (TS[i] - TS[i-1]) / N;
avg_D +=3D D[i]/N;
}
avg_ST =3D T*limit if avg_ST > T*limit;
avg_ST =3D T/limit if avg_ST < T/limit;

next_D =3D avg_D * T / avg_ST * adjust;

# Tim Olsen suggested changing reward to protect against hash attacks= .
# Karbowanek coin suggested something similar.
# I could not find anything better than the simplest idea below.
# It was a great surprise that coin issuance rate came out perfect. <= /span>
# BaseReward =3D coins per block

next_reward =3D BaseReward * avg_ST / T;

=3D=3D=3D=3D=3D=3D=3D end algo =3D=3D=3D=3D

Due to the limit and keeping negative solvetimes in a true average, <= /span>
timestamp errors resulting in negative solvetimes are corrected in th= e next
block. Otherwise, one would need to do like Zcash and cause a 5-block=
delay in the response by resorting to the median of past 11 blocks (M= PT)
as the most recent timestamp, offsetting the timestamps from their
corresponding difficulties by 5 blocks. (it does not cause an averagi= ng
problem, but it does cause a 5-block delay in the response.) _______________________________________________
bitcoin-dev mailing list
bitcoin-dev@= lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
--_000_BN6PR15MB17610E1A315F50BDD54D336CB1750BN6PR15MB1761namp_--