Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B1C21A5E for ; Wed, 5 Apr 2017 10:41:48 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-oln040092003045.outbound.protection.outlook.com [40.92.3.45]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 380A390 for ; Wed, 5 Apr 2017 10:41:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=hotmail.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=3hn64p52QZxHeQkavy8vjGr3NvyFMJQxum1BSBTkTns=; b=ar+fx2UuFu7vHFqhFkJmG5VLzT2CPVyPfXUEsWzSF3LE3+B1JvZU0Quh6MFWArh02p6ywXNVQ9lN2NQ5CBRB7yPioJZygVUC2S0zjR2RurCcJVcnQTHF653m55J3yz2v41GmIutZ2UXFFIxEc6LsS+8mYmGvFMLd2L58y9LMQhsCPOngxHiM15no7FkQw1Pmu6rb4y+78LoSfxV0oPaoUSlbrh8BZbxBRlBf4ZqXBuuLmwyUYu0Q8xUHKLGF7noy7cSH+87Exaa7wWfCQQae4RrdL3QEKx7eAeYt857WNBbB5lPRhCy9O+rRj6C5nA7DWsfU+rM2p5Cm7qukZKbW5A== Received: from CY1NAM02FT041.eop-nam02.prod.protection.outlook.com (10.152.74.57) by CY1NAM02HT184.eop-nam02.prod.protection.outlook.com (10.152.75.42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P384) id 15.1.1005.5; Wed, 5 Apr 2017 10:41:45 +0000 Received: from BLUPR15MB0051.namprd15.prod.outlook.com (10.152.74.51) by CY1NAM02FT041.mail.protection.outlook.com (10.152.74.156) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.1005.5 via Frontend Transport; Wed, 5 Apr 2017 10:41:45 +0000 Received: from BLUPR15MB0051.namprd15.prod.outlook.com ([10.161.124.25]) by BLUPR15MB0051.namprd15.prod.outlook.com ([10.161.124.25]) with mapi id 15.01.1005.019; Wed, 5 Apr 2017 10:41:46 +0000 From: greg misiorek To: Johnson Lau , Bitcoin Protocol Discussion Thread-Topic: [bitcoin-dev] A different approach to define and understand softforks and hardforks Thread-Index: AQHSrfdgSYehsY5SnUaPlU2LRf7vEKG2loEg Date: Wed, 5 Apr 2017 10:41:45 +0000 Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: authentication-results: xbt.hk; dkim=none (message not signed) header.d=none;xbt.hk; dmarc=none action=none header.from=hotmail.com; x-incomingtopheadermarker: OriginalChecksum:CCA9C787FFC8B0F9D69106AED63D529A2DA706B304F8EBFA57A2C3BC26095D06; UpperCasedChecksum:A290AEC0F71FCD0042A2D7A2BFEA559911F8B6291377AC35C3A9685F17567DE6; SizeAsReceived:8207; Count:42 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [uA76uyd28WBhtetLhxM07ctnArdeUYZl] x-microsoft-exchange-diagnostics: 1; CY1NAM02HT184; 5:mVkLIjmUhRntm5nPIU2+ytCs1++dTzgEvgGdQ/eF4yzWkafU8TWDoDEbclGBZoL3cH7aJ2Omtu8JyXOZXOo38+CZFx3Anp39r+qZ7pqbqW0C+iGWwuGcH0fGhp4NKFTzWvWGNIA0PSkUQnhx4eFTRA==; 24:wxOoeHAUirwBX2SGiS4XEL8OYWf24AHt7bLeQOMbmgDlAsHwglGqxqnsQRO1c7y0gef3fa9S5yrrxFlSle0AbCcWdBXnQVtoKc6Id04XvWs=; 7:4v04YsOHs8VbyXwr6c4e5zGG7APC0TQwaaw3XJPQ74amW77y3xTz0WMXgp/odkLOBcMiblM3I9OvEx3ix0aub0DaEivYu49bgxFKc3M1Q3wf+cw/5Opb0DQTGMzPoxQc40Jjw+PiBdEJdfQUabmgy0xykd4WQjjwz8x1gGnfIjOLoRXGLC3h82fjL+ktv+ueHmob6BZ/73HEvJS1+KubFPCFLF9GdeAGYDQGRw8oCECZ1XBJgSjSYXzS0rCOPZTHP1H/71h7uNnI/EHyWUea9Ub8dqDXct/upRpho4vkMvs3nhhRm+MgQ7D/dkUGRW1b x-incomingheadercount: 42 x-eopattributedmessage: 0 x-forefront-antispam-report: EFV:NLI; SFV:NSPM; SFS:(7070007)(98901004); DIR:OUT; SFP:1901; SCL:1; SRVR:CY1NAM02HT184; H:BLUPR15MB0051.namprd15.prod.outlook.com; FPR:; SPF:None; LANG:en; x-ms-office365-filtering-correlation-id: bfa4afe2-b40f-46b9-3508-08d47c105b66 x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(22001)(201702061074)(5061506573)(5061507331)(1603103135)(2017031320274)(2017031322274)(1603101448)(1601125374)(1701031045); SRVR:CY1NAM02HT184; x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(444000031); SRVR:CY1NAM02HT184; BCL:0; PCL:0; RULEID:; SRVR:CY1NAM02HT184; x-forefront-prvs: 0268246AE7 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: multipart/alternative; boundary="_000_BLUPR15MB0051872C3E1FB74DF3FCA989B10A0BLUPR15MB0051namp_" MIME-Version: 1.0 X-OriginatorOrg: hotmail.com X-MS-Exchange-CrossTenant-originalarrivaltime: 05 Apr 2017 10:41:45.9567 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Internet X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY1NAM02HT184 X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Wed, 05 Apr 2017 13:18:09 +0000 Subject: Re: [bitcoin-dev] A different approach to define and understand softforks and hardforks X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 05 Apr 2017 10:41:48 -0000 --_000_BLUPR15MB0051872C3E1FB74DF3FCA989B10A0BLUPR15MB0051namp_ Content-Type: text/plain; charset="windows-1256" Content-Transfer-Encoding: quoted-printable I'm not an expert to refute this classification, but would love to see thos= e points addressed by those in the know, without resorting to ad hominem, e= ven though I know it's really hard. thx, gm ________________________________ From: Johnson Lau via bitcoin-dev Sent: =FD4/=FD5/=FD2017 6:28 AM To: bitcoin-dev Subject: [bitcoin-dev] A different approach to define and understand softfo= rks and hardforks Softforks and hardforks are usually defined in terms of block validity (BIP= 99): making valid blocks invalid is a softfork, making invalid blocks valid= is a hardfork, and SFs are usually considered as less disruptive as it is = considered to be =93opt-in=94. However, as shown below this technical defin= ition could be very misleading. Here I=92m trying to redefine the terminolo= gy in terms of software upgrade necessity and difficulty. Softforks are defined as consensus rule changes that non-upgraded software = will be able to function exactly as usual, as if the rule changes have neve= r happened Hardforks are defined as consensus rule changes that non-upgraded software = will cease to function or be severely handicapped SFs and HFs under this definitions is a continuum, which I call it =93hardf= ork-ness=94. A pure softfork has no hardfork-ness. *Mining node Under this definitions, for miners, any trivial consensus rule changes is s= omewhat a hardfork, as miners can=92t reliably use non-upgraded software to= create blocks. However, there is still 3 levels of =93hardfork-ness=94, fo= r example: 1. Those with lower hardfork-ness would be the SFs that miners do not need = to upgrade their software at all. Instead, the minimum requirement is to se= tup a boarder node with latest rules to make sure they won=92t mine on top = of an invalid block. Examples include CSV and Segwit 2. Some SFs have higher hardfork-ness, for example BIP65 and BIP66. The min= imum actions needed include setting up a boarder node and change the block = version. BIP34 has even higher hardfork-ness as more actions are needed to = follow the new consensus. 3. Anything else, ranging from simple HFs like BIP102 to complete HFs like = spoonnet, or soft-hardfork like forcenet, have the highest hardfork-ness. I= n these cases, boarder nodes are completely useless. Miners have to upgrade= their servers in order to stay with the consensus. *Non-mining full node Similarly, in terms of non-mining full node, as the main function is to ful= ly-validate all applicable rules on the network, any consensus change is a = hardfork for this particular function. However, a technical SF would have m= uch lower hardfork-ness than a HF, as a border node is everything needed in= a SF. Just consider a company has some difficult-to-upgrade software that = depends on Bitcoin Core 0.8. Using a 0.13.1+ boarder node will make sure th= ey will always follow the latest rules. In case of a HF, they have no choic= e but to upgrade the backend system. So we may use the costs of running a boarder node to further define the har= dfork-ness of SFs, and it comes to the additional resources needed: 1. Things like BIP34, 65, 66, and CSV involves trivial resources use so the= y have lowest hardfork-ness. 2. Segwit is higher because of increased block size. 3. Extension block has very high hardfork-ness as people may not have enoug= h resources to run a boarder node. * Fully validating wallets In terms of the wallet function in full node, without considering the issue= s of validation, the hardfork-ness could be ranked as below: 1. BIP34, 65, 66, CSV, segwit all have no hardfork-ness for wallets. Non-up= graded wallets will work exactly in the same way as before. Users won=92t n= otice any change at all. (In some cases they may not see a new tx until it = has 1 confirmation, but this is a mild issue and 0-conf is unsafe anyway) 2. Extension block, as presented in my January post ( https://lists.linuxfo= undation.org/pipermail/bitcoin-dev/2017-January/013490.html ), has higher h= ardfork-ness, as users of legacy wallets may find it difficult to receive p= ayments from upgraded wallet. However, once they got paid, the user experie= nce is same as before 3. Another extension block proposal ( https://github.com/tothemoon-org/exte= nsion-blocks ) has very high hardfork-ness for wallets, as legacy wallets w= ill frequently and suddenly find that incoming and outgoing txs becoming in= valid, and need to sign the invalidated txs again, even no one is trying to= double spend. 4. Hardfork rule changes have highest hardfork-ness for full node wallets I=92ll explain the issues with extension block in a separate post in detail= s * Real SPV wallet The SPV wallets as proposed by Satoshi should have the ability to fully val= idate the rules when needed, so they could be somehow seen as fully validat= ing wallets. So far, real SPV wallet is just vapourware. * Fake SPV wallet, aka light wallet All the so-called SPV wallets we have today are fake SPV according to white= paper definition. Since they validate nothing, the hardfork-ness profile is= very different: 1. BIP34, 65, 66, CSV, segwit has no hardfork-ness for light wallets. Block= size HF proposals (BIP10x) and Bitcoin Unlimited also have no hardfork-nes= s (superficially, but not philosophically). Along the same line, even an in= flation hardfork has no hardfork-ness for light wallets. 2. Extension block has the same kind of hardfork-ness issue as I mentioned. 3. HFs that deliberately breaks light wallets, such as spoonnet, is a compl= ete hardfork. While some people try to leverage weakness of light wallets, the inability = to validate any important rules like block size, double spending, and infla= tion is a serious vulnerability. =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D Before I finish, I=92d also like to analyse some other interesting cases. 1. Soft-hardfork: which requires miners to mine empty blocks with 0 reward,= and put the tx merkle tree in the legacy coinbase (e.g. https://github.com= /luke-jr/bips/blob/bip-mmhf/bip-mmhf.mediawiki ). This allows most hardfork= -ing changes including block size and inflation. In terms of block validity= this is a softfork. But with the definition I presented, soft-hardforks ar= e clearly hardforks for every practical purposes. 2. On-chain KYC, blacklist, account freezing: technically softforks, but al= l are very disruptive hardforks in terms of user experience. 3. Lightning network and side chains are not consensus rule changes, and th= ey could provide new features without any hardfork-ness. --_000_BLUPR15MB0051872C3E1FB74DF3FCA989B10A0BLUPR15MB0051namp_ Content-Type: text/html; charset="windows-1256" Content-Transfer-Encoding: quoted-printable
I'm not an ex= pert to refute this classification, but would love to see those points addr= essed by those in the know, without resorting to ad hominem, even though I = know it's really hard.

thx, gm
From: Johnson Lau via bitcoin= -dev
Sent: =FD4/= =FD5/=FD2017 6:28 AM
To: bitcoin-dev<= br> Subject: [bitc= oin-dev] A different approach to define and understand softforks and hardfo= rks

Softforks and hardforks are usually defined in terms of block validity= (BIP99): making valid blocks invalid is a softfork, making invalid blocks = valid is a hardfork, and SFs are usually considered as less disruptive as i= t is considered to be =93opt-in=94. However, as shown below this technical definition could be very misleading= . Here I=92m trying to redefine the terminology in terms of software upgrad= e necessity and difficulty.

Softforks are defined as consensus rule changes that non-up= graded software will be able to function exactly as usual, as if the rule c= hanges have never happened

Hardforks are defined as consensus rule changes that non-up= graded software will cease to function or be severely handicapped

SFs and HFs under this definitions is a continuum, which I = call it =93hardfork-ness=94. A pure softfork has no hardfork-ness.

*Mining node

Under this definitions, for miners, any trivial consensus r= ule changes is somewhat a hardfork, as miners can=92t reliably use non-upgr= aded software to create blocks. However, there is still 3 levels of =93hard= fork-ness=94, for example:

1. Those with lower hardfork-ness would be the SFs that min= ers do not need to upgrade their software at all. Instead, the minimum requ= irement is to setup a boarder node with latest rules to make sure they won= =92t mine on top of an invalid block. Examples include CSV and Segwit

2. Some SFs have higher hardfork-ness, for example BIP65 an= d BIP66. The minimum actions needed include setting up a boarder node and c= hange the block version. BIP34 has even higher hardfork-ness as more action= s are needed to follow the new consensus.

3. Anything else, ranging from simple HFs like BIP102 to co= mplete HFs like spoonnet, or soft-hardfork like forcenet, have the highest = hardfork-ness. In these cases, boarder nodes are completely useless. Miners= have to upgrade their servers in order to stay with the consensus.

*Non-mining full node

Similarly, in terms of non-mining full node, as the main fu= nction is to fully-validate all applicable rules on the network, any consen= sus change is a hardfork for this particular function. However, a technical= SF would have much lower hardfork-ness than a HF, as a border node is everything needed in a SF. Just consider a = company has some difficult-to-upgrade software that depends on Bitcoin Core= 0.8. Using a 0.13.1+ boarder node will make sure they will always foll= ow the latest rules. In case of a HF, they have no choice but to upgrade the backend system.

So we may use the costs of running a boarder node to furthe= r define the hardfork-ness of SFs, and it comes to the additional resources= needed:

1. Things like BIP34, 65, 66, and CSV involves trivial reso= urces use so they have lowest hardfork-ness.

2. Segwit is higher because of increased block size.

3. Extension block has very high hardfork-ness as people ma= y not have enough resources to run a boarder node.

* Fully validating wallets

In terms of the wallet function in full node, without consi= dering the issues of validation, the hardfork-ness could be ranked as below= :

1. BIP34, 65, 66, CSV, segwit all have no hardfork-ness for= wallets. Non-upgraded wallets will work exactly in the same way as before.= Users won=92t notice any change at all. (In some cases they may not see a = new tx until it has 1 confirmation, but this is a mild issue and 0-conf is unsafe anyway)

2. Extension block, as presented in my January post ( = https://lists.linuxfoundation.org/pipermail/bi= tcoin-dev/2017-January/013490.html ), has higher hardfork-ness, as users of legacy wallets may find it difficult to = receive payments from upgraded wallet. However, once they got paid, the use= r experience is same as before

3. Another extension block proposal ( https://github.com= /tothemoon-org/extension-blocks ) has very high hardfork-ness for = wallets, as legacy wallets will frequently and suddenly find that incoming and outgoing txs becoming invalid, and need to sign the= invalidated txs again, even no one is trying to double spend.

4. Hardfork rule changes have highest hardfork-ness for ful= l node wallets

I=92ll explain the issues with extension block in a separat= e post in details

* Real SPV wallet

The SPV wallets as proposed by Satoshi should have the abil= ity to fully validate the rules when needed, so they could be somehow seen = as fully validating wallets. So far, real SPV wallet is just vapourware.

* Fake SPV wallet, aka light wallet

All the so-called SPV wallets we have today are fake SPV ac= cording to whitepaper definition. Since they validate nothing, the hardfork= -ness profile is very different:

1. BIP34, 65, 66, CSV, segwit has no hardfork-ness for ligh= t wallets. Block size HF proposals (BIP10x) and Bitcoin Unlimited also have= no hardfork-ness (superficially, but not philosophically). Along the same = line, even an inflation hardfork has no hardfork-ness for light wallets.

2. Extension block has the same kind of hardfork-ness issue= as I mentioned.

3. HFs that deliberately breaks light wallets, such as spoo= nnet, is a complete hardfork.

While some people try to leverage weakness of light wallets= , the inability to validate any important rules like block size, double spe= nding, and inflation is a serious vulnerability.

=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D

Before I finish, I=92d also like to analyse some other inte= resting cases.

1. Soft-hardfork: which requires miners to mine empty block= s with 0 reward, and put the tx merkle tree in the legacy coinbase (e.g.&nb= sp;https://github.com/luke-jr/bips/blob/bip-mmhf/bip-mmhf.medi= awiki ). This allows most hardfork-ing changes including block size and inflation. = In terms of block validity this is a softfork. But with the definition I pr= esented, soft-hardforks are clearly hardforks for every practical purposes.=

2. On-chain KYC, blacklist, account freezing: technically s= oftforks, but all are very disruptive hardforks in terms of user experience= .

3. Lightning network and side chains are not consensus rule= changes, and they could provide new features without any hardfork-ness.


--_000_BLUPR15MB0051872C3E1FB74DF3FCA989B10A0BLUPR15MB0051namp_--