From: Johnson Lau <jl2012@xbt.hk>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Mac OS X Mail 10.2 \(3259\))
Date: Tue, 7 Feb 2017 02:06:22 +0800
References: <15A0A5F0-726F-4E8F-9F16-F33D5A30C5F4@xbt.hk>
To: Johnson Lau <jl2012@xbt.hk>,
	bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org>
In-Reply-To: <15A0A5F0-726F-4E8F-9F16-F33D5A30C5F4@xbt.hk>
Message-Id: <D5B37586-3A96-4BD4-97A8-8A9335BFA698@xbt.hk>
Subject: Re: [bitcoin-dev] Spoonnet: another experimental hardfork
Precedence: list

I fixed a flaw in my original design. It allowed a miner to create a =
fake transaction to cheat light wallets.

Also, with a second thought, I removed the backward compatibility with =
light wallets. Everyone, light or full, should opt-in to a hardfork.

The extended header is amended as follow

* There must be exactly 2 witness items in coinbase witness (Header1; =
Header2)
**Header1 is the miner dedicated space. Nothing here has no consensus =
meaning and is used for extranonce and merge mining. However, if the =
size is larger than 14 bytes, the first 6 bytes must be 0. For a legacy =
light node, this will look like a version 0 transaction with zero input =
and output. [No need for 14 bytes or below: even with the most broken =
design, a wallet can=E2=80=99t misinterpret data with this small size as =
a valid tx]
**Header2 is the secondary header. It must be 70-128 bytes. The first 4 =
bytes must be little-endian encoded number of transactions (minimum 1). =
The next 2 bytes must be 0. The next 32 bytes must be the transaction =
Merkle root. The next 32 bytes must be the witness Merkle root (to be =
defined later). The rest, if any, has no consensus meaning. However, =
miners MUST NOT use this space of non-bitcoin purpose [the additional =
space allows non-censensus enforced data to be included, easily =
accessible to light wallets; the structure of the first 6 bytes make it =
looks like an empty tx]
* The main header commitment is H(H(Header1)|H(Header2))  H() =3D =
dSHA256() [legacy light wallets are broken and they must upgrade to join =
the new network]

The implementation is updated accordingly


> On 6 Feb 2017, at 20:39, Johnson Lau via bitcoin-dev =
<bitcoin-dev@lists.linuxfoundation.org> wrote:
>=20
> Finally got some time over the Chinese New Year holiday to code and =
write this up. This is not the same as my previous forcenet ( =
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/01347=
2.html ). It is much simpler. Trying to activate it on testnet will get =
you banned. Trying to activate it on mainnet before consensus is reached =
will make you lose money.
>=20
> This proposal includes the following features:
>=20
> 1. A fixed starting time. Not dependent on miner signalling. However, =
it requires at least 51% of miners to actually build the new block =
format in order to get activated.
>=20
> 2. It has no mechanism to prevent a split. If 49% of miners insist on =
the original chain, they could keep going. Split prevention is a social =
problem, not a technical one.
>=20
> 3. It is compatible with existing Stratum mining protocol. Only pool =
software upgrade is needed
>=20
> 4. A new extended and flexible header is located at the witness field =
of the coinbase transaction
>=20
> 5. It is backward compatible with existing light wallets
>=20
> 6. Dedicated space for miners to put anything they want, which bitcoin =
users could completely ignore. Merge-mining friendly.
>=20
> 7. Small header space for miners to include non-consensus enforced =
bitcoin related data, useful for fee estimation etc.
>=20
> 8. A new transaction weight formula to encourage responsible use of =
UTXO
>=20
> 9. A linear growth of actual block size until certain limit
>=20
> 10. Sighash O(n^2) protection for legacy (non-segwit) outputs
>=20
> 11. Optional anti-transaction replay
>=20
> 12. A new optional coinbase tx format that allows additional inputs, =
including spending of immature previous coinbase outputs
>=20
>=20
>=20
> Specification [Rationales]:
>=20
>=20
> Activation:
>=20
> * A "hardfork signalling block" is a block with the sign bit of header =
nVersion is set [Clearly invalid for old nodes; easy opt-out for light =
wallets]
>=20
> * If the median-time-past of the past 11 blocks is smaller than the =
HardForkTime (exact time to be determined), a hardfork signalling block =
is invalid.
>=20
> * Child of a hardfork signalling block MUST also be a hardfork =
signalling block
>=20
> * Initial hardfork signalling is optional, even if the HardForkTime =
has past [requires at least 51% of miners to actually build the new =
block format]
>=20
> * HardForkTime is determined by a broad consensus of the Bitcoin =
community. This is the only way to prevent a split.
>=20
>=20
> Extended header:
>=20
> * Main header refers to the original 80 bytes bitcoin block header
>=20
> * A hardfork signalling block MUST have a additional extended header
>=20
> * The extended header is placed at the witness field of the coinbase =
transaction [There are 2 major advantages: 1. coinbase witness is =
otherwise useless; 2. Significantly simply the implementation with its =
stack structure]
>=20
> * There must be exactly 3 witness items (Header1; Header2 ; Header3)
> **Header1 must be exactly 32 bytes of the original transaction hash =
Merkle root.
> **Header2 is the secondary header. It must be 36-80 bytes. The first 4 =
bytes must be little-endian encoded number of transactions (minimum 1). =
The next 32 bytes must be the witness Merkle root (to be defined later). =
The rest, if any, has no consensus meaning. However, miners MUST NOT use =
this space of non-bitcoin purpose [the additional space allows =
non-censensus enforced data to be included, easily accessible to light =
wallets]
> **Header3 is the miner dedicated space. It must not be larger than 252 =
bytes. Anything put here has no consensus meaning [space for merge =
mining; non-full nodes could completely ignore data in this space; 252 =
is the maximum size allowed for signal byte CompactSize]
>=20
> * The main header commitment is H(Header1|H(H(Header2)|H(Header3)))  =
H() =3D dSHA256() [The hardfork is transparent to light wallets, except =
one more 32-byte hash is needed to connect a transaction to the root]
>=20
> * To place the ext header, segwit becomes mandatory after hardfork
>=20
>=20
> A =E2=80=9Cbackdoor=E2=80=9D softfork the relax the size limit of =
Header 2 and Header 3:
>=20
> * A special BIP9 softfork is defined with bit-15. If this softfork is =
activated, full nodes will not enforce the size limit for Header 2 and =
Header 3. [To allow header expansion without a hardfork. Avoid miner =
abuse while providing flexibility. Expansion might be needed for new =
commitments like fraud proof commitments]
>=20
>=20
> Anti-tx-replay:
>=20
> * Hardfork network version bit is 0x02000000. A tx is invalid if the =
highest nVersion byte is not zero, and the network version bit is not =
set.
>=20
> * Masked tx version is nVersion with the highest byte masked. If =
masked version is 3 or above, sighash for OP_CHECKSIG alike is =
calculated using BIP143, except 0x02000000 is added to the nHashType =
(the nHashType in signature is still a 1-byte value) [ensure a clean =
split of signatures; optionally fix the O(n^2) problem]
>=20
> * Pre-hardfork policy change: nVersion is determined by the masked tx =
version for policy purpose. Setting of Pre-hardfork network version bit =
0x01000000 is allowed.
>=20
> * Details: =
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/01347=
3.html
>=20
>=20
> Sighash limitation:
>=20
> * Sighash impact is estimated by =E2=80=9CLoose estimation=E2=80=9D in =
https://github.com/jl2012/bips/blob/065ea7429035d43ff90965f42b086fb7e15172=
91/bip-sighash.mediawiki
>=20
> * Only txs with masked version below 3 are counted. [because they are =
fixed by the BIP-143 like signature]
>=20
> * Each SigHashSize is defined as 1 tx weight (defined later).
>=20
> * SIGHASH_SCALE_FACTOR is 90 (see the BIP above)
>=20
>=20
> New tx weight definition:
>=20
> * Weight of a transaction is the maximum of the 4 following metrics:
>=20
> ** The total serialised size * 2 * SIGHASH_SCALE_FACTOR  (size defined =
by the witness tx format in BIP144)
>=20
> ** The adjusted size =3D (Transaction weight by BIP141 - (number of =
inputs - number of non-OP_RETURN outputs) * 41) * SIGHASH_SCALE_FACTOR
>=20
> ** nSigOps * 50 * SIGHASH_SCALE_FACTOR. All SigOps are equal (no =
witness scaling). For non-segwit txs, the sigops in output scriptPubKey =
are not counted, while the sigops in input scriptPubKey are counted.
>=20
> ** SigHashSize defined in the last section
>=20
> Translating to new metric, the current BIP141 limit is 360,000,000. =
This is equivalent to 360MB of sighashing, 2MB of serialised size, 4MB =
of adjusted size, or 80000 nSigOp.
>=20
> See rationales in this post: =
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2017-January/01347=
2.html
>=20
>=20
> Block weight growing by time:
>=20
> * Numbers for example only. Exact number to be determined.
>=20
> * Block weight at HardForkTime is (5,000,000 * SIGHASH_SCALE_FACTOR)
>=20
> * By every 16 seconds growth of the median-time-past, the weight is =
increased by (1 * SIGHASH_SCALE_FACTOR)
>=20
> * The growth stops at (16,000,000 * SIGHASH_SCALE_FACTOR)
>=20
> * The growth does not dependent on the actual hardfork time. It=E2=80=99=
s only based on median-time-past [using median-time-past so miners have =
no incentive to use a fake timestamp]
>=20
> * The limit for serialized size is 2.5 to 8MB in about 8 years. =
[again, numbers for example only]
>=20
>=20
> New coinbase transaction format:
>=20
> * Existing coinbase format is allowed, except the new extended header =
in the coinbase witness. No OP_RETURN witness commitment is needed.
>=20
> * A new coinbase format is defined. The tx may have 1 or more inputs. =
The outpoint of the first input MUST have an n value of 0xffffffff, and =
use the previous block hash as the outpoint hash [This allows paying to =
the child of a particular block by signing the block hash]
>=20
> * ScriptSig of the first (coinbase) input is not executed. The size =
limit increased from 100 to 252 (same for old coinbase format)
>=20
> * Additional inputs MUST provide a valid scriptSig and/or witness for =
spending
>=20
> * Additional inputs may come from premature previous coinbase outputs =
[this allows previous blocks paying subsequent blocks to encourage =
confirmations]
>=20
>=20
> Witness merkle root:
>=20
> * If the coinbase is in old format, the witness merkle root is same as =
BIP141 by setting the witness hash of the coinbase tx as 0 (without the =
32 byte witness reserved value)
>=20
> * If the coinbase is in new format, the witness hash of the coinbase =
tx is calculated by first removing the extended header
>=20
> * The witness merkle root is put in the extended header 2, not as an =
OP_RETURN output in coinbase tx.
>=20
> * The witness merkle root becomes mandatory. (It was optional in =
BIP141)
>=20
>=20
> Other consensus changes:
>=20
> * BIP9 will ignore the sign bit. [Setting the sign bit now is invalid =
so this has no real consensus impact]
>=20
> =3D=3D=3D=3D=3D=3D=3D=3D
>=20
> An experimental implementation of the above spec could be found at =
https://github.com/jl2012/bitcoin/tree/spoonnet1
>=20
> Not the same as my previous effort on the =E2=80=9Cforcenet=E2=80=9D, =
the =E2=80=9Cspoonnet=E2=80=9D is a full hardfork that will get you =
banned on the existing network.
>=20
> Haven=E2=80=99t got the time to test the codes yet, not independently =
reviewed. But it passes all existing tests in Bitcoin Core. No one =
should use this in production, but I *think* it works fine on testnet =
like a normal bitcoind (as long as it is not activated)
>=20
> Things not implemented yet:
>=20
> 1. Automated testing
>=20
> 2. Post-hardfork support for old light wallets
>=20
> 3. Wallet support, especially anti-tx-replay
>=20
> 4. New p2p message to transmit secondary header (lower priority)
>=20
> 5. Full mining and mempool support (not my priority)
>=20
> =3D=3D=3D=3D=3D=3D=3D=3D
>=20
> Potential second stage change:
>=20
> Relative to the actual activation time, there could be a second stage =
with more drastic changes to fix one or both of the following problems:
>=20
> 1. SHA256 shortcut like ASICBoost. All fixes to ASICBoost are not very =
elegant. But the question is, is it acceptable to have bitcoin-specific =
patent in the consensus protocol? Still, I believe the best way to solve =
this problem is the patent holder(s) to kindly somehow release the right =
to the community.=20
>=20
> 2. Providing more nonce space in the 80-byte main header. However, =
this depends on ASICBoost being a free technology.
>=20
> 3. Block withholding attack. There are pros and cons, but I generally =
agree with the analysis by Peter Todd at =
https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/0120=
46.html . One point he didn=E2=80=99t mention is that only small really =
needs pool mining, for the purpose of variance reduction. Big miners =
using pools are just lazy, and they work well without pool. That means =
only big solo miners are able to attack pools (i.e. small miners), while =
pools cannot do any counterattack. This obviously shows why fixing this =
is pro-small-miners. Also, with same hash rate, block withholding attack =
is more effective against a smaller pool than a big pool.
>=20
> All of these changes involve a header change and require light wallets =
to upgrade. They also require firmware upgrade for all existing miners =
(change 2 doesn=E2=80=99t). I think these shouldn=E2=80=99t happen at =
least 2 years after the actual activation of the hardfork so people will =
have enough time to upgrade.
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev