Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id B84567D for ; Fri, 20 May 2016 09:47:30 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from s47.web-hosting.com (s47.web-hosting.com [199.188.200.16]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id EAA381FC for ; Fri, 20 May 2016 09:47:29 +0000 (UTC) Received: from neubau-gw.kalkbreite.net ([62.12.170.156]:45468 helo=[172.27.201.177]) by server47.web-hosting.com with esmtpsa (TLSv1:DHE-RSA-AES256-SHA:256) (Exim 4.86_1) (envelope-from ) id 1b3h1k-0016FW-GB; Fri, 20 May 2016 05:47:28 -0400 Content-Type: multipart/signed; boundary="Apple-Mail=_009F9BD2-07A6-4A0F-83E3-55C62937D140"; protocol="application/pgp-signature"; micalg=pgp-sha512 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) X-Pgp-Agent: GPGMail 2.6b2 From: Johnson Lau In-Reply-To: <20160517132311.GA21656@fedora-21-dvm> Date: Fri, 20 May 2016 11:46:32 +0200 Message-Id: <68A4772D-D423-45F9-ADB7-95BEB3E66F43@xbt.hk> References: <20160517132311.GA21656@fedora-21-dvm> To: Peter Todd , bitcoin-dev X-Mailer: Apple Mail (2.3124) X-AntiAbuse: This header was added to track abuse, please include it with any abuse report X-AntiAbuse: Primary Hostname - server47.web-hosting.com X-AntiAbuse: Original Domain - lists.linuxfoundation.org X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12] X-AntiAbuse: Sender Address Domain - xbt.hk X-Get-Message-Sender-Via: server47.web-hosting.com: authenticated_id: jl2012@xbt.hk X-Authenticated-Sender: server47.web-hosting.com: jl2012@xbt.hk X-Source: X-Source-Args: X-Source-Dir: X-From-Rewrite: unmodified, already matched X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Subject: Re: [bitcoin-dev] Making UTXO Set Growth Irrelevant With Low-Latency Delayed TXO Commitments X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 May 2016 09:47:30 -0000 --Apple-Mail=_009F9BD2-07A6-4A0F-83E3-55C62937D140 Content-Type: multipart/alternative; boundary="Apple-Mail=_8871C9B6-9DFD-42D8-B917-373CE934C7BA" --Apple-Mail=_8871C9B6-9DFD-42D8-B917-373CE934C7BA Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii How is this compared to my earlier proposal: = https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-December/0119= 52.html = ? In my proposal, only the (pruned) UTXO set, and 32 bytes per archived = block, are required for mining. But it is probably more difficult for = people to spend an archived output. They need to know the status of = other archived outputs from the same block. A full re-scan of the = blockchain may be needed to generate the proof but this could be done by = a third party archival node. >=20 >=20 >=20 > ## Implementation >=20 > Our proposed high-performance/low-latency delayed commitment full-node > implementation needs to store the following data: >=20 > 1) UTXO set >=20 > Low-latency K:V map of txouts definitely known to be unspent. = Similar to > existing UTXO implementation, but with the key difference that old, > unspent, outputs may be pruned from the UTXO set. >=20 >=20 > 2) STXO set >=20 > Low-latency set of transaction outputs known to have been spent by > transactions after the most recent TXO commitment, but created = prior to the > TXO commitment. >=20 >=20 > 3) TXO journal >=20 > FIFO of outputs that need to be marked as spent in the TXO MMR. = Appends > must be low-latency; removals can be high-latency. >=20 >=20 > 4) TXO MMR list >=20 > Prunable, ordered list of TXO MMR's, mainly the highest pending = commitment, > backed by a reference counted, cryptographically hashed object = store > indexed by digest (similar to how git repos work). High-latency ok. = We'll > cover this in more in detail later. >=20 >=20 --Apple-Mail=_8871C9B6-9DFD-42D8-B917-373CE934C7BA Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=us-ascii How is this compared to my earlier proposal: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-De= cember/011952.html ?

In my proposal, only the (pruned) UTXO set, and 32 bytes per = archived block, are required for mining. But it is probably more = difficult for people to spend an archived output. They need to know the = status of other archived outputs from the same block. A full re-scan of = the blockchain may be needed to generate the proof but this could be = done by a third party archival node.




## Implementation

Our proposed = high-performance/low-latency delayed commitment full-node
implementation needs to store the following data:

1) UTXO set

=    Low-latency K:V map of txouts definitely known to be = unspent. Similar to
   existing UTXO = implementation, but with the key difference that old,
=    unspent, outputs may be pruned from the UTXO set.


2) STXO set

   Low-latency set of transaction outputs = known to have been spent by
=    transactions after the most recent TXO commitment, but = created prior to the
   TXO commitment.


3) TXO journal

   FIFO of outputs that need to be marked as = spent in the TXO MMR. Appends
   must be = low-latency; removals can be high-latency.


4) TXO MMR list

=    Prunable, ordered list of TXO MMR's, mainly the = highest pending commitment,
   backed by a = reference counted, cryptographically hashed object store
=    indexed by digest (similar to how git repos work). = High-latency ok. We'll
   cover this in = more in detail later.



= --Apple-Mail=_8871C9B6-9DFD-42D8-B917-373CE934C7BA-- --Apple-Mail=_009F9BD2-07A6-4A0F-83E3-55C62937D140 Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=signature.asc Content-Type: application/pgp-signature; name=signature.asc Content-Description: Message signed with OpenPGP using GPGMail -----BEGIN PGP SIGNATURE----- Comment: GPGTools - https://gpgtools.org iQGcBAEBCgAGBQJXPt0tAAoJEO6eVSA0viTSwE4MAK7xbzaLv577x9wdYWZZAHDs GbAq95HISuQT5x0z+s+mKPqV2Y60YztW6qEo2tzkPKMQdwKYyNRmAPjCYDE5jsfz 0QQFcbB8AKp2W2LT43Lf9++FaqX6BL3SJY14rE4qyCd/hqHIkblzdbziOrRz507G JRolGcP6mgylizjRn+Bsb/r1uB317xgNyjCX9nzS+VtOCSIn+fxoUpPXpe9/tssY egeoOTZFBOOJwy4s79BUBTCRuBsqNJvaIc6SMN9ZpGfrzp+F2HQLOEmwj6r1aNQj FslHzp0otcW7eifyVQo2bBE7RuQ5kyqYwhIz8DfeW95w4Qb+9W4Uwwa4ncTBppqj x6yVNjaGAYL/Gs2egPbX41uIDsvQ0ZpO08qjLkj6ZSC70TbRZ1SiJTp+hlfyLgWb WMrTSVuiLY3BFVU4CrQ6+ihyklkjZDBHBRpefEPt1nz/3teO0HzD1shkRQgKdcdu STSTIi5As7qdB9duPhIF7s5WmzrglAGcOa4XuOqlMA== =PKSg -----END PGP SIGNATURE----- --Apple-Mail=_009F9BD2-07A6-4A0F-83E3-55C62937D140--