Return-Path: <jl2012@xbt.hk> Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id D13FCB6B for <bitcoin-dev@lists.linuxfoundation.org>; Wed, 14 Dec 2016 15:45:50 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from sender163-mail.zoho.com (sender163-mail.zoho.com [74.201.84.163]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id A956BD5 for <bitcoin-dev@lists.linuxfoundation.org>; Wed, 14 Dec 2016 15:45:49 +0000 (UTC) Received: from [10.8.8.2] (119246245241.ctinets.com [119.246.245.241]) by mx.zohomail.com with SMTPS id 1481730341136520.0080372076295; Wed, 14 Dec 2016 07:45:41 -0800 (PST) Content-Type: multipart/alternative; boundary="Apple-Mail=_FB5945C7-6573-4111-B774-636C74A57867" Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) From: Johnson Lau <jl2012@xbt.hk> In-Reply-To: <CAE-z3OWEkwuxu+LiT1VsWDBnnoi5pQHDDiiKpi7i8oDOtPrB4A@mail.gmail.com> Date: Wed, 14 Dec 2016 23:45:37 +0800 Message-Id: <DB6F4DDF-1424-4FC4-84B3-5D16963E8589@xbt.hk> References: <FB8593E6-3CD7-46D5-8FC8-A73A0EF1AE9A@xbt.hk> <CAE-z3OUpbUA2yviYoZouuZ0fp1WbbVdehWwNCd3juNsN-u9csA@mail.gmail.com> <201612102141.58206.luke@dashjr.org> <CAE-z3OX5QW0jy_ZvU7=onaVoynJrsuyeCdc=q7wj=d5O4XLO7Q@mail.gmail.com> <02E75CD7-24A8-4E2F-9466-B5497D0B77F8@xbt.hk> <CAE-z3OWEkwuxu+LiT1VsWDBnnoi5pQHDDiiKpi7i8oDOtPrB4A@mail.gmail.com> To: Tier Nolan <tier.nolan@gmail.com> X-Mailer: Apple Mail (2.3124) X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> Subject: Re: [bitcoin-dev] Forcenet: an experimental network with a new header format X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org> List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe> List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/> List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org> List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help> List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>, <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe> X-List-Received-Date: Wed, 14 Dec 2016 15:45:50 -0000 --Apple-Mail=_FB5945C7-6573-4111-B774-636C74A57867 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 I think that=E2=80=99s too much tech debt just for softforkability. The better way would be making the sum tree as an independent tree with = a separate commitment, and define a special type of softfork (e.g. a = special BIP9 bit). When the softfork is activated, the legacy full node = will stop validating the sum tree. This doesn=E2=80=99t really degrade = the security by more than a normal softfork, as the legacy full node = would still validate the total weight and nSigOp based on its own rules. = The only purpose of the sum tree is to help SPV nodes to validate. This = way we could even completely redefine the structure and data committed = in the sum tree. I=E2=80=99d like to combine the size weight and sigOp weight, but not = sure if we could. The current size weight limit is 4,000,000 and sigop = limit is 80,000. It=E2=80=99s 50:1. If we maintain this ratio, and = define weight =3D n * (total size + 3 * base size) + sigop , with n =3D 50 a block may have millions of sigops which is totally unacceptable. On the other hand, if we make n too low, we may allow either too few = sigop, or a too big block size. Signature aggregation will make this a bigger problem as one signature = may spend thousands of sigop > On 14 Dec 2016, at 20:52, Tier Nolan <tier.nolan@gmail.com> wrote: >=20 >=20 >=20 > On Wed, Dec 14, 2016 at 10:55 AM, Johnson Lau <jl2012@xbt.hk = <mailto:jl2012@xbt.hk>> wrote: > In a sum tree, however, since the nSigOp is implied, any redefinition = requires either a hardfork or a new sum tree (and the original sum tree = becomes a placebo for old nodes. So every softfork of this type creates = a new tree) >=20 > That's a good point. > =20 > The only way to fix this is to explicitly commit to the weight and = nSigOp, and the committed value must be equal to or larger than the real = value. Only in this way we could redefine it with softfork. However, = that means each tx will have an overhead of 16 bytes (if two int64 are = used) >=20 > The weight and sigop count could be transmitted as variable length = integers. That would be around 2 bytes for the sigops and 3 bytes for = the weight, per transaction. >=20 > It would mean that the block format would have to include the raw = transaction, "extra"/tree information and witness data for each = transaction. >=20 > On an unrelated note, the two costs could be combined into a unified = cost. For example, a sigop could have equal cost to 250 bytes. This = would make it easier for miners to decide what to charge. >=20 > On the other hand, CPU cost and storage/network costs are not = completely interchangeable. >=20 > Is there anything that would need to be summed fees, raw tx size, = weight and sigops that the greater or equal rule wouldn't cover? >=20 > On 12 Dec 2016, at 00:40, Tier Nolan via bitcoin-dev = <bitcoin-dev@lists.linuxfoundation.org = <mailto:bitcoin-dev@lists.linuxfoundation.org>> wrote: >>=20 >> On Sat, Dec 10, 2016 at 9:41 PM, Luke Dashjr <luke@dashjr.org = <mailto:luke@dashjr.org>> wrote: >> On Saturday, December 10, 2016 9:29:09 PM Tier Nolan via bitcoin-dev = wrote: >> > Any new merkle algorithm should use a sum tree for partial = validation and >> > fraud proofs. >>=20 >> PR welcome. >>=20 >> Fair enough. It is pretty basic. >>=20 >> https://github.com/luke-jr/bips/pull/2 = <https://github.com/luke-jr/bips/pull/2> >>=20 >> It sums up sigops, block size, block cost (that is "weight" right?) = and fees. >> _______________________________________________ >> bitcoin-dev mailing list >> bitcoin-dev@lists.linuxfoundation.org = <mailto:bitcoin-dev@lists.linuxfoundation.org> >> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev = <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev> >=20 >=20 --Apple-Mail=_FB5945C7-6573-4111-B774-636C74A57867 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html = charset=3Dutf-8"></head><body style=3D"word-wrap: break-word; = -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" = class=3D"">I think that=E2=80=99s too much tech debt just for = softforkability.<div class=3D""><br class=3D""></div><div class=3D"">The = better way would be making the sum tree as an independent tree with a = separate commitment, and define a special type of softfork (e.g. a = special BIP9 bit). When the softfork is activated, the legacy full node = will stop validating the sum tree. This doesn=E2=80=99t really degrade = the security by more than a normal softfork, as the legacy full node = would still validate the total weight and nSigOp based on its own rules. = The only purpose of the sum tree is to help SPV nodes to validate. This = way we could even completely redefine the structure and data committed = in the sum tree.</div><div class=3D""><br class=3D""></div><div = class=3D"">I=E2=80=99d like to combine the size weight and sigOp weight, = but not sure if we could. The current size weight limit is 4,000,000 and = sigop limit is 80,000. It=E2=80=99s 50:1. If we maintain this ratio, and = define</div><div class=3D"">weight =3D n * (total size + 3 * base = size) + sigop , with n =3D 50</div><div class=3D"">a block may have = millions of sigops which is totally unacceptable.</div><div class=3D""><br= class=3D""></div><div class=3D"">On the other hand, if we make n too = low, we may allow either too few sigop, or a too big block = size.</div><div class=3D""><br class=3D""></div><div class=3D"">Signature = aggregation will make this a bigger problem as one signature may spend = thousands of sigop</div><div class=3D""><br class=3D""></div><div = class=3D""><br class=3D""></div><div class=3D""><br = class=3D""><div><blockquote type=3D"cite" class=3D""><div class=3D"">On = 14 Dec 2016, at 20:52, Tier Nolan <<a = href=3D"mailto:tier.nolan@gmail.com" = class=3D"">tier.nolan@gmail.com</a>> wrote:</div><br = class=3D"Apple-interchange-newline"><div class=3D""><div dir=3D"ltr" = class=3D""><br class=3D""><div class=3D"gmail_extra"><br class=3D""><div = class=3D"gmail_quote">On Wed, Dec 14, 2016 at 10:55 AM, Johnson Lau = <span dir=3D"ltr" class=3D""><<a href=3D"mailto:jl2012@xbt.hk" = target=3D"_blank" class=3D"">jl2012@xbt.hk</a>></span> wrote:<br = class=3D""><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 = .8ex;border-left:1px #ccc solid;padding-left:1ex"><div = style=3D"word-wrap:break-word" class=3D""><div class=3D"">In a sum tree, = however, since the nSigOp is implied, any redefinition requires either a = hardfork or a new sum tree (and the original sum tree becomes a placebo = for old nodes. So every softfork of this type creates a new = tree)</div></div></blockquote><div class=3D""><br class=3D""></div><div = class=3D"">That's a good point.<br class=3D""></div><div = class=3D""> </div><blockquote class=3D"gmail_quote" style=3D"margin:0= 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div = style=3D"word-wrap:break-word" class=3D""><div class=3D"">The only way = to fix this is to explicitly commit to the weight and nSigOp, and the = committed value must be equal to or larger than the real value. Only in = this way we could redefine it with softfork. However, that means each tx = will have an overhead of 16 bytes (if two int64 are = used)</div></div></blockquote><div class=3D""><br class=3D""></div><div = style=3D"word-wrap:break-word" class=3D"">The weight and sigop count = could be transmitted as variable length integers. That would be = around 2 bytes for the sigops and 3 bytes for the weight, per = transaction.<br class=3D""><br class=3D""></div><div = style=3D"word-wrap:break-word" class=3D"">It would mean that the block = format would have to include the raw transaction, "extra"/tree = information and witness data for each transaction.<br class=3D""><br = class=3D""></div>On an unrelated note, the two costs could be combined = into a unified cost. For example, a sigop could have equal cost to = 250 bytes. This would make it easier for miners to decide what to = charge.<br class=3D""><br class=3D""></div><div class=3D"gmail_quote">On = the other hand, CPU cost and storage/network costs are not completely = interchangeable.<br class=3D""><br class=3D"">Is there anything that = would need to be summed fees, raw tx size, weight and sigops that the = greater or equal rule wouldn't cover?<br class=3D""></div><div = class=3D"gmail_quote"><div style=3D"word-wrap:break-word" class=3D""><br = class=3D""></div>On 12 Dec 2016, at 00:40, Tier Nolan via bitcoin-dev = <<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" = target=3D"_blank" class=3D"">bitcoin-dev@lists.<wbr = class=3D"">linuxfoundation.org</a>> wrote:<div = style=3D"word-wrap:break-word" class=3D""><div class=3D""><blockquote = type=3D"cite" class=3D""><div class=3D""><div class=3D"h5"><br = class=3D"m_6699621031753262314Apple-interchange-newline"></div></div><div = class=3D""><div class=3D""><div class=3D"h5"><div dir=3D"ltr" = class=3D""><div class=3D"gmail_extra"><div class=3D"gmail_quote">On Sat, = Dec 10, 2016 at 9:41 PM, Luke Dashjr <span dir=3D"ltr" class=3D""><<a = href=3D"mailto:luke@dashjr.org" target=3D"_blank" = class=3D"">luke@dashjr.org</a>></span> wrote:<br class=3D""><blockquote= class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px = solid rgb(204,204,204);padding-left:1ex"><span = class=3D"m_6699621031753262314gmail-">On Saturday, December 10, 2016 = 9:29:09 PM Tier Nolan via bitcoin-dev wrote:<br class=3D""> > Any new merkle algorithm should use a sum tree for partial = validation and<br class=3D""> > fraud proofs.<br class=3D""> <br class=3D""> </span>PR welcome.<br class=3D""></blockquote><div class=3D""><br = class=3D""></div><div class=3D"">Fair enough. It is pretty = basic.<br class=3D""><br class=3D""><a = href=3D"https://github.com/luke-jr/bips/pull/2" target=3D"_blank" = class=3D"">https://github.com/luke-jr/<wbr class=3D"">bips/pull/2</a><br = class=3D""><br class=3D""></div><div class=3D"">It sums up sigops, block = size, block cost (that is "weight" right?) and fees.<br = class=3D""></div></div></div></div></div></div><span class=3D""> ______________________________<wbr class=3D"">_________________<br = class=3D"">bitcoin-dev mailing list<br class=3D""><a = href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_blank" = class=3D"">bitcoin-dev@lists.<wbr class=3D"">linuxfoundation.org</a><br = class=3D""><a = href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" = target=3D"_blank" class=3D"">https://lists.linuxfoundation.<wbr = class=3D"">org/mailman/listinfo/bitcoin-<wbr class=3D"">dev</a><br = class=3D""></span></div></blockquote></div><br class=3D""></div></div><br = class=3D""></div></div> </div></blockquote></div><br class=3D""></div></body></html>= --Apple-Mail=_FB5945C7-6573-4111-B774-636C74A57867--