Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 47E9B197F for ; Mon, 5 Oct 2015 18:51:25 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-qk0-f176.google.com (mail-qk0-f176.google.com [209.85.220.176]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 0D05C159 for ; Mon, 5 Oct 2015 18:51:24 +0000 (UTC) Received: by qkcf65 with SMTP id f65so72920750qkc.3 for ; Mon, 05 Oct 2015 11:51:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=80a/anTEykK1i/+UsYXeBdD1DiMe9e/wnN1RKbO0/dQ=; b=O9anqIHipWXH7s0PXTiKiA2NGlt/HwwHtJJfBLX1NQtBFxBRod/qNPMhuuhj9BeAOA wWhKaqUH1EXXF26SgzSzq6KIosL6KTsqq4yvv/o5QIry11yqV4A2xlVT3xtNqWWfPaEY 2UMncphNjk3ggHMkcUEdWauJxsSfR+yY3IZv4s4EypbuEgF3V2UMssuRi0MPpT34ysQU 2zxAKOUMp5+llF7B98UN4NjhZF3g+jGJjXJzIm7AoT0c9nBTy6vvrUUpzrvzljLYMado FVFpPs5zbSaf3UACs0/ItpG/sCGSWD9xDo5DMJlbuB7GgynIzykRH+91zK2cj09p2WKY oikA== MIME-Version: 1.0 X-Received: by 10.55.221.142 with SMTP id u14mr41120345qku.78.1444071083136; Mon, 05 Oct 2015 11:51:23 -0700 (PDT) Received: by 10.55.22.69 with HTTP; Mon, 5 Oct 2015 11:51:23 -0700 (PDT) In-Reply-To: References: <55D77A7F.40402@mattcorallo.com> Date: Mon, 5 Oct 2015 11:51:23 -0700 Message-ID: From: Danny Thorpe To: Alex Morcos Content-Type: multipart/alternative; boundary=001a1147a3b0f0debc05215fff92 X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev Subject: Re: [bitcoin-dev] Proposed new policy for transactions that depend on other unconfirmed transactions X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Development Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Oct 2015 18:51:25 -0000 --001a1147a3b0f0debc05215fff92 Content-Type: text/plain; charset=UTF-8 What does "package" mean here? When you say 25 txs, does that mean maximum linked chain depth, or total number of dependent transactions regardless of chain depth? Thanks, -Danny On Mon, Oct 5, 2015 at 11:45 AM, Alex Morcos via bitcoin-dev < bitcoin-dev@lists.linuxfoundation.org> wrote: > I'd like to propose updates to the new policy limits on unconfirmed > transaction chains. > > The existing limits in master and scheduled for release in 0.12 are: > Ancestor packages = 100 txs and 900kb total size > Descendant packages = 1000 txs and 2500kb total size > > Before 0.12 is released I would like to propose a significant reduction in > these limits. In the course of analyzing algorithms for mempool limiting, > it became clear that large packages of unconfirmed transactions were the > primary vector for mempool clogging or relay fee boosting attacks. Feedback > from the initial proposed limits was that they were too generous anyway. > > The proposed new limits are: > Ancestor packages = 25 txs and 100kb total size > Descendant packages = 25 txs and 100kb total size > > Based on historical transaction data, the most restrictive of these limits > is the 25 transaction count on descendant packages. Over the period of > April and May of this year (before stress tests), 5.8% of transactions > would have violated this limit alone. Applying all the limits together > would have affected 6.1% of transactions. > > Please keep in mind these are policy limits that affect transactions which > depend on other unconfirmed transactions only. They are not a change to > consensus rules and do not affect how many chained txs a valid block may > contain. Furthermore, any transaction that was unable to be relayed due to > these limits need only wait for some of its unconfirmed ancestors to be > included in a block and then it could be successfully broadcast. This is > unlikely to affect the total time from creation to inclusion in a block. > Finally, these limits are command line arguments that can easily be changed > on an individual node basis in Bitcoin Core. > > Please give your feedback if you know of legitimate use cases that would > be hindered by these limits. > > Thanks, > Alex > > On Mon, Sep 21, 2015 at 11:02 AM, Alex Morcos wrote: > >> Thanks for everyone's review. These policy changes have been merged in >> to master in 6654 , which >> just implements these limits and no mempool limiting yet. The default >> ancestor package size limit is 900kb not 1MB. >> >> Yes I think these limits are generous, but they were designed to be as >> generous as was computationally feasible so they were unobjectionable >> (since the existing policy was no limits). This does not preclude future >> changes to policy that would reduce these limits. >> >> >> >> >> >> On Fri, Aug 21, 2015 at 3:52 PM, Danny Thorpe >> wrote: >> >>> The limits Alex proposed are generous (bordering on obscene!), but >>> dropping that down to allowing only two levels of chained unconfirmed >>> transactions is too tight. >>> >>> Use case: Brokered asset transfers may require sets of transactions with >>> a dependency tree depth of 3 to be published together. ( N seller txs, 1 >>> broker bridge tx, M buyer txs ) >>> >>> If the originally proposed depth limit of 100 does not provide a >>> sufficient cap on memory consumption or loop/recursion depth, a depth limit >>> of 10 would provide plenty of headroom for this 3 level use case and >>> similar patterns. >>> >>> -Danny >>> >>> On Fri, Aug 21, 2015 at 12:22 PM, Matt Corallo via bitcoin-dev < >>> bitcoin-dev@lists.linuxfoundation.org> wrote: >>> >>>> I dont see any problem with such limits. Though, hell, if you limited >>>> entire tx dependency trees (ie transactions and all required unconfirmed >>>> transactions for them) to something like 10 txn, maximum two levels >>>> deep, I also wouldnt have a problem. >>>> >>>> Matt >>>> >>>> On 08/14/15 19:33, Alex Morcos via bitcoin-dev wrote: >>>> > Hi everyone, >>>> > >>>> > >>>> > I'd like to propose a new set of requirements as a policy on when to >>>> > accept new transactions into the mempool and relay them. This policy >>>> > would affect transactions which have as inputs other transactions >>>> which >>>> > are not yet confirmed in the blockchain. >>>> > >>>> > The motivation for this policy is 6470 >>>> > which aims to limit >>>> the >>>> > size of a mempool. As discussed in that pull >>>> > >>> >, >>>> > once the mempool is full a new transaction must be able to pay not >>>> only >>>> > for the transaction it would evict, but any dependent transactions >>>> that >>>> > would be removed from the mempool as well. In order to make sure this >>>> > is always feasible, I'm proposing 4 new policy limits. >>>> > >>>> > All limits are command line configurable. >>>> > >>>> > The first two limits are required to make sure no chain of >>>> transactions >>>> > will be too large for the eviction code to handle: >>>> > >>>> > Max number of descendant txs : No transaction shall be accepted if it >>>> > would cause another transaction in the mempool to have too many >>>> > descendant transactions (all of which would have to be evicted if the >>>> > ancestor transaction was evicted). Default: 1000 >>>> > >>>> > Max descendant size : No transaction shall be accepted if it would >>>> cause >>>> > another transaction in the mempool to have the total size of all its >>>> > descendant transactions be too great. Default : maxmempool / 200 = >>>> 2.5MB >>>> > >>>> > The third limit is required to make sure calculating the state >>>> required >>>> > for sorting and limiting the mempool and enforcing the first 2 limits >>>> is >>>> > computationally feasible: >>>> > >>>> > Max number of ancestor txs: No transaction shall be accepted if it >>>> has >>>> > too many ancestor transactions which are not yet confirmed (ie, in the >>>> > mempool). Default: 100 >>>> > >>>> > The fourth limit is required to maintain the pre existing policy goal >>>> > that all transactions in the mempool should be mineable in the next >>>> block. >>>> > >>>> > Max ancestor size: No transaction shall be accepted if the total size >>>> of >>>> > all its unconfirmed ancestor transactions is too large. Default: 1MB >>>> > >>>> > (All limits include the transaction itself.) >>>> > >>>> > For reference, these limits would have affected less than 2% of >>>> > transactions entering the mempool in April or May of this year. >>>> During >>>> > the period of 7/6 through 7/14, while the network was under stress >>>> test, >>>> > as many as 25% of the transactions would have been affected. >>>> > >>>> > The code to implement the descendant package tracking and new policy >>>> > limits can be found in 6557 >>>> > which is built off of >>>> 6470. >>>> > >>>> > Thanks, >>>> > Alex >>>> > >>>> > >>>> > >>>> > _______________________________________________ >>>> > bitcoin-dev mailing list >>>> > bitcoin-dev@lists.linuxfoundation.org >>>> > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>> > >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists.linuxfoundation.org >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>>> >>> >>> >> > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev > > --001a1147a3b0f0debc05215fff92 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
What does "package" mean here? =C2=A0

When you say 25 txs, does that mean maximum linked chain depth, or= total number of dependent transactions regardless of chain depth?

Thanks,
-Danny


=

On Mo= n, Oct 5, 2015 at 11:45 AM, Alex Morcos via bitcoin-dev &= lt;bitcoin-dev@lists.linuxfoundation.org> wrote:
I'd like to propose updates to t= he new policy limits on unconfirmed transaction chains.=C2=A0

The existing limits in master and scheduled for release in 0.12 are:= =C2=A0
Ancestor packages =3D 100 txs and 900kb total size
Descendant packages =3D 1000 txs and 2500kb total size=C2=A0
<= br>
Before 0.12 is released I would like to propose a significant= reduction in these limits. In the course of analyzing algorithms for mempo= ol limiting, it became clear that large packages of unconfirmed transaction= s were the primary vector for mempool clogging or relay fee boosting attack= s. Feedback from the initial proposed limits was that they were too generou= s anyway.=C2=A0

The proposed new limits are:=C2=A0=
Ancestor packages =3D 25 txs and 100kb total size
Desc= endant packages =3D 25 txs and 100kb total size=C2=A0

<= div>Based on historical transaction data, the most restrictive of these lim= its is the 25 transaction count on descendant packages. Over the period of = April and May of this year (before stress tests), 5.8% of transactions woul= d have violated this limit alone. Applying all the limits together would ha= ve affected 6.1% of transactions.=C2=A0

Please kee= p in mind these are policy limits that affect transactions which depend on = other unconfirmed transactions only. They are not a change to consensus rul= es and do not affect how many chained txs a valid block may contain. Furthe= rmore, any transaction that was unable to be relayed due to these limits ne= ed only wait for some of its unconfirmed ancestors to be included in a bloc= k and then it could be successfully broadcast. This is unlikely to affect t= he total time from creation to inclusion in a block. Finally, these limits = are command line arguments that can easily be changed on an individual node= basis in Bitcoin Core.=C2=A0

Please give your fee= dback if you know of legitimate use cases that would be hindered by these l= imits.=C2=A0

Thanks,=C2=A0
Alex

On Mon, Sep 21, 2015 at 11:02 AM, Alex Morcos <morcos@gmail.co= m> wrote:
= Thanks for everyone's review.=C2=A0 These policy changes have been merg= ed in to master in 6654, which just implements these limits and no memp= ool limiting yet.=C2=A0 The default ancestor package size limit is 900kb no= t 1MB.

Yes I think these limits are generous, but they w= ere designed to be as generous as was computationally feasible so they were= unobjectionable (since the existing policy was no limits).=C2=A0 This does= not preclude future changes to policy that would reduce these limits.





On Fri, Aug 21,= 2015 at 3:52 PM, Danny Thorpe <danny.thorpe@gmail.com>= wrote:
The limits Alex = proposed are generous (bordering on obscene!), but dropping that down to al= lowing only two levels of chained unconfirmed transactions is too tight. = =C2=A0

Use case: Brokered asset transfers may require se= ts of transactions with a dependency tree depth of 3 to be published togeth= er. ( N seller txs, 1 broker bridge tx, M buyer txs )

<= div>If the originally proposed depth limit of 100 does not provide a suffic= ient cap on memory consumption or loop/recursion depth, a depth limit of 10= would provide plenty of headroom for this 3 level use case and similar pat= terns.

-Danny
<= /font>

On Fri, Aug 21, 2015 at 12:22 PM, Matt Corallo via bitcoin-dev = <bitcoin-dev@lists.linuxfoundation.org> = wrote:
I dont see any problem with such l= imits. Though, hell, if you limited
entire tx dependency trees (ie transactions and all required unconfirmed transactions for them) to something like 10 txn, maximum two levels
deep, I also wouldnt have a problem.

Matt

On 08/14/15 19:33, Alex Morcos via bitcoin-dev wrote:
> Hi everyone,
>
>
> I'd like to propose a new set of requirements as a policy on when = to
> accept new transactions into the mempool and relay them.=C2=A0 This po= licy
> would affect transactions which have as inputs other transactions whic= h
> are not yet confirmed in the blockchain.
>
> The motivation for this policy is 6470
> <https://github.com/bitcoin/bitcoin/pull/6470> which aims to limit the
> size of a mempool.=C2=A0 As discussed in that pull
> <https://github.com/bitco= in/bitcoin/pull/6470#issuecomment-125324736>,
> once the mempool is full a new transaction must be able to pay not onl= y
> for the transaction it would evict, but any dependent transactions tha= t
> would be removed from the mempool as well.=C2=A0 In order to make sure= this
> is always feasible, I'm proposing 4 new policy limits.
>
> All limits are command line configurable.
>
> The first two limits are required to make sure no chain of transaction= s
> will be too large for the eviction code to handle:
>
> Max number of descendant txs : No transaction shall be accepted if it<= br> > would cause another transaction in the mempool to have too many
> descendant transactions (all of which would have to be evicted if the<= br> > ancestor transaction was evicted).=C2=A0 Default: 1000
>
> Max descendant size : No transaction shall be accepted if it would cau= se
> another transaction in the mempool to have the total size of all its > descendant transactions be too great.=C2=A0 Default : maxmempool / 200= =C2=A0 =3D=C2=A0 2.5MB
>
> The third limit is required to make sure calculating the state require= d
> for sorting and limiting the mempool and enforcing the first 2 limits = is
> computationally feasible:
>
> Max number of ancestor txs:=C2=A0 No transaction shall be accepted if = it has
> too many ancestor transactions which are not yet confirmed (ie, in the=
> mempool). Default: 100
>
> The fourth limit is required to maintain the pre existing policy goal<= br> > that all transactions in the mempool should be mineable in the next bl= ock.
>
> Max ancestor size: No transaction shall be accepted if the total size = of
> all its unconfirmed ancestor transactions is too large.=C2=A0 Default:= 1MB
>
> (All limits include the transaction itself.)
>
> For reference, these limits would have affected less than 2% of
> transactions entering the mempool in April or May of this year.=C2=A0 = During
> the period of 7/6 through 7/14, while the network was under stress tes= t,
> as many as 25% of the transactions would have been affected.
>
> The code to implement the descendant package tracking and new policy > limits can be found in 6557
> <https://github.com/bitcoin/bitcoin/pull/6557> which is built off of 6470.
>
> Thanks,
> Alex
>
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org= /mailman/listinfo/bitcoin-dev
>
_______________________________________________
bitcoin-dev mailing list
= bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev




_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.= linuxfoundation.org
https://lists.linuxfoundation.org/mail= man/listinfo/bitcoin-dev


--001a1147a3b0f0debc05215fff92--