summaryrefslogtreecommitdiff
path: root/8d/f7c0f456034eff1e5c5b8a73149a874bdb5955
blob: bee780cf269aaf7a8f0ffda78491ff00e56e8c4f (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
Return-Path: <johanth@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id A6A2D1062;
	Mon, 28 Oct 2019 09:45:54 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-lj1-f194.google.com (mail-lj1-f194.google.com
	[209.85.208.194])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B6E36876;
	Mon, 28 Oct 2019 09:45:52 +0000 (UTC)
Received: by mail-lj1-f194.google.com with SMTP id q78so10571613lje.5;
	Mon, 28 Oct 2019 02:45:52 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:references:in-reply-to:from:date:message-id:subject:to
	:cc; bh=7jNv972mhsrOVapbAXo3fXObKpiDxj+9KyLxar1TDYM=;
	b=ujaYBJFyJiYF4syjnPqEIn0qfWypQiLwX7SVy1zBwWhtvFD0k5axHL6wLeQAItBU9c
	QO5p3IxoWRqdZ1xMEhS6HZJ/vB+fjQ9q4Fmlf1hIpJ0MCJF2IB+bdwGrzQ2nOPKWs4rC
	mK9eQMPPdDzoWDWzBv6MXZtJdnhisI9PfD9APzOD8iK16d7vxpJm5Ur1LdRpnh99HQ9r
	4GL1n2EqILHU5k5QUld42d3CX1eFn8r40o0sKi4FKxvUqb5PEV73rdgYP8mhCLdnFakT
	vJWxLvlTDoSfSb0VxGlOVjX0ImpOyq6DLDxle7y5Mikn1VdWsSDFI985zCC1KVDVg/AX
	6iWg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:references:in-reply-to:from:date
	:message-id:subject:to:cc;
	bh=7jNv972mhsrOVapbAXo3fXObKpiDxj+9KyLxar1TDYM=;
	b=kJHSxqxa8FRK2rUhFCmKM01yKExYQRJHFBsQvBtFuQsrUWT/sHHhDg1ImksrnCYX9X
	q9nr2fCy5qq0f1NIvdnzNHS3HSr3Y+vQ//CpypEeKRygiIyuz5wQ4370/oJpbjRc2dTv
	TXR3DtjwV1K9oefP065JC0wBB2NnqzFB91aJVSst9Y0ejITXsjPJb6avNbkqUdIbN5dT
	M+qbClpITcKF8Votm3mOp0JquECNgi5R5FnaKkoR6UsofqFj1PPo6EQjug7qEqtidxcy
	E2qJD4RniqPwAuOK59Lgbt6Gmn1ot8K+6kVQ4PepEuhyvTPPxVjHAs0BXswkY6Yy93Yf
	dIRQ==
X-Gm-Message-State: APjAAAU8+xtTgP7gITWj0gOi7/O5rGWogxJLtTlH8wpXUnydDtJ/m+kQ
	8di+fnyWQHU1Q6tHC2oVSqBLWYsabZ5IyGrDKk4=
X-Google-Smtp-Source: APXvYqySUDr6Xc92u+axwJbdLDw64iArpat3V3BYwg5o+jEnoTn/JMzxq+zg31lpL9p3dQrV7Ku3wfAkGYdsjdWgmTw=
X-Received: by 2002:a2e:9655:: with SMTP id z21mr4213871ljh.120.1572255950831; 
	Mon, 28 Oct 2019 02:45:50 -0700 (PDT)
MIME-Version: 1.0
References: <CAD3i26Cf+QpbFXh63NiMig9eceeKaezZwk89A_h_76S_XKkQ9g@mail.gmail.com>
	<6728FF51-E378-4AED-99BA-ECB83688AA9C@mattcorallo.com>
	<CAD5xwhhK4xexDe=aBv78BsK5BvE=4S0OcqeXYHVAfN3wTOr51Q@mail.gmail.com>
In-Reply-To: <CAD5xwhhK4xexDe=aBv78BsK5BvE=4S0OcqeXYHVAfN3wTOr51Q@mail.gmail.com>
From: =?UTF-8?Q?Johan_Tor=C3=A5s_Halseth?= <johanth@gmail.com>
Date: Mon, 28 Oct 2019 10:45:39 +0100
Message-ID: <CAD3i26DTxnDwhQd+kfS609W==A8oFA8DVJfwMiPt6NSXqhqw4w@mail.gmail.com>
To: Jeremy <jlrubin@mit.edu>
Content-Type: multipart/alternative; boundary="0000000000007206bd0595f55dcb"
X-Spam-Status: No, score=1.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, DOS_RCVD_IP_TWICE_B, FREEMAIL_FROM,
	HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=no version=3.3.1
X-Spam-Level: *
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
X-Mailman-Approved-At: Mon, 28 Oct 2019 10:06:48 +0000
Cc: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>,
	lightning-dev <lightning-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] [Lightning-dev] CPFP Carve-Out for Fee-Prediction
 Issues in Contracting Applications (eg Lightning)
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Mon, 28 Oct 2019 09:45:54 -0000

--0000000000007206bd0595f55dcb
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

>
>
> I don=E2=80=99te see how? Let=E2=80=99s imagine Party A has two spendable=
 outputs, now
> they stuff the package size on one of their spendable outlets until it is
> right at the limit, add one more on their other output (to meet the
> Carve-Out), and now Party B can=E2=80=99t do anything.

Matt: With the proposed change, party B would always be able to add a child
to its output, regardless of what games party A is playing.


Thanks for the explanation, Jeremy!


> In terms of relay cost, if an ancestor can be replaced, it will invalidat=
e
> all it's children, meaning that no one paid for that broadcasting. This c=
an
> be fixed by appropriately assessing Replace By Fee update fees to
> encapsulate all descendants, but there are some tricky edge cases that ma=
ke
> this non-obvious to do.


Relay cost is the obvious problem with just naively removing all limits.
Relaxing the current rules by allowing to add a child to each output as
long as it has a single unconfirmed parent would still only allow free
relay of O(size of parent) extra data (which might not be that bad? Similar
to the carve-out rule we could put limits on the child size). This would be
enough for the current LN use case (increasing fee of commitment tx), but
not for OP_SECURETHEBAG I guess, as you need the tree of children, as you
mention.

I imagine walking the mempool wouldn't change much, as you would only have
one extra child per output. But here I'm just speculating, as I don't know
the code well enough know what the diff would look like.


> OP_SECURETHEBAG can help with the LN issue by putting all HTLCS into a
> tree where they are individualized leaf nodes with a preceding CSV. Then,
> the above fix would ensure each HTLC always has time to close properly as
> they would have individualized lockpoints. This is desirable for some
> additional reasons and not for others, but it should "work".


This is interesting for an LN commitment! You could really hide every
output of the commitment within OP_STB, which could either allow bypassing
the fee-pinning attack entirely (if the output cannot be spent unconfirmed)
or adding fees to the commitment using SIGHASH_SINGLE|ANYONECANPAY.

- Johan

On Sun, Oct 27, 2019 at 8:13 PM Jeremy <jlrubin@mit.edu> wrote:

> Johan,
>
> The issues with mempool limits for OP_SECURETHEBAG are related, but have
> distinct solutions.
>
> There are two main categories of mempool issues at stake. One is relay
> cost, the other is mempool walking.
>
> In terms of relay cost, if an ancestor can be replaced, it will invalidat=
e
> all it's children, meaning that no one paid for that broadcasting. This c=
an
> be fixed by appropriately assessing Replace By Fee update fees to
> encapsulate all descendants, but there are some tricky edge cases that ma=
ke
> this non-obvious to do.
>
> The other issue is walking the mempool -- many of the algorithms we use i=
n
> the mempool can be N log N or N^2 in the number of descendants. (simple
> example: an input chain of length N to a fan out of N outputs that are al=
l
> spent, is O(N^2) to look up ancestors per-child, unless we're caching).
>
> The other sort of walking issue is where the indegree or outdegree for a
> transaction is high. Then when we are computing descendants or ancestors =
we
> will need to visit it multiple times. To avoid re-expanding a node, we
> currently cache it with a set. This uses O(N) extra memory and makes O(N
> Log N) (we use std::set not unordered_set) comparisons.
>
> I just opened a PR which should help with some of the walking issues by
> allowing us to cheaply cache which nodes we've visited on a run. It makes=
 a
> lot of previously O(N log N) stuff O(N) and doesn't allocate as much new
> memory. See: https://github.com/bitcoin/bitcoin/pull/17268.
>
>
> Now, for OP_SECURETHEBAG we want a particular property that is very
> different from with lightning htlcs (as is). We want that an unlimited
> number of child OP_SECURETHEBAG txns may extend from a confirmed
> OP_SECURETHEBAG, and then at the leaf nodes, we want the same rule as
> lightning (one dangling unconfirmed to permit channels).
>
> OP_SECURETHEBAG can help with the LN issue by putting all HTLCS into a
> tree where they are individualized leaf nodes with a preceding CSV. Then,
> the above fix would ensure each HTLC always has time to close properly as
> they would have individualized lockpoints. This is desirable for some
> additional reasons and not for others, but it should "work".
>
>
>
> --
> @JeremyRubin <https://twitter.com/JeremyRubin>
> <https://twitter.com/JeremyRubin>
>
>
> On Fri, Oct 25, 2019 at 10:31 AM Matt Corallo <lf-lists@mattcorallo.com>
> wrote:
>
>> I don=E2=80=99te see how? Let=E2=80=99s imagine Party A has two spendabl=
e outputs, now
>> they stuff the package size on one of their spendable outlets until it i=
s
>> right at the limit, add one more on their other output (to meet the
>> Carve-Out), and now Party B can=E2=80=99t do anything.
>>
>> On Oct 24, 2019, at 21:05, Johan Tor=C3=A5s Halseth <johanth@gmail.com> =
wrote:
>>
>> =EF=BB=BF
>> It essentially changes the rule to always allow CPFP-ing the commitment
>> as long as there is an output available without any descendants. It chan=
ges
>> the commitment from "you always need at least, and exactly, one non-CSV
>> output per party. " to "you always need at least one non-CSV output per
>> party. "
>>
>> I realize these limits are there for a reason though, but I'm wondering
>> if could relax them. Also now that jeremyrubin has expressed problems wi=
th
>> the current mempool limits.
>>
>> On Thu, Oct 24, 2019 at 11:25 PM Matt Corallo <lf-lists@mattcorallo.com>
>> wrote:
>>
>>> I may be missing something, but I'm not sure how this changes anything?
>>>
>>> If you have a commitment transaction, you always need at least, and
>>> exactly, one non-CSV output per party. The fact that there is a size
>>> limitation on the transaction that spends for carve-out purposes only
>>> effects how many other inputs/outputs you can add, but somehow I doubt
>>> its ever going to be a large enough number to matter.
>>>
>>> Matt
>>>
>>> On 10/24/19 1:49 PM, Johan Tor=C3=A5s Halseth wrote:
>>> > Reviving this old thread now that the recently released RC for bitcoi=
nd
>>> > 0.19 includes the above mentioned carve-out rule.
>>> >
>>> > In an attempt to pave the way for more robust CPFP of on-chain
>>> contracts
>>> > (Lightning commitment transactions), the carve-out rule was added in
>>> > https://github.com/bitcoin/bitcoin/pull/15681. However, having worked
>>> on
>>> > an implementation of a new commitment format for utilizing the Bring
>>> > Your Own Fees strategy using CPFP, I=E2=80=99m wondering if the speci=
al case
>>> > rule should have been relaxed a bit, to avoid the need for adding a 1
>>> > CSV to all outputs (in case of Lightning this means HTLC scripts woul=
d
>>> > need to be changed to add the CSV delay).
>>> >
>>> > Instead, what about letting the rule be
>>> >
>>> > The last transaction which is added to a package of dependent
>>> > transactions in the mempool must:
>>> >   * Have no more than one unconfirmed parent.
>>> >
>>> > This would of course allow adding a large transaction to each output =
of
>>> > the unconfirmed parent, which in effect would allow an attacker to
>>> > exceed the MAX_PACKAGE_VIRTUAL_SIZE limit in some cases. However, is
>>> > this a problem with the current mempool acceptance code in bitcoind? =
I
>>> > would imagine evicting transactions based on feerate when the max
>>> > mempool size is met handles this, but I=E2=80=99m asking since it see=
ms like
>>> > there has been several changes to the acceptance code and eviction
>>> > policy since the limit was first introduced.
>>> >
>>> > - Johan
>>> >
>>> >
>>> > On Wed, Feb 13, 2019 at 6:57 AM Rusty Russell <rusty@rustcorp.com.au
>>> > <mailto:rusty@rustcorp.com.au>> wrote:
>>> >
>>> >     Matt Corallo <lf-lists@mattcorallo.com
>>> >     <mailto:lf-lists@mattcorallo.com>> writes:
>>> >     >>> Thus, even if you imagine a steady-state mempool growth,
>>> unless the
>>> >     >>> "near the top of the mempool" criteria is "near the top of th=
e
>>> next
>>> >     >>> block" (which is obviously *not* incentive-compatible)
>>> >     >>
>>> >     >> I was defining "top of mempool" as "in the first 4 MSipa", ie.
>>> next
>>> >     >> block, and assumed you'd only allow RBF if the old package
>>> wasn't
>>> >     in the
>>> >     >> top and the replacement would be.  That seems incentive
>>> >     compatible; more
>>> >     >> than the current scheme?
>>> >     >
>>> >     > My point was, because of block time variance, even that criteri=
a
>>> >     doesn't hold up. If you assume a steady flow of new transactions
>>> and
>>> >     one or two blocks come in "late", suddenly "top 4MWeight" isn't
>>> >     likely to get confirmed until a few blocks come in "early". Given
>>> >     block variance within a 12 block window, this is a relatively
>>> likely
>>> >     scenario.
>>> >
>>> >     [ Digging through old mail. ]
>>> >
>>> >     Doesn't really matter.  Lightning close algorithm would be:
>>> >
>>> >     1.  Give bitcoind unileratal close.
>>> >     2.  Ask bitcoind what current expidited fee is (or survey your
>>> mempool).
>>> >     3.  Give bitcoind child "push" tx at that total feerate.
>>> >     4.  If next block doesn't contain unilateral close tx, goto 2.
>>> >
>>> >     In this case, if you allow a simpified RBF where 'you can replace
>>> if
>>> >     1. feerate is higher, 2. new tx is in first 4Msipa of mempool, 3.
>>> >     old tx isnt',
>>> >     it works.
>>> >
>>> >     It allows someone 100k of free tx spam, sure.  But it's simple.
>>> >
>>> >     We could further restrict it by marking the unilateral close
>>> somehow to
>>> >     say "gonna be pushed" and further limiting the child tx weight
>>> (say,
>>> >     5kSipa?) in that case.
>>> >
>>> >     Cheers,
>>> >     Rusty.
>>> >     _______________________________________________
>>> >     Lightning-dev mailing list
>>> >     Lightning-dev@lists.linuxfoundation.org
>>> >     <mailto:Lightning-dev@lists.linuxfoundation.org>
>>> >     https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>> >
>>>
>> _______________________________________________
>> Lightning-dev mailing list
>> Lightning-dev@lists.linuxfoundation.org
>> https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
>>
>

--0000000000007206bd0595f55dcb
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><blockq=
uote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1p=
x solid rgb(204,204,204);padding-left:1ex"><br>I don=E2=80=99te see how? Le=
t=E2=80=99s imagine Party A has two spendable outputs, now they stuff the p=
ackage size on one of their spendable outlets until it is right at the limi=
t, add one more on their other output (to meet the Carve-Out), and now Part=
y B can=E2=80=99t do anything.</blockquote><div>Matt: With the proposed cha=
nge, party B would always be able to add a child to its output, regardless =
of what games party A is playing.=C2=A0</div><div><br></div><div><br></div>=
<div>Thanks for the explanation, Jeremy!</div><div>=C2=A0</div><blockquote =
class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px sol=
id rgb(204,204,204);padding-left:1ex">In terms of relay cost, if an ancesto=
r can be replaced, it will invalidate all it&#39;s children, meaning that n=
o one paid for that broadcasting. This can be fixed by appropriately assess=
ing Replace By Fee update fees to encapsulate all descendants, but there ar=
e some tricky edge cases that make this non-obvious to do.</blockquote><div=
><br></div><div>Relay cost is the obvious problem with just naively removin=
g all limits. Relaxing the current rules by allowing to add a child to each=
 output as long as it has a single unconfirmed=C2=A0parent would still only=
 allow free relay of O(size of parent) extra data (which might not be that =
bad? Similar to the carve-out rule we could put limits on the child size). =
This would be enough for the current LN use case (increasing fee of commitm=
ent tx), but not for OP_SECURETHEBAG I guess, as you need the tree of child=
ren, as you mention.</div><div><br></div><div>I imagine walking the mempool=
 wouldn&#39;t change much, as you would only have one extra child per outpu=
t. But here I&#39;m just speculating, as I don&#39;t know the code well eno=
ugh know what the diff would look like.</div><div><br></div><blockquote cla=
ss=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid =
rgb(204,204,204);padding-left:1ex"><br>OP_SECURETHEBAG can help with the LN=
 issue by putting all HTLCS into a tree where they are individualized leaf =
nodes with a preceding CSV. Then, the above fix would ensure each HTLC alwa=
ys has time to close properly as they would have individualized lockpoints.=
 This is desirable for some additional reasons and not for others, but it s=
hould &quot;work&quot;.</blockquote><div><br></div><div>This is interesting=
 for an LN commitment! You could really hide every output of the commitment=
 within OP_STB, which could either allow bypassing the fee-pinning attack e=
ntirely (if the output cannot be spent unconfirmed) or adding fees to the c=
ommitment using SIGHASH_SINGLE|ANYONECANPAY.</div><div><br></div><div>- Joh=
an</div><br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr=
">On Sun, Oct 27, 2019 at 8:13 PM Jeremy &lt;<a href=3D"mailto:jlrubin@mit.=
edu">jlrubin@mit.edu</a>&gt; wrote:<br></div><blockquote class=3D"gmail_quo=
te" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204=
);padding-left:1ex"><div dir=3D"ltr"><div style=3D"font-family:arial,helvet=
ica,sans-serif;font-size:small;color:rgb(0,0,0)">Johan,</div><div style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><b=
r></div><div style=3D"font-family:arial,helvetica,sans-serif;font-size:smal=
l;color:rgb(0,0,0)">The issues with mempool limits for OP_SECURETHEBAG are =
related, but have distinct solutions.</div><div style=3D"font-family:arial,=
helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style=
=3D"font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)=
">There are two main categories of mempool issues at stake. One is relay co=
st, the other is mempool walking.</div><div style=3D"font-family:arial,helv=
etica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">In=
 terms of relay cost, if an ancestor can be replaced, it will invalidate al=
l it&#39;s children, meaning that no one paid for that broadcasting. This c=
an be fixed by appropriately assessing Replace By Fee update fees to encaps=
ulate all descendants, but there are some tricky edge cases that make this =
non-obvious to do.</div><div style=3D"font-family:arial,helvetica,sans-seri=
f;font-size:small;color:rgb(0,0,0)"><br></div><div style=3D"font-family:ari=
al,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)">The other issue i=
s walking the mempool -- many of the algorithms we use in the mempool can b=
e N log N or N^2 in the number of descendants. (simple example: an input ch=
ain of length N to a fan out of N outputs that are all spent, is O(N^2) to =
look up ancestors per-child, unless we&#39;re caching).</div><div style=3D"=
font-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><b=
r></div><div style=3D"font-family:arial,helvetica,sans-serif;font-size:smal=
l;color:rgb(0,0,0)">The other sort of walking issue is where the indegree o=
r outdegree for a transaction is high. Then when we are computing descendan=
ts or ancestors we will need to visit it multiple times. To avoid re-expand=
ing a node, we currently cache it with a set. This uses O(N) extra memory a=
nd makes O(N Log N) (we use std::set not unordered_set) comparisons. <br></=
div><div style=3D"font-family:arial,helvetica,sans-serif;font-size:small;co=
lor:rgb(0,0,0)"><br></div><div style=3D"font-family:arial,helvetica,sans-se=
rif;font-size:small;color:rgb(0,0,0)">I just opened a PR which should help =
with some of the walking issues by allowing us to cheaply cache which nodes=
 we&#39;ve visited on a run. It makes a lot of previously O(N log N) stuff =
O(N) and doesn&#39;t allocate as much new memory. See: <a href=3D"https://g=
ithub.com/bitcoin/bitcoin/pull/17268" target=3D"_blank">https://github.com/=
bitcoin/bitcoin/pull/17268</a>.</div><div style=3D"font-family:arial,helvet=
ica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div><div style=3D"fo=
nt-family:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br>=
</div><div style=3D"font-family:arial,helvetica,sans-serif;font-size:small;=
color:rgb(0,0,0)">Now, for OP_SECURETHEBAG we want a particular property th=
at is very different from with lightning htlcs (as is). We want that an unl=
imited number of child OP_SECURETHEBAG txns may extend from a confirmed OP_=
SECURETHEBAG, and then at the leaf nodes, we want the same rule as lightnin=
g (one dangling unconfirmed to permit channels).</div><div style=3D"font-fa=
mily:arial,helvetica,sans-serif;font-size:small;color:rgb(0,0,0)"><br></div=
><div style=3D"font-family:arial,helvetica,sans-serif;font-size:small;color=
:rgb(0,0,0)">OP_SECURETHEBAG can help with the LN issue by putting all HTLC=
S into a tree where they are individualized leaf nodes with a preceding CSV=
. Then, the above fix would ensure each HTLC always has time to close prope=
rly as they would have individualized lockpoints. This is desirable for som=
e additional reasons and not for others, but it should &quot;work&quot;.<br=
></div><div style=3D"font-family:arial,helvetica,sans-serif;font-size:small=
;color:rgb(0,0,0)"><br></div><div><div dir=3D"ltr"><div dir=3D"ltr"><br></d=
iv><div dir=3D"ltr"><br></div><div dir=3D"ltr">--<br><a href=3D"https://twi=
tter.com/JeremyRubin" target=3D"_blank">@JeremyRubin</a><a href=3D"https://=
twitter.com/JeremyRubin" target=3D"_blank"></a></div></div></div><br></div>=
<br><div class=3D"gmail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Fri=
, Oct 25, 2019 at 10:31 AM Matt Corallo &lt;<a href=3D"mailto:lf-lists@matt=
corallo.com" target=3D"_blank">lf-lists@mattcorallo.com</a>&gt; wrote:<br><=
/div><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;bo=
rder-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir=3D"auto"><d=
iv dir=3D"ltr">I don=E2=80=99te see how? Let=E2=80=99s imagine Party A has =
two spendable outputs, now they stuff the package size on one of their spen=
dable outlets until it is right at the limit, add one more on their other o=
utput (to meet the Carve-Out), and now Party B can=E2=80=99t do anything.</=
div><div dir=3D"ltr"><br><blockquote type=3D"cite">On Oct 24, 2019, at 21:0=
5, Johan Tor=C3=A5s Halseth &lt;<a href=3D"mailto:johanth@gmail.com" target=
=3D"_blank">johanth@gmail.com</a>&gt; wrote:<br><br></blockquote></div><blo=
ckquote type=3D"cite"><div dir=3D"ltr">=EF=BB=BF<div dir=3D"ltr"><div dir=
=3D"ltr"><div dir=3D"ltr">It essentially changes the rule to always allow C=
PFP-ing the commitment as long as there is an output available without any =
descendants. It changes the commitment from &quot;you always need at least,=
 and exactly, one non-CSV output per party. &quot; to &quot;you always need=
 at least one non-CSV output per party. &quot;</div><div dir=3D"ltr"><br></=
div><div>I realize these limits are there for a reason though, but I&#39;m =
wondering if could relax them. Also now that jeremyrubin has expressed prob=
lems with the current mempool limits.</div></div></div><br><div class=3D"gm=
ail_quote"><div dir=3D"ltr" class=3D"gmail_attr">On Thu, Oct 24, 2019 at 11=
:25 PM Matt Corallo &lt;<a href=3D"mailto:lf-lists@mattcorallo.com" target=
=3D"_blank">lf-lists@mattcorallo.com</a>&gt; wrote:<br></div><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left:1px solid=
 rgb(204,204,204);padding-left:1ex">I may be missing something, but I&#39;m=
 not sure how this changes anything?<br>
<br>
If you have a commitment transaction, you always need at least, and<br>
exactly, one non-CSV output per party. The fact that there is a size<br>
limitation on the transaction that spends for carve-out purposes only<br>
effects how many other inputs/outputs you can add, but somehow I doubt<br>
its ever going to be a large enough number to matter.<br>
<br>
Matt<br>
<br>
On 10/24/19 1:49 PM, Johan Tor=C3=A5s Halseth wrote:<br>
&gt; Reviving this old thread now that the recently released RC for bitcoin=
d<br>
&gt; 0.19 includes the above mentioned carve-out rule.<br>
&gt; <br>
&gt; In an attempt to pave the way for more robust CPFP of on-chain contrac=
ts<br>
&gt; (Lightning commitment transactions), the carve-out rule was added in<b=
r>
&gt; <a href=3D"https://github.com/bitcoin/bitcoin/pull/15681" rel=3D"noref=
errer" target=3D"_blank">https://github.com/bitcoin/bitcoin/pull/15681</a>.=
 However, having worked on<br>
&gt; an implementation of a new commitment format for utilizing the Bring<b=
r>
&gt; Your Own Fees strategy using CPFP, I=E2=80=99m wondering if the specia=
l case<br>
&gt; rule should have been relaxed a bit, to avoid the need for adding a 1<=
br>
&gt; CSV to all outputs (in case of Lightning this means HTLC scripts would=
<br>
&gt; need to be changed to add the CSV delay).<br>
&gt; <br>
&gt; Instead, what about letting the rule be<br>
&gt; <br>
&gt; The last transaction which is added to a package of dependent<br>
&gt; transactions in the mempool must:<br>
&gt; =C2=A0 * Have no more than one unconfirmed parent.<br>
&gt; <br>
&gt; This would of course allow adding a large transaction to each output o=
f<br>
&gt; the unconfirmed parent, which in effect would allow an attacker to<br>
&gt; exceed the MAX_PACKAGE_VIRTUAL_SIZE limit in some cases. However, is<b=
r>
&gt; this a problem with the current mempool acceptance code in bitcoind? I=
<br>
&gt; would imagine evicting transactions based on feerate when the max<br>
&gt; mempool size is met handles this, but I=E2=80=99m asking since it seem=
s like<br>
&gt; there has been several changes to the acceptance code and eviction<br>
&gt; policy since the limit was first introduced.<br>
&gt; <br>
&gt; - Johan<br>
&gt; <br>
&gt; <br>
&gt; On Wed, Feb 13, 2019 at 6:57 AM Rusty Russell &lt;<a href=3D"mailto:ru=
sty@rustcorp.com.au" target=3D"_blank">rusty@rustcorp.com.au</a><br>
&gt; &lt;mailto:<a href=3D"mailto:rusty@rustcorp.com.au" target=3D"_blank">=
rusty@rustcorp.com.au</a>&gt;&gt; wrote:<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0Matt Corallo &lt;<a href=3D"mailto:lf-lists@mattcor=
allo.com" target=3D"_blank">lf-lists@mattcorallo.com</a><br>
&gt;=C2=A0 =C2=A0 =C2=A0&lt;mailto:<a href=3D"mailto:lf-lists@mattcorallo.c=
om" target=3D"_blank">lf-lists@mattcorallo.com</a>&gt;&gt; writes:<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt;&gt;&gt; Thus, even if you imagine a steady-sta=
te mempool growth, unless the<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt;&gt;&gt; &quot;near the top of the mempool&quot=
; criteria is &quot;near the top of the next<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt;&gt;&gt; block&quot; (which is obviously *not* =
incentive-compatible)<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt;&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt;&gt; I was defining &quot;top of mempool&quot; =
as &quot;in the first 4 MSipa&quot;, ie. next<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt;&gt; block, and assumed you&#39;d only allow RB=
F if the old package wasn&#39;t<br>
&gt;=C2=A0 =C2=A0 =C2=A0in the<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt;&gt; top and the replacement would be.=C2=A0 Th=
at seems incentive<br>
&gt;=C2=A0 =C2=A0 =C2=A0compatible; more<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt;&gt; than the current scheme?<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0&gt; My point was, because of block time variance, =
even that criteria<br>
&gt;=C2=A0 =C2=A0 =C2=A0doesn&#39;t hold up. If you assume a steady flow of=
 new transactions and<br>
&gt;=C2=A0 =C2=A0 =C2=A0one or two blocks come in &quot;late&quot;, suddenl=
y &quot;top 4MWeight&quot; isn&#39;t<br>
&gt;=C2=A0 =C2=A0 =C2=A0likely to get confirmed until a few blocks come in =
&quot;early&quot;. Given<br>
&gt;=C2=A0 =C2=A0 =C2=A0block variance within a 12 block window, this is a =
relatively likely<br>
&gt;=C2=A0 =C2=A0 =C2=A0scenario.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0[ Digging through old mail. ]<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0Doesn&#39;t really matter.=C2=A0 Lightning close al=
gorithm would be:<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A01.=C2=A0 Give bitcoind unileratal close.<br>
&gt;=C2=A0 =C2=A0 =C2=A02.=C2=A0 Ask bitcoind what current expidited fee is=
 (or survey your mempool).<br>
&gt;=C2=A0 =C2=A0 =C2=A03.=C2=A0 Give bitcoind child &quot;push&quot; tx at=
 that total feerate.<br>
&gt;=C2=A0 =C2=A0 =C2=A04.=C2=A0 If next block doesn&#39;t contain unilater=
al close tx, goto 2.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0In this case, if you allow a simpified RBF where &#=
39;you can replace if<br>
&gt;=C2=A0 =C2=A0 =C2=A01. feerate is higher, 2. new tx is in first 4Msipa =
of mempool, 3.<br>
&gt;=C2=A0 =C2=A0 =C2=A0old tx isnt&#39;,<br>
&gt;=C2=A0 =C2=A0 =C2=A0it works.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0It allows someone 100k of free tx spam, sure.=C2=A0=
 But it&#39;s simple.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0We could further restrict it by marking the unilate=
ral close somehow to<br>
&gt;=C2=A0 =C2=A0 =C2=A0say &quot;gonna be pushed&quot; and further limitin=
g the child tx weight (say,<br>
&gt;=C2=A0 =C2=A0 =C2=A05kSipa?) in that case.<br>
&gt; <br>
&gt;=C2=A0 =C2=A0 =C2=A0Cheers,<br>
&gt;=C2=A0 =C2=A0 =C2=A0Rusty.<br>
&gt;=C2=A0 =C2=A0 =C2=A0_______________________________________________<br>
&gt;=C2=A0 =C2=A0 =C2=A0Lightning-dev mailing list<br>
&gt;=C2=A0 =C2=A0 =C2=A0<a href=3D"mailto:Lightning-dev@lists.linuxfoundati=
on.org" target=3D"_blank">Lightning-dev@lists.linuxfoundation.org</a><br>
&gt;=C2=A0 =C2=A0 =C2=A0&lt;mailto:<a href=3D"mailto:Lightning-dev@lists.li=
nuxfoundation.org" target=3D"_blank">Lightning-dev@lists.linuxfoundation.or=
g</a>&gt;<br>
&gt;=C2=A0 =C2=A0 =C2=A0<a href=3D"https://lists.linuxfoundation.org/mailma=
n/listinfo/lightning-dev" rel=3D"noreferrer" target=3D"_blank">https://list=
s.linuxfoundation.org/mailman/listinfo/lightning-dev</a><br>
&gt; <br>
</blockquote></div>
</div></blockquote></div>_______________________________________________<br=
>
Lightning-dev mailing list<br>
<a href=3D"mailto:Lightning-dev@lists.linuxfoundation.org" target=3D"_blank=
">Lightning-dev@lists.linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev=
" rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.org/ma=
ilman/listinfo/lightning-dev</a><br>
</blockquote></div>
</blockquote></div></div></div></div></div>

--0000000000007206bd0595f55dcb--