Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id F2B9A932 for ; Wed, 4 Jan 2017 06:06:35 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-it0-f51.google.com (mail-it0-f51.google.com [209.85.214.51]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 64198FE for ; Wed, 4 Jan 2017 06:06:34 +0000 (UTC) Received: by mail-it0-f51.google.com with SMTP id o141so293365810itc.0 for ; Tue, 03 Jan 2017 22:06:34 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=voskuil-org.20150623.gappssmtp.com; s=20150623; h=mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=J2ISpB60xm58SgLy/pJre6hiTvzlEBui1O2hSSfo9uU=; b=mqX373Y7e9LyrokYCTPlNqAYYN+OU6RM72pAwY4gWK4fJNVWSIEuZTMR2YFjU6RRDk bdlzjbpcVeFNtcyE1ynWu1wqVRdilszEc1JEgr53cbWXA39vY4UQuqDVlrqbToikqNO+ TNmim9RmmLNH//d/eJfxnAkj3Hiez/Cxr1a9iTWtQr5+1HMh5UYHii0PweelGwFJ6Xu+ R2o6eRVVzdeMtqsFZBhHuYrabnoyykdmdKTafN4Glf/lsHRE+zr9w4Er85u0r7IUgSV3 pBM//3hGl8gGzwRkT4J8EoL1wNhDrag/gDSTxrVBS41l9egZ+MiuzN9cqE+IR9yUmKTW BZYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=J2ISpB60xm58SgLy/pJre6hiTvzlEBui1O2hSSfo9uU=; b=SYzZB+X1qA//2LHiFyu//X+liImQNTD5BY2+E5E4vmVHhvrY7dgrZJvzdYhwaCW33b 9967eUFp1M4TaUglj00ZVS2jUWQmnhqY3otIKyW7DtzYTtWSVHcEY7/IgWpzEGOQ/s+F s7JD7uJFSPzn3/wyWWkkHM+1cWmWhDeHzxrF/GQrSvKBVT+ctM6uTU7AkdsVnRQLoIa+ Xk2P718clXoqvyWY12RK3//DStiVT2bxYBEPqgQzYAwwJNTS7e5vjwqUJqr6scXEkKKp letauPHN/i0e2WaS/gdhlERpu32aSpuP2dC1xZ3SQAFWMqzTQVMkS0V29MBs4mMU92l3 7vCg== X-Gm-Message-State: AIkVDXLJGog2NYskK5YbpURktOjPF2B+4+q6emwhpBZd/xdYJnWE1+JwhBGB/XtaPBnPFQ== X-Received: by 10.36.64.21 with SMTP id n21mr51619807ita.27.1483509993726; Tue, 03 Jan 2017 22:06:33 -0800 (PST) Received: from ?IPv6:2601:600:9000:d69e:2443:5ad0:902b:5a3b? ([2601:600:9000:d69e:2443:5ad0:902b:5a3b]) by smtp.gmail.com with ESMTPSA id c9sm6407137ioc.2.2017.01.03.22.06.32 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 03 Jan 2017 22:06:32 -0800 (PST) Content-Type: multipart/alternative; boundary=Apple-Mail-6EBFF3C9-346E-4C8E-A7CA-7F30229947C9 Mime-Version: 1.0 (1.0) From: Eric Voskuil X-Mailer: iPhone Mail (14B100) In-Reply-To: Date: Tue, 3 Jan 2017 22:06:31 -0800 Content-Transfer-Encoding: 7bit Message-Id: <25735AD4-CDCC-4632-AE03-1B657643E757@voskuil.org> References: <71d822e413ac457a530e1c367811cc24@cock.lu> <77b6dd25-0603-a0bd-6a9e-38098e5cb19d@jonasschnelli.ch> <74aeb4760316b59a3db56c0d16d11f28@cock.lu> <22b7d05fb2b8a7a0f1c2fa0b6b375f7e@cock.lu> To: Aaron Voisine , Bitcoin Protocol Discussion X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_LOW, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Wed, 04 Jan 2017 07:09:50 +0000 Subject: Re: [bitcoin-dev] Committed bloom filters for improved wallet performance and SPV security X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 04 Jan 2017 06:06:36 -0000 --Apple-Mail-6EBFF3C9-346E-4C8E-A7CA-7F30229947C9 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Credit card reversals involve an escrow agent with control over the entire n= etwork and with a strong interest in preserving the network. A better analog= y would be blind acceptance of any slip of paper under the assumption that i= t is sufficient currency. It may or may not be so, but you are on your own i= n either case. e > On Jan 3, 2017, at 4:36 PM, Aaron Voisine via bitcoin-dev wrote: >=20 > Knowing that a transaction is property formatted and that it has been broa= dcast to the gossip network is useful in many situations. You're only thinki= ng about whether you can know a transaction is valid and/or settled. This is= not the only possible useful information in actual real world use. Any situ= ation where credit card transactions are accepted today for instance, it is u= seful to know that a transaction has been initiated, even though it can be r= eversed at any time up to 60 days later. >=20 > Aaron Voisine > co-founder and CEO > breadwallet >=20 >> On Tue, Jan 3, 2017 at 4:10 PM, wrote: >> Unfortunately a non validating SPV wallet has absolutely no idea if >> the information about an unconfirmed transaction they are seeing is >> anything but properly formatted. They are connecting to an easily >> manipulated, sybil attacked, and untrusted network and then asking >> them for financial information. Seeing an unconfirmed transaction in a >> wallet that's not also fully validating is at best meaningless. >>=20 >>=20 >>> On 2017-01-03 15:46, Aaron Voisine wrote: >>> If the sender doesn't control the receiver's network connection, then >>> the information the receiver gains by watching the mempool is if the >>> transaction has propagated across the bitcoin network. This is useful >>> to know in all kinds of situations. >>>=20 >>> Aaron Voisine >>> co-founder and CEO >>> breadwallet [2] >>>=20 >>> On Tue, Jan 3, 2017 at 3:06 PM, adiabat wrote: >>>=20 >>>> Mempool transactions have their place, but "unconfirmed" and "SPV" >>>> don't belong together. Only a full node can tell if a transaction >>>> may get confirmed, or is nonsense. Unfortunately all the light / >>>> SPV wallets I know of show mempool transactions, which makes it hard >>>> to go back... (e.g. "why doesn't your software show 0-conf! your >>>> wallet is broken!", somewhat akin to people complaining about RBF) >>>>=20 >>>> So, this is easy, just don't worry about mempool filtering. Why are >>>> light clients looking at the mempool anyway? Maybe if there were >>>> some way to provide SPV proofs of all inputs, but that's a bit of a >>>> mess for full nodes to do. >>>>=20 >>>> Without mempool filtering, I think the committed bloom filters would >>>> be a great improvement over the current bloom filter setup, >>>> especially for lightning network use cases (with lightning, not >>>> finding out about a transaction can make you lose money). I want to >>>> work on it and may be able to at some point as it's somewhat related >>>> to lightning. >>>>=20 >>>> Also, if you're running a light client, and storing the filters the >>>> way you store block headers, there's really no reason to go all the >>>> way back to height 0. You can start grabbing headers at some point >>>> a while ago, before your set of keys was generated. I think it'd be >>>> very worth it even with GB-scale disk usage. >>>>=20 >>>> -Tadge >>>>=20 >>>> On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev >>>> wrote: >>>>=20 >>>> Unconfirmed transactions are incredibly important for real world >>>> use. Merchants for instance are willing to accept credit card >>>> payments of thousands of dollars and ship the goods despite the fact >>>> that the transaction can be reversed up to 60 days later. There is a >>>> very large cost to losing the ability to have instant transactions >>>> in many or even most situations. This cost is typically well above >>>> the fraud risk. >>>>=20 >>>> It's important to recognize that bitcoin serves a wide variety of >>>> use cases with different profiles for time sensitivity and fraud >>>> risk. >>>>=20 >>>> Aaron >>>>=20 >>>> On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev >>>> wrote: >>>> The concept combined with the weak blocks system where miners commit >>>>=20 >>>> to potential transaction inclusion with fractional difficulty blocks >>>>=20 >>>> is possible. I'm not personally convinced that unconfirmed >>>> transaction >>>>=20 >>>> display in a wallet is worth the privacy trade-off. The user has >>>> very >>>>=20 >>>> little to gain from this knowledge until the txn is in a block. >>>>=20 >>>> On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote: >>>>=20 >>>>> Hi >>>>=20 >>>>>> We introduce several concepts that rework the lightweight Bitcoin >>>>=20 >>>>>> client model in a manner which is secure, efficient and privacy >>>>=20 >>>>>> compatible. >>>>=20 >>>>>>=20 >>>>=20 >>>>>> The BFD can be used verbatim in replacement of BIP37, where the >>>> filter >>>>=20 >>>>>> can be cached between clients without needing to be recomputed. >>>> It can >>>>=20 >>>>>> also be used by normal pruned nodes to do re-scans locally of >>>> their >>>>=20 >>>>>> wallet without needing to have the block data available to scan, >>>> or >>>>=20 >>>>>> without reading the entire block chain from disk. >>>>=20 >>>>> I started exploring the potential of BFD after this specification. >>>>=20 >>>>>=20 >>>>=20 >>>>> What would be the preferred/recommended way to handle >>>> 0-conf/mempool >>>>=20 >>>>> filtering =E2=80=93 if & once BDF would have been deployed (any type, >>>>=20 >>>>> semi-trusted oracles or protocol-level/softfork)? >>>>=20 >>>>>=20 >>>>=20 >>>>> =46rom the user-experience perspective, this is probably pretty >>>> important >>>>=20 >>>>> (otherwise the experience will be that incoming funds can take >>>> serval >>>>=20 >>>>> minutes to hours until they appear). >>>>=20 >>>>> Using BIP37 bloom filters just for mempool filtering would >>>> obviously >>>>=20 >>>>> result in the same unwanted privacy-setup. >>>>=20 >>>>>=20 >>>>=20 >>>>> >>>>=20 >>>>>=20 >>>>=20 >>>>>=20 >>>>=20 >>>>> _______________________________________________ >>>>=20 >>>>> bitcoin-dev mailing list >>>>=20 >>>>> bitcoin-dev@lists.linuxfoundation.org >>>>=20 >>>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1] >>>>=20 >>>> _______________________________________________ >>>>=20 >>>> bitcoin-dev mailing list >>>>=20 >>>> bitcoin-dev@lists.linuxfoundation.org >>>>=20 >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1] >>>>=20 >>>> _______________________________________________ >>>> bitcoin-dev mailing list >>>> bitcoin-dev@lists.linuxfoundation.org >>>> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1] >>>=20 >>>=20 >>>=20 >>> Links: >>> ------ >>> [1] https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev >>> [2] http://breadwallet.com >=20 > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev --Apple-Mail-6EBFF3C9-346E-4C8E-A7CA-7F30229947C9 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Credit card reversals invol= ve an escrow agent with control over the entire network and with a strong in= terest in preserving the network. A better analogy would be blind acceptance= of any slip of paper under the assumption that it is sufficient currency. I= t may or may not be so, but you are on your own in either case.
e

On Jan 3, 2017, at 4:36 PM, Aaron Voisine via b= itcoin-dev <bitc= oin-dev@lists.linuxfoundation.org> wrote:

Knowing that a transaction is property for= matted and that it has been broadcast to the gossip network is useful in man= y situations. You're only thinking about whether you can know a transaction i= s valid and/or settled. This is not the only possible useful information in a= ctual real world use. Any situation where credit card transactions are accep= ted today for instance, it is useful to know that a transaction has been ini= tiated, even though it can be reversed at any time up to 60 days later.
<= div class=3D"gmail_extra">
Aaron Voisine
co-founder and CEO
breadwallet

On Tue, Jan 3, 2017 at 4:10 PM, <bfd@cock.lu&g= t; wrote:
Unfortunately a non validat= ing SPV wallet has absolutely no idea if
the information about an unconfirmed transaction they are seeing is
anything but properly formatted. They are connecting to an easily
manipulated, sybil attacked, and untrusted network and then asking
them for financial information. Seeing an unconfirmed transaction in a
wallet that's not also fully validating is at best meaningless.


On 2017-01-03 15:46, Aaron Voisine wrote:
If the sender doesn't control the receiver's network connection, then
the information the receiver gains by watching the mempool is if the
transaction has propagated across the bitcoin network. This is useful
to know in all kinds of situations.

Aaron Voisine
co-founder and CEO
 breadwallet [2]

On Tue, Jan 3, 2017 at 3:06 PM, adiabat <rx@awsomnet.org> wrote:

Mempool transactions have their place, but "unconfirmed" and "SPV"
don't belong together.  Only a full node can tell if a transaction
may get confirmed, or is nonsense.  Unfortunately all the light /
SPV wallets I know of show mempool transactions, which makes it hard
to go back... (e.g. "why doesn't your software show 0-conf! your
wallet is broken!", somewhat akin to people complaining about RBF)

So, this is easy, just don't worry about mempool filtering.  Why are light clients looking at the mempool anyway?  Maybe if there were
some way to provide SPV proofs of all inputs, but that's a bit of a
mess for full nodes to do.

Without mempool filtering, I think the committed bloom filters would
be a great improvement over the current bloom filter setup,
especially for lightning network use cases (with lightning, not
finding out about a transaction can make you lose money).  I want to work on it and may be able to at some point as it's somewhat related
to lightning.

Also, if you're running a light client, and storing the filters the
way you store block headers, there's really no reason to go all the
way back to height 0.  You can start grabbing headers at some point
= a while ago, before your set of keys was generated.  I think it'd be very worth it even with GB-scale disk usage.

-Tadge

On Tue, Jan 3, 2017 at 5:18 PM, Aaron Voisine via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:

Unconfirmed transactions are incredibly important for real world
use. Merchants for instance are willing to accept credit card
payments of thousands of dollars and ship the goods despite the fact
that the transaction can be reversed up to 60 days later. There is a
very large cost to losing the ability to have instant transactions
in many or even most situations. This cost is typically well above
the fraud risk.

It's important to recognize that bitcoin serves a wide variety of
use cases with different profiles for time sensitivity and fraud
risk.

Aaron

On Tue, Jan 3, 2017 at 12:41 PM bfd--- via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
The concept combined with the weak blocks system where miners commit

to potential transaction inclusion with fractional difficulty blocks

is possible. I'm not personally convinced that unconfirmed
transaction

display in a wallet is worth the privacy trade-off. The user has
very

little to gain from this knowledge until the txn is in a block.

On 2017-01-01 13:01, Jonas Schnelli via bitcoin-dev wrote:

Hi

We introduce several concepts that rework the lightweight Bitcoin

client model in a manner which is secure, efficient and privacy

compatible.



The BFD can be used verbatim in replacement of BIP37, where the
filter

can be cached between clients without needing to be recomputed.
It can

also be used by normal pruned nodes to do re-scans locally of
their

wallet without needing to have the block data available to scan,
or

without reading the entire block chain from disk.

I started exploring the potential of BFD after this specification.



What would be the preferred/recommended way to handle
0-conf/mempool

filtering =E2=80=93 if & once BDF would have been deployed (any type,

semi-trusted oracles or protocol-level/softfork)?



=46rom the user-experience perspective, this is probably pretty
important

(otherwise the experience will be that incoming funds can take
serval

minutes to hours until they appear).

Using BIP37 bloom filters just for mempool filtering would
obviously

result in the same unwanted privacy-setup.



</jonas>





_______________________________________________

bitcoin-dev mailing list

b= itcoin-dev@lists.linuxfoundation.org

https://lists.linuxfoundation.org/m= ailman/listinfo/bitcoin-dev [1]

_______________________________________________

bitcoin-dev mailing list

b= itcoin-dev@lists.linuxfoundation.org

https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1]

_______________________________________________
bitcoin-dev mailing list
b= itcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev [1]



Links:
------
[1] https://lists.linuxfoundation.o= rg/mailman/listinfo/bitcoin-dev
[2] = http://breadwallet.com

____________________= ___________________________
bitcoin-dev mailing list<= br>bitcoin-de= v@lists.linuxfoundation.org
https://lists.linuxfoundation= .org/mailman/listinfo/bitcoin-dev
= --Apple-Mail-6EBFF3C9-346E-4C8E-A7CA-7F30229947C9--