To: Peter Todd <pete@petertodd.org>
References: <55D6AD19.10305@mattcorallo.com> <20150821053819.GA18176@muck>
	<20150821054219.GB18176@muck> <55D7662E.4090104@mattcorallo.com>
	<20150821220603.GC7450@muck>
From: Matt Corallo <lf-lists@mattcorallo.com>
Message-ID: <55D7CB7D.6080100@mattcorallo.com>
Date: Sat, 22 Aug 2015 01:08:13 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
	Thunderbird/38.1.0
MIME-Version: 1.0
In-Reply-To: <20150821220603.GC7450@muck>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Cc: bitcoin-dev@lists.linuxfoundation.org, Mike Hearn <mike@plan99.net>
Subject: Re: [bitcoin-dev] Revisiting NODE_BLOOM: Proposed BIP
Precedence: list


On 08/21/15 22:06, Peter Todd wrote:
> On Fri, Aug 21, 2015 at 05:55:58PM +0000, Matt Corallo wrote:
>> Revised copy follows. re: mentioning the HTTP seeding stuff, I'm not
>> sure we want to encourage more people aside from bitcoinj to use
>> that...I thought about adding a DNS seed section to this bip, but
>> decided against it...still, I think we should add the option to select
>> service bits to DNS seeds ASAP.
> 
> Well, in general relying on seeds every time you start your node is a
> really bad idea; doing so much be carefully weighed against the
> downsides and should be used only as a last resort. Nodes should be
> doing caching and proper gossip protocol participation whenever
> possible. (note how bitcoinj nodes *do* rely on centralized servers,
> implemented with an unauthenticated, unencrypted, protocol - the worst
> of all possible solutions with many possible MITM vectors and privacy
> security holes)
>
> To that end, I'd be inclined to leave the DNS seed protocol as it is and
> let others solve the centralized server use-case, for which Cartographer
> isn't all that bad of a load balancing mechanism. Also as gmaxwell noted
> on IRC, adding flag bits does have privacy implications.

Had a discussion on IRC and with Pieter, and I kinda agree that the more
optimal way is for DNS seeds to, instead of returning NODE_NETWORK
nodes, return any node which responds to getaddr, allowing clients to
connect to a few DNS seeds by name, do a getaddr, then disconnect (like
Bitcoin Core does now if you're using Tor). They can then select the
peers they want based on nServices.

>> Re: need to "shard" the blockchain: not sure what you're referring to
>> here. The bloom filter stuff requires you to download the chain
>> in-order, sure, but you have to do that for headers anyway, and
>> hopefully your total data isnt too much more than headers alone.
> 
> Any protocol change that would split blocks themselves into multiples.
> Not an easy problem to solve, but given the inherent O(n^2) scaling of
> global consensus blockchains, it's the only kind of solution that could
> in the future make the blockchain itself have reasonable scalability.

Meh, whatever, justification is already provided well enough without
having to go into "but if we did this long into the future"  arguments.

>> Anyone have the best reference for the DoS issues?
> 
> Well actually, we can reference the DoS attacks that Bitcoin XT nodes
> are undergoing right now - part of the attack is repeated Bloom filter
> requests to soak up disk IO bandwidth. I've CC'd Gavin and Mike - as far
> as I know they haven't published details of those attacks - a write-up
> would be very helpful.
> 
> While so far those are being directed only at XT nodes, obviously this
> is a potential issue for Core nodes as well. Like I mentioned last time
> around, it's critical that miners aren't affected by these attacks -
> nodes simply serving SPV wallet clients are much less latency sensitive,
> so a good DoS attack mitigation strategy would be to have the two
> classes of nodes out there "in the wild"

Ehh, I was going more for the oldest mention.

>> BIP: ?
>> Title: NODE_BLOOM service bit
>> Author: Matt Corallo <bip@bluematt.me>, Peter Todd <pete@petertodd.org>
>> Type: Standards Track (draft)
>> Created: 20-08-2015
>>
>> Abstract
>> ========
>>
>> This BIP extends BIP 37, Connection Bloom filtering, by defining a
>> service bit to allow peers to advertise that they support bloom filters
>> explicitly. It also bumps the protocol version to allow peers to
>> identify old nodes which allow bloom filtering of the connection despite
>> lacking the new service bit.
>>
>>
>> Motivation
>> ==========
>>
>> BIP 37 did not specify a service bit for the bloom filter service, thus
>> implicitly assuming that all nodes that serve peers data support it.
>> However, the connection filtering algorithm proposed in BIP 37, and
>> implemented in several clients today, has been shown to provide little
>> to no privacy[1], as well as being a large DoS risk on some nodes[2].
>> Thus, allowing node operators to disable connection bloom filtering is a
>> much-needed feature.
>>
>>
>> Specification
>> =============
>>
>> The following protocol bit is added:
>>
>>     NODE_BLOOM = (1 << 2)
>>
>> Nodes which support bloom filters should set that protocol bit.
>> Otherwise it should remain unset. In addition the protocol version is
>> increased from 70002 to 70011 in the reference implementation. It is
>> often the case that nodes which have a protocol version smaller than
>> 70011, but larger than 70000 support bloom filtered connections without
>> the NODE_BLOOM bit set, however clients which require bloom filtered
>> connections should avoid making this assumption.
>>
>> NODE_BLOOM is distinct from NODE_NETWORK, and it is legal to advertise
>> NODE_BLOOM but not NODE_NETWORK (eg for nodes running in pruned mode
>> which, nonetheless, provide filtered access to the data which they do have).
>>
>> If a node does not support bloom filters but receives a "filterload",
>> "filteradd", or "filterclear" message from a peer the node should
>> disconnect that peer immediately. For backwards compatibility, in
>> initial implementations, nodes may choose to only disconnect nodes which
>> have the new protocol version set and attempt to send a filter command.
>>
>> While outside the scope of this BIP it is suggested that DNS seeds and
>> other peer discovery mechanisms support the ability to specify the
>> services required; current implementations simply check only that
>> NODE_NETWORK is set.
>>
>>
>> Design rational
>> ===============
>>
>> A service bit was chosen as applying a bloom filter is a service.
>>
>> The increase in protocol version is for backwards compatibility. In
>> initial implementations, old nodes which are not yet aware of NODE_BLOOM
>> and use a protocol version < 70011 may still send filter* messages to a
>> node without NODE_BLOOM. This feature may be removed after there are
>> sufficient NODE_BLOOM nodes available and SPV clients have upgraded,
>> allowing node operators to fully close the bloom-related DoS vectors.
>>
>>
>> Reference Implementation
>> ========================
>>
>> https://github.com/bitcoin/bitcoin/pull/6579
>>
>>
>> Copyright
>> =========
>>
>> This document is placed in the public domain.
>>
>>
>> References
>> ==========
>>
>> [1] http://eprint.iacr.org/2014/763
>> [2] ???? is one example where the issues were found, though others
>> independently discovered issues as well. Sample DoS exploit code
>> available at https://github.com/petertodd/bloom-io-attack.
>