Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1YsG0u-00022J-Bi for bitcoin-development@lists.sourceforge.net; Tue, 12 May 2015 19:38:48 +0000 Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of bitpay.com designates 209.85.218.45 as permitted sender) client-ip=209.85.218.45; envelope-from=jgarzik@bitpay.com; helo=mail-oi0-f45.google.com; Received: from mail-oi0-f45.google.com ([209.85.218.45]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1YsG0s-0004vk-GJ for bitcoin-development@lists.sourceforge.net; Tue, 12 May 2015 19:38:48 +0000 Received: by oift201 with SMTP id t201so14548738oif.3 for ; Tue, 12 May 2015 12:38:40 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc:content-type; bh=Mf7bqRtG8DCM5GWSISIRiFzJycUgEF0em2bx8rrzHik=; b=IDH9iTigG6HyvPooKKkvRohh24KUhn6BUGBxk840xBe723lNiymKCF1gctGXS2jjb4 eihgD0uR4Kg2aiXaidCbF1x3CeKh2T006N5pM3n3dKcZKO44cYlocONyOuIZ+kzyDaTy xVbhrCYIEc0qjMAJhbtrFMTHBosfpbbAbl2SwnMG9TRVXfe12e6Qz+FEFyHWWsj1cmlq 1U4SdsvMMcoOh5lIZt7Pfna2rO/ZuJU1HVABNm9Ai8DerF05/X+FII0+no+Dw5tJrItU FdsHBxu8j9y1epabqOUA+xbVA4EuoHTNFBEF6305x6EvXdccRKHTZhdoUXblrfdbaZg6 Fpcg== X-Gm-Message-State: ALoCoQlh1xP8XEPgNbBeqvr9tvPkfOQgUT5t9LxPqb1XqQJ8pHTfl1iijoPtxXeZLlqEPp/RDdPf X-Received: by 10.60.42.161 with SMTP id p1mr13398993oel.7.1431459520752; Tue, 12 May 2015 12:38:40 -0700 (PDT) MIME-Version: 1.0 Received: by 10.202.108.149 with HTTP; Tue, 12 May 2015 12:38:20 -0700 (PDT) In-Reply-To: References: <20150512171640.GA32606@savin.petertodd.org> From: Jeff Garzik Date: Tue, 12 May 2015 12:38:20 -0700 Message-ID: To: gabe appleton Content-Type: multipart/alternative; boundary=001a11c24d803ec3420515e7a432 X-Spam-Score: -0.6 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain -0.0 SPF_PASS SPF: sender matches SPF record 1.0 HTML_MESSAGE BODY: HTML included in message -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1YsG0s-0004vk-GJ Cc: Bitcoin Dev Subject: Re: [Bitcoin-development] Proposed additional options for pruned nodes X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 May 2015 19:38:48 -0000 --001a11c24d803ec3420515e7a432 Content-Type: text/plain; charset=UTF-8 One general problem is that security is weakened when an attacker can DoS a small part of the chain by DoS'ing a small number of nodes - yet the impact is a network-wide DoS because nobody can complete a sync. On Tue, May 12, 2015 at 12:24 PM, gabe appleton wrote: > 0, 1, 3, 4, 5, 6 can be solved by looking at chunks chronologically. Ie, > give the signed (by sender) hash of the first and last block in your range. > This is less data dense than the idea above, but it might work better. > > That said, this is likely a less secure way to do it. To improve upon > that, a node could request a block of random height within that range and > verify it, but that violates point 2. And the scheme in itself definitely > violates point 7. > On May 12, 2015 3:07 PM, "Gregory Maxwell" wrote: > >> It's a little frustrating to see this just repeated without even >> paying attention to the desirable characteristics from the prior >> discussions. >> >> Summarizing from memory: >> >> (0) Block coverage should have locality; historical blocks are >> (almost) always needed in contiguous ranges. Having random peers >> with totally random blocks would be horrific for performance; as you'd >> have to hunt down a working peer and make a connection for each block >> with high probability. >> >> (1) Block storage on nodes with a fraction of the history should not >> depend on believing random peers; because listening to peers can >> easily create attacks (e.g. someone could break the network; by >> convincing nodes to become unbalanced) and not useful-- it's not like >> the blockchain is substantially different for anyone; if you're to the >> point of needing to know coverage to fill then something is wrong. >> Gaps would be handled by archive nodes, so there is no reason to >> increase vulnerability by doing anything but behaving uniformly. >> >> (2) The decision to contact a node should need O(1) communications, >> not just because of the delay of chasing around just to find who has >> someone; but because that chasing process usually makes the process >> _highly_ sybil vulnerable. >> >> (3) The expression of what blocks a node has should be compact (e.g. >> not a dense list of blocks) so it can be rumored efficiently. >> >> (4) Figuring out what block (ranges) a peer has given should be >> computationally efficient. >> >> (5) The communication about what blocks a node has should be compact. >> >> (6) The coverage created by the network should be uniform, and should >> remain uniform as the blockchain grows; ideally it you shouldn't need >> to update your state to know what blocks a peer will store in the >> future, assuming that it doesn't change the amount of data its >> planning to use. (What Tier Nolan proposes sounds like it fails this >> point) >> >> (7) Growth of the blockchain shouldn't cause much (or any) need to >> refetch old blocks. >> >> I've previously proposed schemes which come close but fail one of the >> above. >> >> (e.g. a scheme based on reservoir sampling that gives uniform >> selection of contiguous ranges, communicating only 64 bits of data to >> know what blocks a node claims to have, remaining totally uniform as >> the chain grows, without any need to refetch -- but needs O(height) >> work to figure out what blocks a peer has from the data it >> communicated.; or another scheme based on consistent hashes that has >> log(height) computation; but sometimes may result in a node needing to >> go refetch an old block range it previously didn't store-- creating >> re-balancing traffic.) >> >> So far something that meets all those criteria (and/or whatever ones >> I'm not remembering) has not been discovered; but I don't really think >> much time has been spent on it. I think its very likely possible. >> >> >> ------------------------------------------------------------------------------ >> One dashboard for servers and applications across Physical-Virtual-Cloud >> Widest out-of-the-box monitoring support with 50+ applications >> Performance metrics, stats and reports that give you Actionable Insights >> Deep dive visibility with transaction tracing using APM Insight. >> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y >> _______________________________________________ >> Bitcoin-development mailing list >> Bitcoin-development@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/bitcoin-development >> > > > ------------------------------------------------------------------------------ > One dashboard for servers and applications across Physical-Virtual-Cloud > Widest out-of-the-box monitoring support with 50+ applications > Performance metrics, stats and reports that give you Actionable Insights > Deep dive visibility with transaction tracing using APM Insight. > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ --001a11c24d803ec3420515e7a432 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
One general problem is that security is weakened when an a= ttacker can DoS a small part of the chain by DoS'ing a small number of = nodes - yet the impact is a network-wide DoS because nobody can complete a = sync.


On Tue, May 12, 2015 at 12:24 PM, gabe appleton = <gappleto97@gm= ail.com> wrote:

0, 1, 3, 4, 5, 6 can be solved by looking at chunks chronologically. Ie= , give the signed (by sender) hash of the first and last block in your rang= e. This is less data dense than the idea above, but it might work better. <= /p>

That said, this is likely a less secure way to do it. To imp= rove upon that, a node could request a block of random height within that r= ange and verify it, but that violates point 2. And the scheme in itself def= initely violates point 7.

On May 12, 2015 3:07 PM, "Gregory Maxwell&q= uot; <gmaxwell@g= mail.com> wrote:
It's a little frustrating to see this just repeated without even paying attention to the desirable characteristics from the prior
discussions.

Summarizing from memory:

(0) Block coverage should have locality; historical blocks are
(almost) always needed in contiguous ranges.=C2=A0 =C2=A0Having random peer= s
with totally random blocks would be horrific for performance; as you'd<= br> have to hunt down a working peer and make a connection for each block
with high probability.

(1) Block storage on nodes with a fraction of the history should not
depend on believing random peers; because listening to peers can
easily create attacks (e.g. someone could break the network; by
convincing nodes to become unbalanced) and not useful-- it's not like the blockchain is substantially different for anyone; if you're to the<= br> point of needing to know coverage to fill then something is wrong.
Gaps would be handled by archive nodes, so there is no reason to
increase vulnerability by doing anything but behaving uniformly.

(2) The decision to contact a node should need O(1) communications,
not just because of the delay of chasing around just to find who has
someone; but because that chasing process usually makes the process
_highly_ sybil vulnerable.

(3) The expression of what blocks a node has should be compact (e.g.
not a dense list of blocks) so it can be rumored efficiently.

(4) Figuring out what block (ranges) a peer has given should be
computationally efficient.

(5) The communication about what blocks a node has should be compact.

(6) The coverage created by the network should be uniform, and should
remain uniform as the blockchain grows; ideally it you shouldn't need to update your state to know what blocks a peer will store in the
future, assuming that it doesn't change the amount of data its
planning to use. (What Tier Nolan proposes sounds like it fails this
point)

(7) Growth of the blockchain shouldn't cause much (or any) need to
refetch old blocks.

I've previously proposed schemes which come close but fail one of the a= bove.

(e.g. a scheme based on reservoir sampling that gives uniform
selection of contiguous ranges, communicating only 64 bits of data to
know what blocks a node claims to have, remaining totally uniform as
the chain grows, without any need to refetch -- but needs O(height)
work to figure out what blocks a peer has from the data it
communicated.;=C2=A0 =C2=A0or another scheme based on consistent hashes tha= t has
log(height) computation; but sometimes may result in a node needing to
go refetch an old block range it previously didn't store-- creating
re-balancing traffic.)

So far something that meets all those criteria (and/or whatever ones
I'm not remembering) has not been discovered; but I don't really th= ink
much time has been spent on it. I think its very likely possible.

---------------------------------------------------------------------------= ---
One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment

-----------------------------------------------------------= -------------------
One dashboard for servers and applications across Physical-Virtual-Cloud Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
= _______________________________________________
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment




--
Jeff Garzik
Bitcoin core developer and open sourc= e evangelist
BitPay, Inc. =C2=A0 =C2=A0 =C2=A0https://bitpay.com/
--001a11c24d803ec3420515e7a432--