Content-Type: multipart/alternative;
	boundary="Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46"
Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\))
From: Richard Moore <me@ricmoo.com>
In-Reply-To: <CANEZrP1vVY1jnV8Kdg5OWEt=Rba2PPcs3ke4tMWc6fS4wGPpOw@mail.gmail.com>
Date: Thu, 5 Jun 2014 13:43:38 -0400
Message-Id: <2ADB9019-825E-410A-ADED-A7237CBC323C@ricmoo.com>
References: <34798C1C-FDA7-4A4C-B136-DBD4E59C254D@ricmoo.com>
	<CANEZrP1vVY1jnV8Kdg5OWEt=Rba2PPcs3ke4tMWc6fS4wGPpOw@mail.gmail.com>
To: Mike Hearn <mike@plan99.net>
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Future Feature Proposal - getgist
Precedence: list


--Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252

I was considering names like getcheckpoints() to use the terminology =
that already seemed to be in place, but they were too long :)

I have been using getheaders() in my thick client to quickly grab all =
the headers before downloading the full blocks since I can grab more at =
a time. Even with getblocks(), there is the case for a  getgist() call. =
Right now you call getblocks(), which can take some time to get the =
corresponding inv(), at which time you can then start the call to =
getdata() as well as the next call to getblocks().

With a gist, for example of segment_count 50, you could call getgist(), =
then with the response, you could request 50 getblocks() each with a =
block_locator of 1 hash from the gist (and optimally the stop_hash of =
the next hash in the gist) to 50 different peers, providing 25,000 (50 x =
500) block hashes.

Currently:
>>> getblocks()
<<< inv()
>>> getdata()
<<< block(), block(), block(), =85 (x 500)

Saturates one peer, while leaving the rest un-used. Step 1 and 2 can be =
repeated and dispatched to different peers, but there is still the =
latency between the two calls.

Gist:
>>> getgist()
<<< inv()
>>> getblocks(), getblocks(), getblocks(), =85 (x segment_count, 1 per =
peer)
<<< inv(), inv(), inv(), =85 (x segment_count, 1 per peer)
>>> getdata(), getdata(), getdata(), =85 (x segment_count, 1 per peer)
<<< block(), block(), block(), =85 (x (500 * segment_count), ie. 500 in =
per peer)

Each peer can be saturated.

I will try to run some experiments this weekend to get numbers as to =
whether there is actually any performance improvement using a gist, or =
whether the getdata(), block() latency ends up dominating the time =
anyways.


RicMoo


On Jun 4, 2014, at 11:42 PM, Mike Hearn <mike@plan99.net> wrote:

> Why do you want to optimise this? getheaders is used by SPV clients =
not full nodes. SPV clients like bitcoinj can and do simply ship with =
gist files in them, then getheaders from the last "checkpoint"   (I wish =
I hadn't reused terminology like that but this is what bitcoinj calls =
them).
>=20
> In practice when I look at performance issues with current SPV =
clients, getheaders speed is not on my radar.

.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=
=B7.=B8=B8.=B7=B4=AF`=B7.=B8><(((=BA>

Richard Moore ~ Founder
Genetic Mistakes Software inc.
phone: (778) 882-6125
email: ricmoo@geneticmistakes.com
www: http://GeneticMistakes.com


--Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=windows-1252

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div =
style=3D"font-size: 13px;">I was considering names like getcheckpoints() =
to use the terminology that already seemed to be in place, but they were =
too long :)</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">I have been using getheaders() in my thick =
client to quickly grab all the headers before downloading the full =
blocks since I can grab more at a time. Even with getblocks(), there is =
the case for a &nbsp;getgist() call. Right now you call getblocks(), =
which can take some time to get the corresponding inv(), at which time =
you can then start the call to getdata() as well as the next call to =
getblocks().</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">With a gist, for example of segment_count 50, =
you could call getgist(), then with the response, you could request 50 =
getblocks() each with a block_locator of 1 hash from the gist (and =
optimally the stop_hash of the next hash in the gist) to 50 different =
peers, providing 25,000 (50 x 500) block hashes.</div><div =
style=3D"font-size: 13px;"><br></div><div style=3D"font-size: =
13px;">Currently:</div><div style=3D"font-size: =
13px;"><ol><li>&gt;&gt;&gt; getblocks()</li><li>&lt;&lt;&lt; =
inv()</li><li>&gt;&gt;&gt; getdata()</li><li>&lt;&lt;&lt; block(), =
block(), block(), =85 (x 500)</li></ol></div><div style=3D"font-size: =
13px;"><br></div><div style=3D"font-size: 13px;">Saturates one peer, =
while leaving the rest un-used. Step 1 and 2 can be repeated and =
dispatched to different peers, but there is still the latency between =
the two calls.</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">Gist:</div><div style=3D"font-size: =
13px;"><ol><li>&gt;&gt;&gt; getgist()</li><li>&lt;&lt;&lt; =
inv()</li><li>&gt;&gt;&gt; getblocks(), getblocks(), getblocks(), =85 (x =
segment_count, 1 per peer)</li><li>&lt;&lt;&lt; inv(), inv(), inv(), =85 =
(x segment_count, 1 per peer)</li><li>&gt;&gt;&gt; getdata(), getdata(), =
getdata(), =85 (x segment_count, 1 per peer)</li><li>&lt;&lt;&lt; =
block(), block(), block(), =85 (x (500 * segment_count), ie. 500 in per =
peer)</li></ol></div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">Each peer can be saturated.</div><div =
style=3D"font-size: 13px;"><br></div><div style=3D"font-size: 13px;">I =
will try to run some experiments this weekend to get numbers as to =
whether there is actually any performance improvement using a gist, or =
whether the getdata(), block() latency ends up dominating the time =
anyways.</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;"><br></div><div style=3D"font-size: =
13px;">RicMoo</div><div><div><br></div><div><br></div><div>On Jun 4, =
2014, at 11:42 PM, Mike Hearn &lt;<a =
href=3D"mailto:mike@plan99.net">mike@plan99.net</a>&gt; wrote:</div><br =
class=3D"Apple-interchange-newline"><blockquote type=3D"cite"><div =
dir=3D"ltr">Why do you want to optimise this? getheaders is used by SPV =
clients not full nodes. SPV clients like bitcoinj can and do simply ship =
with gist files in them, then getheaders from the last "checkpoint" =
&nbsp; (I wish I hadn't reused terminology like that but this is what =
bitcoinj calls them).<div>
<br></div><div>In practice when I look at performance issues with =
current SPV clients, getheaders speed is not on my radar.</div></div>
</blockquote></div><br><div apple-content-edited=3D"true">
<span class=3D"Apple-style-span" style=3D"border-collapse: separate; =
border-spacing: =
0px;">.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=
=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8&gt;&lt;(((=BA&gt;<br><br>Richard Moore =
~ Founder<br>Genetic Mistakes Software inc.<br>phone: (778) =
882-6125<br>email:&nbsp;<a =
href=3D"mailto:ricmoo@geneticmistakes.com">ricmoo@geneticmistakes.com</a><=
br>www:&nbsp;<a =
href=3D"http://GeneticMistakes.com/">http://GeneticMistakes.com</a></span>=

</div>
<br></body></html>=

--Apple-Mail=_63C57CA8-3DBB-40F8-8AED-41399E2FEF46--