From: Richard Moore <me@ricmoo.com>
Content-Type: multipart/alternative;
	boundary="Apple-Mail=_0604438E-29BB-487A-830E-47C672397825"
Message-Id: <34798C1C-FDA7-4A4C-B136-DBD4E59C254D@ricmoo.com>
Date: Wed, 4 Jun 2014 15:30:10 -0400
To: bitcoin-development@lists.sourceforge.net
Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\))
Subject: [Bitcoin-development] Future Feature Proposal - getgist
Precedence: list


--Apple-Mail=_0604438E-29BB-487A-830E-47C672397825
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=windows-1252

Bitcoin development team,

I recently started implementing my own Python full-node, and had an =
idea, so I=92m prowling through BIP 001 for this proposal, which says to =
e-mail you kind folks to make sure the idea is original (enough) and =
that there aren=92t other existing means to accomplish what I was =
thinking. :)

The only way to grab all the headers seems to be to serially get one, =
then the next and so on, as you need the last hash of a headers() call =
to the next getheaders(). But we are on a peer-to-peer network, =
potentially able to getheaders() from many peers simultaneously, if we =
only knew the hash to look for.

What I was thinking is something to the effect of a getgist() command, =
fully backward compatible (any node not understanding it, can silently =
ignore it=85 Otherwise version or services could be used to announce the =
capability, but that seems like a little overkill). The inputs to =
getgist() would be similar to getheaders(); version, hash_count, =
block_locator_hash, stop_hash and an additional field, segment_count. =
The response would be a normal headers() message, except, not sequential =
block headers=85 Rather they would be spaced out, preferably =
2000-block-hash-aligned from the first block hash. So, for example, if =
you have a blockchain with 198,005 blocks, and you passed it the block =
locator of height 0 (the genesis block), and a segment_count of 25, you =
would expect (approximately, the actual algorithm needs to be figured =
out), the block hashes at the following 25 (segment_count) heights:

1, 8000, 16000, 24000, 32000, 40000, 48000, 56000, 64000, 72000, 80000, =
88000, 96000, 104000, 112000, 120000, 128000, 136000, 144000, 152000, =
160000, 168000, 176000, 184000, 192000

Which can now be spread across 25 different nodes, fetching the block =
headers (albeit, out of order, possibly increasing the complexity of the =
local block chain database) but vastly increasing the speed the whole =
blockchain can have all headers synced.

I still need to perform some tests to see what type of speed gains there =
are, but I would suspect it should be segment_count times faster.

Existing methods could be to use checkpoint-ish nodes or bootstrap data =
files, but these begin relying on semi-cetralizenesses.

Ideas? Suggestions? Concerns? Prior it-ain=92t-gonna-works?

Thanks!

RicMoo


.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=
=B7.=B8=B8.=B7=B4=AF`=B7.=B8><(((=BA>

Richard Moore ~ Founder
Genetic Mistakes Software inc.
phone: (778) 882-6125
email: ricmoo@geneticmistakes.com
www: http://GeneticMistakes.com


--Apple-Mail=_0604438E-29BB-487A-830E-47C672397825
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=windows-1252

<html><head><meta http-equiv=3D"Content-Type" content=3D"text/html =
charset=3Dwindows-1252"></head><body style=3D"word-wrap: break-word; =
-webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div =
style=3D"font-size: 13px;">Bitcoin development team,</div><div =
style=3D"font-size: 13px;"><br></div><div style=3D"font-size: 13px;">I =
recently started implementing my own Python full-node, and had an idea, =
so I=92m prowling through BIP 001 for this proposal, which says to =
e-mail you kind folks to make sure the idea is original (enough) and =
that there aren=92t other existing means to accomplish what I was =
thinking. :)</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">The only way to grab all the headers seems to =
be to serially get one, then the next and so on, as you need the last =
hash of a headers() call to the next getheaders(). But we are on a =
peer-to-peer network, potentially able to getheaders() from many peers =
simultaneously, if we only knew the hash to look for.</div><div =
style=3D"font-size: 13px;"><br></div><div style=3D"font-size: =
13px;">What I was thinking is something to the effect of a getgist() =
command, fully backward compatible (any node not understanding it, can =
silently ignore it=85 Otherwise version or services could be used to =
announce the capability, but that seems like a little overkill). The =
inputs to getgist() would be similar to getheaders(); version, =
hash_count, block_locator_hash, stop_hash and an additional field, =
segment_count. The response would be a normal headers() message, except, =
not sequential block headers=85 Rather they would be spaced out, =
preferably 2000-block-hash-aligned from the first block hash. So, for =
example, if you have a blockchain with 198,005 blocks, and you passed it =
the block locator of height 0 (the genesis block), and a segment_count =
of 25, you would expect (approximately, the actual algorithm needs to be =
figured out), the block hashes at the following 25 (segment_count) =
heights:</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;"><div style=3D"margin: 0px; font-size: 11px; =
font-family: Menlo;">1, 8000, 16000, 24000, 32000, 40000, 48000, 56000, =
64000, 72000, 80000, 88000, 96000, 104000, 112000, 120000, 128000, =
136000, 144000, 152000, 160000, 168000, 176000, 184000, =
192000</div></div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">Which can now be spread across 25 different =
nodes, fetching the block headers (albeit, out of order, possibly =
increasing the complexity of the local block chain database) but vastly =
increasing the speed the whole blockchain can have all headers =
synced.</div><div style=3D"font-size: 13px;"><br></div><div =
style=3D"font-size: 13px;">I still need to perform some tests to see =
what type of speed gains there are, but I would suspect it should be =
segment_count times faster.</div><div style=3D"font-size: =
13px;"><br></div><div style=3D"font-size: 13px;">Existing methods could =
be to use checkpoint-ish nodes or bootstrap data files, but these begin =
relying on semi-cetralizenesses.</div><div style=3D"font-size: =
13px;"><br></div><div style=3D"font-size: 13px;">Ideas? Suggestions? =
Concerns? Prior it-ain=92t-gonna-works?</div><div style=3D"font-size: =
13px;"><br></div><div style=3D"font-size: 13px;">Thanks!</div><div =
style=3D"font-size: 13px;"><br></div><div style=3D"font-size: =
13px;">RicMoo</div><div><br></div><div><br></div><div =
apple-content-edited=3D"true">
<span class=3D"Apple-style-span" style=3D"border-collapse: separate; =
border-spacing: =
0px;">.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8=B8.=B7=B4=
=AF`=B7.=B8=B8.=B7=B4=AF`=B7.=B8&gt;&lt;(((=BA&gt;<br><br>Richard Moore =
~ Founder<br>Genetic Mistakes Software inc.<br>phone: (778) =
882-6125<br>email:&nbsp;<a =
href=3D"mailto:ricmoo@geneticmistakes.com">ricmoo@geneticmistakes.com</a><=
br>www:&nbsp;<a =
href=3D"http://GeneticMistakes.com/">http://GeneticMistakes.com</a></span>=

</div>
<br></body></html>=

--Apple-Mail=_0604438E-29BB-487A-830E-47C672397825--