1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
|
Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
helo=mx.sourceforge.net)
by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <mh.in.england@gmail.com>) id 1UWUTZ-0006Yi-50
for bitcoin-development@lists.sourceforge.net;
Sun, 28 Apr 2013 16:29:21 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.219.47 as permitted sender)
client-ip=209.85.219.47; envelope-from=mh.in.england@gmail.com;
helo=mail-oa0-f47.google.com;
Received: from mail-oa0-f47.google.com ([209.85.219.47])
by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1UWUTV-00026e-1d
for bitcoin-development@lists.sourceforge.net;
Sun, 28 Apr 2013 16:29:21 +0000
Received: by mail-oa0-f47.google.com with SMTP id n9so5305432oag.20
for <bitcoin-development@lists.sourceforge.net>;
Sun, 28 Apr 2013 09:29:11 -0700 (PDT)
MIME-Version: 1.0
X-Received: by 10.182.226.136 with SMTP id rs8mr27034059obc.50.1367166551674;
Sun, 28 Apr 2013 09:29:11 -0700 (PDT)
Sender: mh.in.england@gmail.com
Received: by 10.76.167.169 with HTTP; Sun, 28 Apr 2013 09:29:11 -0700 (PDT)
In-Reply-To: <CAPg+sBjSe23eADMxu-1mx0Kg2LGkN+BSNByq0PtZcMxAMh0uTg@mail.gmail.com>
References: <CAPg+sBjSe23eADMxu-1mx0Kg2LGkN+BSNByq0PtZcMxAMh0uTg@mail.gmail.com>
Date: Sun, 28 Apr 2013 18:29:11 +0200
X-Google-Sender-Auth: ujnmoW7a4z2XRLETJQ9v-gooNpA
Message-ID: <CANEZrP3FA-5z3gAC1aYbG2EOKM2eDyv7zX3S9+ia2ZJ0LPkKiA@mail.gmail.com>
From: Mike Hearn <mike@plan99.net>
To: Pieter Wuille <pieter.wuille@gmail.com>
Content-Type: multipart/alternative; boundary=001a11c3018aa9592104db6e44fc
X-Spam-Score: -0.5 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(mh.in.england[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
1.0 HTML_MESSAGE BODY: HTML included in message
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1UWUTV-00026e-1d
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Service bits for pruned nodes
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Sun, 28 Apr 2013 16:29:21 -0000
--001a11c3018aa9592104db6e44fc
Content-Type: text/plain; charset=UTF-8
I'd imagined that nodes would be able to pick their own ranges to keep
rather than have fixed chosen intervals. "Everything or two weeks" is
rather restrictive - presumably node operators are constrained by physical
disk space, which means the quantity of blocks they would want to keep can
vary with sizes of blocks, cost of storage, etc.
Adding new fields to the addr message and relaying those fields to newer
nodes means every node could advertise the height at which it pruned. I
know it means a longer time before the data is available everywhere vs
service bits, but it seems like most nodes won't be pruning right away
anyway. There's plenty of time for upgrades. If an old node connected to a
new node and getdata-d blocks that had been pruned, immediate disconnection
should make the old node go find a different one. It means the combination
of old node+not run for a long time might take a while before it can find a
node that has what it wants, but that doesn't seem like a big deal.
What is the use case for NODE_VALIDATE? Nodes that throw away blocks almost
immediately? Why would a node do that?
On Sun, Apr 28, 2013 at 5:51 PM, Pieter Wuille <pieter.wuille@gmail.com>wrote:
> Hello all,
>
> I think it is time to move forward with pruning nodes, i.e. nodes that
> fully validate and relay blocks and transactions, but which do not keep
> (all) historic blocks around, and thus cannot be queried for these.
>
> The biggest roadblock is making sure new and old nodes that start up are
> able to find nodes to synchronize from. To help them find peers, I would
> like to propose adding two extra service bits to the P2P protocol:
> * NODE_VALIDATE: relay and validate blocks and transactions, but is only
> guaranteed to answer getdata requests for (recently) relayed blocks and
> transactions, and mempool transactions.
> * NODE_BLOCKS_2016: can be queried for the last 2016 blocks, but without
> guarantee for relaying/validating new blocks and transactions.
> * NODE_NETWORK (which existed before) will imply NODE_VALIDATE and
> guarantee availability of all historic blocks.
>
> The idea is to separate the different responsibilities of network nodes
> into separate bits, so they can - at some point - be
> implemented independently. Perhaps we want more than just one degree (2016
> blocks), maybe also 144 or 210000, but those can be added later if
> necessary. I monitored the frequency of block depths requested from my
> public node, and got this frequency distribution:
> http://bitcoin.sipa.be/depth-small.png so it seems 2016 nicely matches
> the set of frequently-requested blocks (indicating that few nodes are
> offline for more than 2 weeks consecutively.
>
> I'll write a BIP to formalize this, but wanted to get an idea of how much
> support there is for a change like this.
>
> Cheers,
>
> --
> Pieter
>
>
>
>
>
> ------------------------------------------------------------------------------
> Try New Relic Now & We'll Send You this Cool Shirt
> New Relic is the only SaaS-based application performance monitoring service
> that delivers powerful full stack analytics. Optimize and monitor your
> browser, app, & servers with just a few lines of code. Try New Relic
> and get this awesome Nerd Life shirt! http://p.sf.net/sfu/newrelic_d2d_apr
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
>
--001a11c3018aa9592104db6e44fc
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">I'd imagined that nodes would be able to pick their ow=
n ranges to keep rather than have fixed chosen intervals. "Everything =
or two weeks" is rather restrictive - presumably node operators are co=
nstrained by physical disk space, which means the quantity of blocks they w=
ould want to keep can vary with sizes of blocks, cost of storage, etc.<div>
<br></div><div style>Adding new fields to the addr message and relaying tho=
se fields to newer nodes means every node could advertise the height at whi=
ch it pruned. I know it means a longer time before the data is available ev=
erywhere vs service bits, but it seems like most nodes won't be pruning=
right away anyway. There's plenty of time for upgrades. If an old node=
connected to a new node and getdata-d blocks that had been pruned, immedia=
te disconnection should make the old node go find a different one. It means=
the combination of old node+not run for a long time might take a while bef=
ore it can find a node that has what it wants, but that doesn't seem li=
ke a big deal.</div>
<div style><br></div><div style>What is the use case for NODE_VALIDATE? Nod=
es that throw away blocks almost immediately? Why would a node do that?</di=
v></div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">On Su=
n, Apr 28, 2013 at 5:51 PM, Pieter Wuille <span dir=3D"ltr"><<a href=3D"=
mailto:pieter.wuille@gmail.com" target=3D"_blank">pieter.wuille@gmail.com</=
a>></span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"ltr">Hello all,<div><br></div><d=
iv>I think it is time to move forward with pruning nodes, i.e. nodes that f=
ully validate and relay blocks and transactions, but which do not keep (all=
) historic blocks around, and thus cannot be queried for these.</div>
<div><br></div><div>The biggest roadblock is making sure new and old nodes =
that start up are able to find nodes to synchronize from. To help them find=
peers, I would like to propose adding two extra service bits to the P2P pr=
otocol:</div>
<div>* NODE_VALIDATE: relay and validate blocks and transactions, but is on=
ly guaranteed to answer getdata requests for (recently) relayed blocks and =
transactions, and mempool transactions.</div><div>* NODE_BLOCKS_2016: can b=
e queried for the last 2016 blocks, but without guarantee for relaying/vali=
dating new blocks and transactions.</div>
<div>* NODE_NETWORK (which existed before) will imply NODE_VALIDATE and gua=
rantee availability of all historic blocks.</div><div><br></div><div>The id=
ea is to separate the different=C2=A0responsibilities=C2=A0of network nodes=
into separate bits, so they can - at some point - be implemented=C2=A0inde=
pendently. Perhaps we want more than just one degree (2016 blocks), maybe a=
lso 144 or 210000, but those can be added later if necessary. I monitored t=
he frequency of block depths requested from my public node, and got this fr=
equency distribution:=C2=A0<a href=3D"http://bitcoin.sipa.be/depth-small.pn=
g" target=3D"_blank">http://bitcoin.sipa.be/depth-small.png</a>=C2=A0so it =
seems 2016 nicely matches the set of frequently-requested blocks (indicatin=
g that few nodes are offline for more than 2 weeks=C2=A0consecutively.</div=
>
<div><br></div><div>I'll write a BIP to formalize this, but wanted to g=
et an idea of how much support there is for a change like this.</div><div><=
br></div><div>Cheers,</div><span class=3D"HOEnZb"><font color=3D"#888888"><=
div>
<br></div><div>
--=C2=A0</div><div>Pieter</div><div>=C2=A0</div><div><br></div><div><br></d=
iv></font></span></div>
<br>-----------------------------------------------------------------------=
-------<br>
Try New Relic Now & We'll Send You this Cool Shirt<br>
New Relic is the only SaaS-based application performance monitoring service=
<br>
that delivers powerful full stack analytics. Optimize and monitor your<br>
browser, app, & servers with just a few lines of code. Try New Relic<br=
>
and get this awesome Nerd Life shirt! <a href=3D"http://p.sf.net/sfu/newrel=
ic_d2d_apr" target=3D"_blank">http://p.sf.net/sfu/newrelic_d2d_apr</a><br>_=
______________________________________________<br>
Bitcoin-development mailing list<br>
<a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-develo=
pment@lists.sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development=
" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
velopment</a><br>
<br></blockquote></div><br></div>
--001a11c3018aa9592104db6e44fc--
|
