1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
|
Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
helo=mx.sourceforge.net)
by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <jeremy@taplink.co>) id 1VZQoS-0001dZ-Qc
for bitcoin-development@lists.sourceforge.net;
Thu, 24 Oct 2013 19:43:20 +0000
Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of taplink.co
designates 50.117.27.232 as permitted sender)
client-ip=50.117.27.232; envelope-from=jeremy@taplink.co;
helo=mail.taplink.co;
Received: from mail.taplink.co ([50.117.27.232])
by sog-mx-3.v43.ch3.sourceforge.com with smtp (Exim 4.76)
id 1VZQoQ-0003Ct-Ot for bitcoin-development@lists.sourceforge.net;
Thu, 24 Oct 2013 19:43:20 +0000
Received: from laptop-air ([192.168.168.135]) by mail.taplink.co ;
Thu, 24 Oct 2013 12:50:40 -0700
Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes
To: "Bitcoin Development" <bitcoin-development@lists.sourceforge.net>
References: <791a727f-2188-4848-bd77-ea733c8c5c2c@me.com>
<201310211947.59640.luke@dashjr.org> <52661DB7.7040805@250bpm.com>
<FAE2A544-9295-4087-96DE-D4602D109CBD@me.com>
<CAAS2fgS2f=gYRSr1n2DzK7CUH3xG3J2JMnDreCKBoCcJcpGLxg@mail.gmail.com>
<52662AA1.5050509@250bpm.com>
<CAJHLa0NDus+Ou5go8b_OHvjYW8f7oxXbpxnHTG3dcvxGR49nxA@mail.gmail.com>
<52677CF7.9070609@250bpm.com> <20131023194039.GB31497@petertodd.org>
<52682C24.30700@250bpm.com> <20131023202731.GA31783@petertodd.org>
<CAPg+sBjv5We415atZocrZniKexFnKmUXB+bMC-tSG4ehQK9rwQ@mail.gmail.com>
<5268C632.3030005@250bpm.com>
<CALxbBHVEdTmo1VWL5zj2ZBV6c-Td4TfpbQSQOWuLGbpxHSDRdA@mail.gmail.com>
Date: Thu, 24 Oct 2013 12:43:15 -0700
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: "Jeremy Spilman" <jeremy@taplink.co>
Organization: TapLink
Message-ID: <op.w5g42djqyldrnw@laptop-air>
In-Reply-To: <CALxbBHVEdTmo1VWL5zj2ZBV6c-Td4TfpbQSQOWuLGbpxHSDRdA@mail.gmail.com>
User-Agent: Opera Mail/1.0 (Win32)
oclient: 192.168.168.135#jeremy@taplink.co#465
X-Spam-Score: -2.0 (--)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
-0.0 SPF_PASS SPF: sender matches SPF record
-0.4 RP_MATCHES_RCVD Envelope sender domain matches handover relay
domain
0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked.
See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block
for more information. [URIs: blockexplorer.com]
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1VZQoQ-0003Ct-Ot
Subject: Re: [Bitcoin-development] Revisiting the BIPS process, a proposal
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Thu, 24 Oct 2013 19:43:21 -0000
Thanks Christian, this is a really interesting bit of history. My own
personal experience from when I wrote my own client and BCCAPI-ish server
was that the protocol specification on the Wiki was hugely valuable, and
rarely sent me astray. Supplement that with the occasional questions on
#bitcoin-dev, and then just coding, coding, coding and getting unit tests
to pass.
Nothing compares (IMO) to stepping through your own code watching the unit
tests run, scripts evaluate, calculating transaction hashes for the
different SIGHASH modes, and finally getting your first transaction into
the block chain. I really appreciated the .json files holding the unit
test data, which were easy to load into my own test harness, the tables on
the Wiki showing what the stack should look like at each point in a script
execution, and the diagrams showing transaction signing.
Bitcoin takes some time to "grok" when you first approach; more than a
day, less than a month, and really no amount of reading documentation or
specs will get you to that "ah ha" moment. When the fog lifts and the
blockchain, scripting, signing, and wallet handling really click, suddenly
the bitcoind code (and many other great public sources in just about any
language you could want) actually does starts to feel fairly simple and
obvious. But it certainly doesn't start out that way on day one.
I think the majority of client code development is actually people writing
'agents' not end-user P2P wallets, and they tend to be written to connect
to a single bitcoind acting as a proxy to the network. Even some end-user
wallets work this way! As such, I spent very little time in my own client
writing P2P protocol code, no peer discovery code, no anti-DoS, etc.
Clients like this also don't pose much systemic risk, because they don't
mine, they don't connect directly to external nodes, etc. They can
certainly be used to "cause trouble" though, but so can
'sendrawtransaction'.
I chose to speak the P2P protocol to bitcoind versus using some of the
other options like ZeroMQ, but it still didn't take long to get headers,
blocks, and transactions downloading. I remember getting stuck on the very
first version message, because of missing the checksum and user-agent or
something caused the latest bitcoind to just ignore me. A little wireshark
capture of the exchange between two working bitcoind instances cleared it
right up. I didn't mind the leg work, I don't think everything needs to be
spoon fed, and it's certainly not purposefully obfuscated. Maybe one
exception is the mix-matched endianness will throw you off, especially if
you are developing on LE! Anyway, I have huge respect for how much effort
it takes to keep even small bits of documentation up-to-date. For as
"slow" as bitcoin moves, it's actually moving incredibly fast.
Finally, the bitcoind console and debug logs, as well sites like
blockchain.info and blockexplorer.com are hugely helpful for debugging raw
and live transactions for when you get stuck. There's a surprisingly large
tooling and support ecosystem out there.
Moral of the story, I think, is everything you need is there. No, it's not
all in one place. Yes, it takes time to find it and assimilate all that
knowledge. It also really helps that the community is extremely willing to
help and answer technical questions, and point you in the right direction,
even when you're working on your own private client code. The IRC channel
can certainly be intimidating because it seems like every time I hit enter
to send a question, gmaxwell's respond 300ms later would invoke an
immediate forehead slap and a groan of "shit, I knew/should have known
that, now I feel dumb" ;-) but if you're working on bitcoin, you better
get used to not being the smartest person in the room! The responses I got
were never arrogant or disparaging, but they were straight to-the-point
and surprisingly high quality. Ain't no slouches in that channel, yes you
will have to bring your A-game and you are expected to have "tried first"
before just asking. I have fairly limited experience working on open
source projects, but I'm extremely happy with my experience with the
Bitcoin community and found writing Bitcoin code hugely enjoyable.
The flip side to helping people implement their own clients, agents, or
even miners, is helping people to contribute pulls requests, or at the
very highest echelon, a BIP. If you haven't written any significant
Bitcoin code, you might want to consider investing in that first before
submitting a BIP. :-)
For a BIP to be valuable, often it requires widespread or even consensus
adoption. BIPs are probably not the place to toss just any old 'good idea'
because BIPs impose a cost on all active developers. I want to read and
understand 'all the BIPs' because for the most part they are actually
essential, like, how to handle duplicate transactions in BIP30 - if you
don't read BIP30 you very likely totally miss that, until your code throws
exceptions while processing block 91842.
And perhaps the hardest kind of BIP of all is the "lets get wallets to add
this user-facing feature" where it has no bearing on the blockchain or
transaction processing, it doesn't make the network more resilient or add
crucial functionality for increasing scalability. Kind of like JPK's HD
wallet encryption proposal, which I love, and I tried to contribute to in
the forums, but I can totally understand the headwinds for making progress
on BIPs like that one and BIP39. No one is against it per-say, it's just
much harder to articulate and justify the NEED for everyone to implement,
test, and support this new not-yet-standard, nice-to-have feature. For
those kinds of BIPs you probably have to go out and get some wallets to
implement it, or implement it yourself, to prove the value and kick start
critical mass before you will even get enough support for getting a BIP
number assigned. IMO, it's not a Bad Thing.
TL;DR; The current support systems worked very well for me. I was able to
accomplish all my goals, and I would even say it was a pleasure. Keep a
high bar for assigning BIP numbers. And I hope to be able to jump back in
and do more with Bitcoin soon.
Thanks all, sorry if I'm rambling,
Jeremy Spilman
On Thu, 24 Oct 2013 04:11:05 -0700, Christian Decker
<decker.christian@gmail.com> wrote:
> I'd like to add some historical background about how the "protocol
> specification" came to be in the first place.
>
> A bit over three years [1] ago I started an attempt to document the
> network protocol, by reverse engineering it from the satoshi
> client. My goal, back then, was to enable like-minded engineers to
> create alternative clients and move away from the client-monoculture
> that is still predominant today. It was clear from the beginning that
> it would merely be a reverse engineering effort, and that it would
> likely lag a bit behind the changes in the main client. It was meant
> as a help for engineers that are not well versed in C/C++ to enable
> them to contribute by creating new clients, but the satoshi client
> would always be the de-facto standard.
>
> With the move from Google Code to the Bitcoin.it wiki somehow this
> notion of it being a reverse engineering effort was lost and people
> started assuming that if the behavior of the satoshi client did not
> match the protocol description it was a bug on the client
> side. Instead it is because the reverse engineering of the protocol is
> incorrect or simply missing some details. Although the protocol
> description is far more complete than it was back when we started, I
> still don't feel comfortable giving it the name specification.
>
> I still believe that a client monoculture is bad for the system as a
> whole, because a single bug might bring down the whole network. Giving
> people the necessary tools to implement new clients brings
> stability. I do understand the criticism that writing a specification
> might hinder future development as it restricts the possible changes
> to the protocol, but isn't this already the case as long as we have
> legacy versions of the client participating in the network? I would
> also argue that having a specification allows an application
> independent review of the protocol to identify possible improvements
> and bugs.
>
> I think the protocol description has an important place in the
> development of Bitcoin, so much so that we pushed a long time ago to
> separate protocol version from the client version. I would love to see
> the protocol specification becoming official part of the bitcoin
> github repository, which would ideally be maintained alongside the
> satoshi client to keep it up to date.
>
> Regards,
> Christian Decker
>
> [1] https://bitcointalk.org/index.php?topic=231
> --
> Christian Decker
>
>
|
