1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
|
Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194]
helo=mx.sourceforge.net)
by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <arklan.uthoslin@gmail.com>) id 1TLjRe-0005yK-Ac
for bitcoin-development@lists.sourceforge.net;
Tue, 09 Oct 2012 23:42:38 +0000
Received-SPF: pass (sog-mx-4.v43.ch3.sourceforge.com: domain of gmail.com
designates 209.85.219.47 as permitted sender)
client-ip=209.85.219.47; envelope-from=arklan.uthoslin@gmail.com;
helo=mail-oa0-f47.google.com;
Received: from mail-oa0-f47.google.com ([209.85.219.47])
by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
(Exim 4.76) id 1TLjRd-0002tw-7x
for bitcoin-development@lists.sourceforge.net;
Tue, 09 Oct 2012 23:42:38 +0000
Received: by mail-oa0-f47.google.com with SMTP id h1so6166712oag.34
for <bitcoin-development@lists.sourceforge.net>;
Tue, 09 Oct 2012 16:42:31 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.182.78.228 with SMTP id e4mr4246903obx.77.1349826151520; Tue,
09 Oct 2012 16:42:31 -0700 (PDT)
Received: by 10.76.152.97 with HTTP; Tue, 9 Oct 2012 16:42:31 -0700 (PDT)
In-Reply-To: <CA+8xBpchRLVQW4Rv2RQhsRF716cCTmJLuhZeuvCWtWB4sZX2Sw@mail.gmail.com>
References: <CA+8xBpchRLVQW4Rv2RQhsRF716cCTmJLuhZeuvCWtWB4sZX2Sw@mail.gmail.com>
Date: Tue, 9 Oct 2012 17:42:31 -0600
Message-ID: <CAGg41SWAxhGwmMQCnzsBU_0Z=han2JvbTn3CaTFZxqaaTb+AHw@mail.gmail.com>
From: Arklan Uth Oslin <arklan.uthoslin@gmail.com>
To: Jeff Garzik <jgarzik@exmulti.com>
Content-Type: multipart/alternative; boundary=f46d0445170d45180104cba8e4e2
X-Spam-Score: -0.6 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
(arklan.uthoslin[at]gmail.com)
-0.0 SPF_PASS SPF: sender matches SPF record
1.0 HTML_MESSAGE BODY: HTML included in message
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
0.1 DKIM_SIGNED Message has a DKIM or DK signature,
not necessarily valid
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1TLjRd-0002tw-7x
Cc: Bitcoin Development <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] On bitcoin testing
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Tue, 09 Oct 2012 23:42:38 -0000
--f46d0445170d45180104cba8e4e2
Content-Type: text/plain; charset=ISO-8859-1
thanks for the great reply jeff. i'm going to get a virtual machine set up
on my system later tonight so at the very least, i myself can start testing.
steve - haven't heard from you in almost a week. I'd still really like to
get a look at the test cases and such you set up.
Arklan
----------
As long as there is light, the darkness holds no fear. And yet, even in the
deepest black, there is life. - Arklan Uth Oslin
I want to leave this world the same way I came into it: backwards and on
fire. - Arklan Uth Oslin
On Tue, Oct 9, 2012 at 5:12 PM, Jeff Garzik <jgarzik@exmulti.com> wrote:
> Copying from a response posted to "Bitcoin software testing effort"
> https://bitcointalk.org/index.php?topic=117487.0 as it is relevant to
> a recent thread here...
>
> Any level of testing is useful and appreciated. Various types of
> testing that are helpful:
>
> * "it works" testing: Simply run the latest Release Candidate (or
> latest version, if released). Make sure all the basics work (for
> whatever definition of "basics" you desire). This is the level most
> accessible to casual users.
> * Major features testing: Develop a short checklist of must-work
> features, and organize volunteers to work together and go through that
> checklist, item by item. Test each major feature on each major
> platform.
> * Stress and fuzz testing: Attempt to "stress" the system somehow, or
> randomly corrupt bits of data. See what breaks.
> * Regression testing: Record bugs fixed, and develop automated test
> cases that successfully reproduce the bugs on older versions, and
> verify newer versions remain fixed.
> * Unit function testing: Rigorously exercise each C++ class to ensure
> it behaves as expected at a micro level.
> * Full peer automated testing: Automated testing of RPC and P2P
> functions is non-existent, because of the difficulty in doing so.
> Find a solution to this problem.
> * Data-driven tests: If possible, write software-neutral, data-driven
> tests. This enables clients other than the reference one (Satoshi
> client) to be tested. Embed tests in testnet3 chain, if possible.
>
>
> The community at large can be a big help simply by doing the first
> item: download and run the Release Candidates and the latest version,
> and report any problems. Even reporting success is fine by me, for
> example: "Version 0.7.1 works for me on Windows 7/32-bit" posted on a
> forum thread.
>
> It is always very difficult to organize any sort of testing regime
> with open source volunteers that come and go. Each volunteer chooses
> their level of involvement. Any amount of testing and test-case
> writing, large or small, is helpful to bitcoin.
>
> --
> Jeff Garzik
> exMULTI, Inc.
> jgarzik@exmulti.com
>
>
> ------------------------------------------------------------------------------
> Don't let slow site performance ruin your business. Deploy New Relic APM
> Deploy New Relic app performance management and know exactly
> what is happening inside your Ruby, Python, PHP, Java, and .NET app
> Try New Relic at no cost today and get our sweet Data Nerd shirt too!
> http://p.sf.net/sfu/newrelic-dev2dev
> _______________________________________________
> Bitcoin-development mailing list
> Bitcoin-development@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>
--f46d0445170d45180104cba8e4e2
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
thanks for the great reply jeff. i'm going to get a virtual machine set=
up on my system later tonight so at the very least, i myself can start tes=
ting.<div><br></div><div>steve - haven't heard from you in almost a wee=
k. I'd still really like to get a look at the test cases and such you s=
et up.<br clear=3D"all">
<div>=A0</div>
<div>Arklan<br><br>----------<br>
<div>As long as there is light, the darkness holds no fear. And yet, even i=
n the deepest black, there is life. - Arklan Uth Oslin</div></div>
<div>=A0</div>
<div>I want to leave this world the same way I came into it: backwards and =
on fire. - Arklan Uth Oslin</div><br>
<br><br><div class=3D"gmail_quote">On Tue, Oct 9, 2012 at 5:12 PM, Jeff Gar=
zik <span dir=3D"ltr"><<a href=3D"mailto:jgarzik@exmulti.com" target=3D"=
_blank">jgarzik@exmulti.com</a>></span> wrote:<br><blockquote class=3D"g=
mail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-l=
eft:1ex">
Copying from a response posted to "Bitcoin software testing effort&quo=
t;<br>
<a href=3D"https://bitcointalk.org/index.php?topic=3D117487.0" target=3D"_b=
lank">https://bitcointalk.org/index.php?topic=3D117487.0</a> as it is relev=
ant to<br>
a recent thread here...<br>
<br>
Any level of testing is useful and appreciated. =A0Various types of<br>
testing that are helpful:<br>
<br>
* "it works" testing: =A0Simply run the latest Release Candidate =
(or<br>
latest version, if released). =A0Make sure all the basics work (for<br>
whatever definition of "basics" you desire). =A0This is the level=
most<br>
accessible to casual users.<br>
* Major features testing: =A0Develop a short checklist of must-work<br>
features, and organize volunteers to work together and go through that<br>
checklist, item by item. =A0Test each major feature on each major<br>
platform.<br>
* Stress and fuzz testing: =A0Attempt to "stress" the system some=
how, or<br>
randomly corrupt bits of data. =A0See what breaks.<br>
* Regression testing: =A0Record bugs fixed, and develop automated test<br>
cases that successfully reproduce the bugs on older versions, and<br>
verify newer versions remain fixed.<br>
* Unit function testing: =A0Rigorously exercise each C++ class to ensure<br=
>
it behaves as expected at a micro level.<br>
* Full peer automated testing: =A0Automated testing of RPC and P2P<br>
functions is non-existent, because of the difficulty in doing so.<br>
Find a solution to this problem.<br>
* Data-driven tests: If possible, write software-neutral, data-driven<br>
tests. =A0This enables clients other than the reference one (Satoshi<br>
client) to be tested. =A0Embed tests in testnet3 chain, if possible.<br>
<br>
<br>
The community at large can be a big help simply by doing the first<br>
item: =A0download and run the Release Candidates and the latest version,<br=
>
and report any problems. =A0Even reporting success is fine by me, for<br>
example: "Version 0.7.1 works for me on Windows 7/32-bit" posted =
on a<br>
forum thread.<br>
<br>
It is always very difficult to organize any sort of testing regime<br>
with open source volunteers that come and go. =A0Each volunteer chooses<br>
their level of involvement. =A0Any amount of testing and test-case<br>
writing, large or small, is helpful to bitcoin.<br>
<br>
--<br>
Jeff Garzik<br>
exMULTI, Inc.<br>
<a href=3D"mailto:jgarzik@exmulti.com">jgarzik@exmulti.com</a><br>
<br>
---------------------------------------------------------------------------=
---<br>
Don't let slow site performance ruin your business. Deploy New Relic AP=
M<br>
Deploy New Relic app performance management and know exactly<br>
what is happening inside your Ruby, Python, PHP, Java, and .NET app<br>
Try New Relic at no cost today and get our sweet Data Nerd shirt too!<br>
<a href=3D"http://p.sf.net/sfu/newrelic-dev2dev" target=3D"_blank">http://p=
.sf.net/sfu/newrelic-dev2dev</a><br>
_______________________________________________<br>
Bitcoin-development mailing list<br>
<a href=3D"mailto:Bitcoin-development@lists.sourceforge.net">Bitcoin-develo=
pment@lists.sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development=
" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
velopment</a><br>
</blockquote></div><br></div>
--f46d0445170d45180104cba8e4e2--
|
