1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
|
Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192]
helo=mx.sourceforge.net)
by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
(envelope-from <pete@petertodd.org>) id 1TdiBD-00065b-Gf
for bitcoin-development@lists.sourceforge.net;
Wed, 28 Nov 2012 13:59:59 +0000
Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of petertodd.org
designates 62.13.148.110 as permitted sender)
client-ip=62.13.148.110; envelope-from=pete@petertodd.org;
helo=outmail148110.authsmtp.com;
Received: from outmail148110.authsmtp.com ([62.13.148.110])
by sog-mx-2.v43.ch3.sourceforge.com with esmtp (Exim 4.76)
id 1TdiB7-00051X-Az for bitcoin-development@lists.sourceforge.net;
Wed, 28 Nov 2012 13:59:59 +0000
Received: from mail-c233.authsmtp.com (mail-c233.authsmtp.com [62.13.128.233])
by punt7.authsmtp.com (8.14.2/8.14.2/Kp) with ESMTP id
qASCvJl7065364; Wed, 28 Nov 2012 12:57:19 GMT
Received: from savin (206-248-185-49.dsl.teksavvy.com [206.248.185.49])
(authenticated bits=128)
by mail.authsmtp.com (8.14.2/8.14.2/) with ESMTP id qASCvBiv006778
(version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO);
Wed, 28 Nov 2012 12:57:14 GMT
Date: Wed, 28 Nov 2012 07:57:10 -0500
From: Peter Todd <pete@petertodd.org>
To: Mike Hearn <mike@plan99.net>
Message-ID: <20121128125710.GA9893@savin>
References: <CABsx9T0PsGLEAWRCjEDDFWQrb+DnJWQZ7mFLaZewAEX6vD1eHw@mail.gmail.com>
<201211271703.39282.andyparkins@gmail.com>
<CANEZrP0NZykzrvC1=YZv4czbu+EPaR3qpjQ2WZDsA8DhroR2_g@mail.gmail.com>
<201211271726.56370.andyparkins@gmail.com>
<CANEZrP2xww7=y0gSyW=oB734UJu2K1f5wB+UuovPO8mCENRzmA@mail.gmail.com>
<CABsx9T13C4eFC9caTmCRm==YAESknX8xhhumb7Mg0mZNdzEosQ@mail.gmail.com>
<CANEZrP1wgMgo8N5f97KF6zNaPzoYVDU9Q=YABkz=jvTpM10jKQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
protocol="application/pgp-signature"; boundary="h31gzZEtNLTqOjlF"
Content-Disposition: inline
In-Reply-To: <CANEZrP1wgMgo8N5f97KF6zNaPzoYVDU9Q=YABkz=jvTpM10jKQ@mail.gmail.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
X-Server-Quench: 22801040-395b-11e2-a49c-0025907707a1
X-AuthReport-Spam: If SPAM / abuse - report it at:
http://www.authsmtp.com/abuse
X-AuthRoute: OCd2Yg0TA1ZNQRgX IjsJECJaVQIpKltL GxAVKBZePFsRUQkR
aQdMdAoUFloCAgsB AmQbWlZeVVp7WGY7 aQpXcwdZalRPVwB0
VEFWR1pVCwQmQG1i dVhsC19ydwNBf3Y+ ZkZjVngVXkZydUd9
Rk5JFmQDN3phaTUd TRFcflBJcANIexZF bVd/UyIMLwdSbGoL
NQ4vNDcwO3BTJTpY RgYVKF8UXXNDJjct Qh0EAX0XB0oZQC40
K1QeMFkEG10YNhdQ eXonR1UZOBIJTTVX DkRABjNCYGMMTCcq FR9BNQAA
X-Authentic-SMTP: 61633532353630.1021:706
X-AuthFastPath: 0 (Was 255)
X-AuthSMTP-Origin: 206.248.185.49/587
X-AuthVirus-Status: No virus detected - but ensure you scan with your own
anti-virus system.
X-Spam-Score: -1.5 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
See http://spamassassin.org/tag/ for more details.
-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
sender-domain
-0.0 SPF_PASS SPF: sender matches SPF record
X-Headers-End: 1TdiB7-00051X-Az
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Payment Protocol Proposal:
Invoices/Payments/Receipts
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Wed, 28 Nov 2012 13:59:59 -0000
--h31gzZEtNLTqOjlF
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
On Wed, Nov 28, 2012 at 11:43:19AM +0100, Mike Hearn wrote:
> Peter is correct that there are a few degrees of freedom in protobuf
> serialization, though far fewer than with JSON.
FWIW I re-read the specs again and turns out my memory was wrong. (I
last looked at this about four months ago) Duplicated fields are handled
in a defined manner, with the last field seen in the serialization being
the one whose value is used. Again, repeated fields are treated as
elements of a list, preserving order.
It does raise the interesting question do the implementations that don't
preserve order of unknown fields, preserve the order of multiple unknown
fields, either repeated or not?
> I'd like to think upstream would be open to resolving these
> ambiguities.
I gotta admit, I suspect they won't be that open. Protocol buffers was
designed because Google needed a fast serialization method suitable for
many different internal projects. Needing round-trip idempotence seems
like a rare requirement to me, especially for internal use.
> Re-serialization of an Invoice message in the Payment message is a
> potential source of mistakes. There's no need to ever concatenate
> these messages and alternative implementations that don't order
> serialized fields by tag number are missing an important optimization,
> so they could be fixed. The main issue is treatment of unknown fields.
> If/when the Invoice message is extended with other fields that are
> round-tripped through an old client, the data may get lost. JSON
> doesn't help resolve that either, of course. There are a few
> solutions:
Well, actually you can take advantage of the message concatination
ability of protocol buffers to extend a message by simply appending the
new fields to the existing thus either defining new fields, or
overriding old values as required. If you want to de-duplicate though
you run into the problem all over again.
On the other hand JSON handles this case fine too provided that your
JSON implementation supports dictionary objects with arbitrary fields.
Just use the object as is and the unknown fields will be re-serialized
properly at the other end. Some implementations will have to be careful
to handle collisions with existing keys in the namespace. (consider in
Python what would happen if you mapped your object to a class instance,
and the serialization included the key "__init__")
That said, JSON is quite problematic with numbers. For instance, you
have to be careful to keep integers represented as pure integers below
what Javascript can handle, the maximum integer exactly representable in
a double float, or the JSON won't be parsable in Javascript even if many
other languages handle it fine. Protocol buffers is at least pretty
explicit about what size integers are.
> 1) Change the type of the Invoice field in Payment to be "bytes" and
> set it to be the hash of the originally received binary Invoice
> message. Downside, requires merchants to track all outstanding
> invoices.
> 2) Ask protobufs upstream to modify the spec/implementations so
> ordering of unknown fields is specified. The Python implementation
> could be extended to support them so Python implementors don't end up
> with accidental message downgrades.
> 3) Language of the spec could be changed to explicitly state that the
> received Invoice may not be binary-identical to the one that was sent,
> in the case of a client that incorrectly downgrades the message. Thus
> you'd be expected to check what the Invoice was using merchant_data
> which is opaque and could just be, eg, a database key on your own end.
> 4) Instead of submitting the entire Invoice back to the merchant, just
> the merchant_data could be in the Payment message.
>=20
> Of the four options I prefer the last. What is the use case for
> resubmitting the entire invoice anyway? Even if protobufs are improved
Note that I think the SignedInvoice message itself is broken, because
protobuf implementations have no reason to guarantee that they can give
you the serialized bytes of the Invoice sub-message. It's a quite
specific use-case that isn't needed for pretty much anything but crypto.
FWIW I took a quick look at the official API's, C++, Java and Python,
and as far as I can tell none of them support accessing the binary
serialization of a message field other than by re-serializing the
message.
Really the invoice field should be declared as bytes serialized_invoice,
as inconvenient as that is to work with.
> so handling of round-tripping new messages through old [Python]
> clients is more rigorous, some implementors will probably convert the
> protobuf objects into some internal forms for whatever reason (or
> serialize them to a database, etc) and they're very likely to mess up
> the handling of unknown fields when they do it.
Since the Payment message includes an *untrusted* Invoice that the
vendor needs to authenticate the whole invoice no matter what on Payment
reception. In many cases that implies they have to keep some sort of
database of "quotes" or similar anyway as the client can change anything
they want otherwise. Again that leads back to the argument of why not
just stick with the merchant_dat as you suggest, which will usually be
some short invoice number attached to a database? A vendor that wants to
operation a stateless invoicing system can just stuff a HMAC-protected
serialized invoice into the merchant_data
I guess you could use a mutable invoice field as a way of achieving some
sort of negotiation protocol, but I think it's better to stick to the
original concept of just ensuring that the user is really paying the
right amount to the right address.
--=20
'peter'[:-1]@petertodd.org
--h31gzZEtNLTqOjlF
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iQEcBAEBAgAGBQJQtgomAAoJEH+rEUJn5PoECCMH/i1bYVfMnt1h6z56jxZ73c4C
0W2nY3iB2F+B6hnvp4oI1o8Cfu44mOxgylZkqS+/flb2Zj8uew0LFQbx+o9SXlJr
CXy4qHaBaZbaBM1JvfkPTwYcC7nSCwzHeNvWaXGTg0qJ+su5nSK8lR7DeZFnKJMK
c77HIkmgStihl8GiJYKaPZROcUTcNmo5jzQedmZ3AelUIvQRQZYD0Zmvyx7SEAms
9EW8nxw20RwQZ4zYFIyeDWsYfO2kVf8n4oqy6tJvcJanspWF7DAf/XfLLWfNLsLL
2khi0POpAEk7t8exGb5YUP+G/ANnFgnF+dNz81yss8iFyluN2kWVA/LFSd5AFXA=
=XYYi
-----END PGP SIGNATURE-----
--h31gzZEtNLTqOjlF--
|
