1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
|
Return-Path: <aj@erisian.com.au>
Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133])
by lists.linuxfoundation.org (Postfix) with ESMTP id 7E79DC002B
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 17 Feb 2023 15:51:32 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
by smtp2.osuosl.org (Postfix) with ESMTP id 54393402BC
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 17 Feb 2023 15:51:32 +0000 (UTC)
DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 54393402BC
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -1.902
X-Spam-Level:
X-Spam-Status: No, score=-1.902 tagged_above=-999 required=5
tests=[BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001]
autolearn=ham autolearn_force=no
Received: from smtp2.osuosl.org ([127.0.0.1])
by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id Z3hSBqjifJhY
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 17 Feb 2023 15:51:29 +0000 (UTC)
X-Greylist: from auto-whitelisted by SQLgrey-1.8.0
DKIM-Filter: OpenDKIM Filter v2.11.0 smtp2.osuosl.org 5155A40191
Received: from cerulean.erisian.com.au (azure.erisian.com.au [172.104.61.193])
by smtp2.osuosl.org (Postfix) with ESMTPS id 5155A40191
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 17 Feb 2023 15:51:29 +0000 (UTC)
Received: from 60.42.96.58.static.exetel.com.au ([58.96.42.60]
helo=sapphire.erisian.com.au)
by cerulean.erisian.com.au with esmtpsa (TLS1.3) tls
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2)
(envelope-from <aj@erisian.com.au>)
id 1pT319-0004Yt-4G; Sat, 18 Feb 2023 01:51:25 +1000
Received: by sapphire.erisian.com.au (sSMTP sendmail emulation);
Sat, 18 Feb 2023 01:51:19 +1000
Date: Sat, 18 Feb 2023 01:51:19 +1000
From: Anthony Towns <aj@erisian.com.au>
To: Dhruv M <dhruv@bip324.com>,
Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Message-ID: <Y++id6mXsscfxduH@erisian.com.au>
References: <56677685-619a-691f-d5bc-54b69fdb6ed2@bip324.com>
<zxv58iXZ73hf9ge8S0QLTanW-uLzaWjNtMHuKONP9hrqS5RhwitxzfVaMH8hbi3yImgNrKme3lCuDcHYKkpxEQHyGZZHJ8xtReOcnAx3o4g=@wuille.net>
<Y2nK99fHUKxbPHmw@erisian.com.au>
<wDqcIVw-YGTsjdf5M2GO9NNRl_UQuBeka2CUQUyQ329u6u-o7RabW_7S4FD3EDfk02kUczb3bXf8LtHhKLtx773UhQ7djKOl-JPOIrXqBSc=@wuille.net>
<JXfTBjsA71dHE3h9wkxnWXANrwTbMADO4s2w34gEvMbiduKu4PEt5t-KA3EAIz-Xs4urjBHZ15NDFZST2a7e0x_NqyJymUnEORuTp3SNfMs=@wuille.net>
<Y3dBUXPhTskCx+Fu@erisian.com.au>
<gSxFQedPc72pTioi9vuxvLKpaRBsnKFL4gkPKPn2G-EJgz_2Y1pYQ7cHD5SnunyCaLln7UQEHIxnopqP74LlnK__Mf9BURbJW8B5MYTZvCU=@wuille.net>
<Y7dZctMlZtH6PEsz@erisian.com.au> <Y7vMGVQz8TjS4Cad@erisian.com.au>
<6b83c32e-59ca-65ef-2553-d66f8d117e56@bip324.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <6b83c32e-59ca-65ef-2553-d66f8d117e56@bip324.com>
X-Spam_score: -1.0
X-Spam_bar: -
Subject: Re: [bitcoin-dev] Refreshed BIP324
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Fri, 17 Feb 2023 15:51:32 -0000
On Thu, Feb 16, 2023 at 05:43:22PM +0000, Dhruv M via bitcoin-dev wrote:
> Problem:
> - 1 byte message type IDs are lacking a co-ordination mechanism when multiple in-flight BIPs are proposing new message types as the id space is reduced form 12 ASCII bytes to 1 byte.
> - 1 byte IDs are scarce and should be allocated judiciously, especially given that gains on bandwidth are very much non-uniform across message types.
ACK.
> Solutions:
> - Uniform encoding using the high-bit increases the available ID space drastically, however, there's still the issue of making sure that the most frequent message types get the shorter IDs.
> - Making type IDs negotiable(editable, really) per direction per connection solves that issue at the cost of some increased complexity.
>
> Since we don't really know the extent to which the protocol will ossify over time and that BIP324 is already quite a large change, we might want to optimize for the least additional complexity that doesn't close the doors on any of the solutions.
I think it's probably less complex to close *some* of the doors?
In particular, I think there's two questions that have to get answered:
1) how do you distinguish the command from the payload for
non short-ids -- by a length prefix, or by setting the high-bit
of the final command byte?
2) are short ids available/meaningful to send prior to VERACK being
completed?
> How about this:
> - BIP324 restricts type IDs to [1, 127]
Is this for short ids (currently [13-255] per the bip) or for every byte
in a non-short-id command (for p2p v1, IsCommandValid() restricts each
byte to being in the printable ascii range, ie [32-126])?
Here's another approach:
idea: we use short ids to minimise bandwidth, and don't care about
bandwidth for long ids
implementation:
short id 0 is reserved for long commands. when received, we
decode the first 12 bytes of the payload and treat them
exactly the same as a v1 p2p message (trailing 0-bytes, etc)
(if there's not 12 bytes of payload, it's just treated as an
invalid command and dropped)
short ids 1-255 are available for use as aliases of particular
long commands
(That's exactly compatible with p2p v1, and also avoids the temptation
to try to choose short command names rather than descriptive ones -- the
0-padding to 12 bytes prevents you from saving any bandwidth that way;
but that's what we have short ids for anyway)
If we decide we want >255 short ids, we can figure out how to extend
them later, in a fairly open ended way I think, eg by having [128-255]
imply a 2 byte short id, so that seems fine?
> - We remove 1 byte allocations for messages that are sent at most once per connection per direction
I think this leaves 32 commands that get short ids initially:
misc: ADDR, ADDRV2, BLOCK, FEEFILTER, GETBLOCKS, GETDATA, GETHEADERS,
HEADERS, INV, NOTFOUND, PING, PONG, TX
bip 35/37: FILTERADD, FILTERCLEAR, FILTERLOAD, MEMPOOL, MERKLEBLOCK
bip 152: BLOCKTXN, CMPCTBLOCK, GETBLOCKTXN
bip 157: CFCHECKPT, CFHEADERS, CFILTER, GETCFCHCKPT, GETCFHEADERS,
GETCFILTERS
bip 330: RECONCILDIFF, REQRECON, REQSKETCHEXT, SENDCMPCT, SKETCH
which drops:
VERSION, VERACK, GETADDR, SENDADDRV2, SENDHEADERS, SENDTXRCNCL,
WTXIDRELAY
compared to bip 324 currently.
I think the things missing from the current list (and not currently in
use by bitcoin core) are:
bip 61: REJECT
bip 331: GETPKGTXNS, PKGTXNS, ANCPKGINFO
> - Optionally, in the implementation we can attempt to move the type id mapping to the p2p layer away from the transport layer. I suspect this could also be done after the implementation is merged but might be cleaner as the mapping is a p2p concern.
I agree that's fine, though I expect that we'll probably want to do it
not long after bip 331 is ready for merge (or some other p2p improvement
comes along)...
Cheers,
aj
|
