MIME-Version: 1.0
In-Reply-To: <9ab6e32e-db51-4ce4-8f3c-3a77f7b1f9bd@Spark>
References: <9ab6e32e-db51-4ce4-8f3c-3a77f7b1f9bd@Spark>
From: Jim Posen <jim.posen@gmail.com>
Date: Tue, 3 Apr 2018 10:45:34 -0700
Message-ID: <CADZtCSjU78fO4bKcyc-sTT_H_wuugAd5PF3Ncom-4F2uFk_AnQ@mail.gmail.com>
To: Gleb Naumenko <naumenko.gs@gmail.com>, 
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="0000000000000a55f40568f54641"
Subject: Re: [bitcoin-dev] Low-bandwidth transaction relay
Precedence: list

--0000000000000a55f40568f54641
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hey. This idea sounds quite interesting. It'd be helpful to see some more
numbers to evaluate it.

- How much bandwidth is consumed by redundant tx INVs currently? What is
this as a % of overall bandwidth usage?
- How would filtering txs through N=3D2 links affect network propagation?
This probably requires simulation to determine.
- Do you propose setting filters on inbound peers as well?

On Mon, Apr 2, 2018 at 3:18 PM, Gleb Naumenko via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi all,
> I have a couple of ideas regarding transaction relay protocol and wanted
> to share it with and probably get some feedback.
>
> I did some emulation and simulation and found out that around 90% of INV
> messages sent by public-IP nodes are idle (duplicate), obviously because
> each node creates 8 connections.  I also realized that sending INV messag=
es
> is a significant part of the overall bandwidth consumed by a public-IP
> node. At a larger scale, this will result in people not able to run a
> public-IP node.
>
> My idea is in some sense similar to BIP37 but applied to public-IP nodes.
> Here I want to emphasize that all the nodes will still receive *all* of t=
he
> transactions. A new protocol should also keep the same zero-trust,
> robustness, decentralization guarantees and latency.
>
> Idea: while joining the network, a new node agrees on some filter with
> each of 8 nodes it connects to. So that NewNode <-> Node_A will be used t=
o
> relay only a subset of transactions, NewNode <-> Node_B for another subse=
t.
> This will significantly decrease the redundancy.
>
> To keep the guarantees, I would keep some redundancy (for example, each
> transaction INV is sent over 2 links).
>
> To make it robust to attacks, I have 2 extensions in my mind:
> 1. Set reconciliation (for a subset of transactions) with *other* nodes.
> Getting a bloom filter of a subset of the mempool transactions from Node_=
B
> may help to figure out whether Node_A is malicious, very slow, etc.
> 2. Rotating the filters every N minutes (N < 10)
>
> I can see some issues with latency here, but I believe this problem has a
> solution.
>
> Feedback is appreciated!
>
> If you want to look at a draft of the proposal =E2=80=94 please let me kn=
ow.
> If there were any similar ideas =E2=80=94 please let me know.
>
> Best,
> Gleb
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

--0000000000000a55f40568f54641
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hey. This idea sounds quite interesting. It&#39;d be helpf=
ul to see some more numbers to evaluate it.<div><br></div><div>- How much b=
andwidth is consumed by redundant tx INVs currently? What is this as a % of=
 overall bandwidth usage?</div><div>- How would filtering txs through N=3D2=
 links affect network propagation? This probably requires simulation to det=
ermine.</div><div>- Do you propose setting filters on inbound peers as well=
?</div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On M=
on, Apr 2, 2018 at 3:18 PM, Gleb Naumenko via bitcoin-dev <span dir=3D"ltr"=
>&lt;<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_bl=
ank">bitcoin-dev@lists.linuxfoundation.org</a>&gt;</span> wrote:<br><blockq=
uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc =
solid;padding-left:1ex">


<div>
<div name=3D"messageBodySection" style=3D"font-size:14px;font-family:-apple=
-system,BlinkMacSystemFont,sans-serif">
<div>Hi all,</div>
<div>I have a couple of ideas regarding transaction relay protocol and want=
ed to share it with and probably get some feedback.</div>
<div><br></div>
<div>I did some emulation and simulation and found out that around 90% of I=
NV messages sent by public-IP nodes are idle (duplicate), obviously because=
 each node creates 8 connections.=C2=A0 I also realized that sending INV me=
ssages is a significant part of the overall bandwidth consumed by a public-=
IP node. At a larger scale, this will result in people not able to run a pu=
blic-IP node.</div>
<div><br></div>
<div>My idea is in some sense similar to BIP37 but applied to public-IP nod=
es. Here I want to emphasize that all the nodes will still receive *all* of=
 the transactions. A new protocol should also keep the same zero-trust, rob=
ustness, decentralization guarantees and latency.</div>
<div><br></div>
<div>Idea: while joining the network, a new node agrees on some filter with=
 each of 8 nodes it connects to. So that NewNode &lt;-&gt; Node_A will be u=
sed to relay only a subset of transactions, NewNode &lt;-&gt; Node_B for an=
other subset. This will significantly decrease the redundancy.</div>
<div><br></div>
<div>To keep the guarantees, I would keep some redundancy (for example, eac=
h transaction INV is sent over 2 links).</div>
<div><br></div>
<div>To make it robust to attacks, I have 2 extensions in my mind:=C2=A0</d=
iv>
<div>1. Set reconciliation (for a subset of transactions) with *other* node=
s. Getting a bloom filter of a subset of the mempool transactions from Node=
_B may help to figure out whether Node_A is malicious, very slow, etc.</div=
>
<div>2. Rotating the filters every N minutes (N &lt; 10)</div>
<div><br></div>
<div>I can see some issues with latency here, but I believe this problem ha=
s a solution.</div>
<div><br></div>
<div>Feedback is appreciated!</div>
<div><br></div>
<div>If you want to look at a draft of the proposal =E2=80=94 please let me=
 know.</div>
<div>If there were any similar ideas =E2=80=94 please let me know.</div>
<div><br></div>
<div>Best,</div>
<div>Gleb</div>
</div>
<div name=3D"messageReplySection" style=3D"font-size:14px;font-family:-appl=
e-system,BlinkMacSystemFont,sans-serif"><br>
<div></div>
</div>
</div>

<br>______________________________<wbr>_________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.=
<wbr>linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.<wbr>org=
/mailman/listinfo/bitcoin-<wbr>dev</a><br>
<br></blockquote></div><br></div>

--0000000000000a55f40568f54641--