Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192]
	helo=mx.sourceforge.net)
	by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
	(envelope-from <mh.in.england@gmail.com>) id 1SfWo2-00063V-8F
	for bitcoin-development@lists.sourceforge.net;
	Fri, 15 Jun 2012 13:43:18 +0000
Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.212.181 as permitted sender)
	client-ip=209.85.212.181; envelope-from=mh.in.england@gmail.com;
	helo=mail-wi0-f181.google.com; 
Received: from mail-wi0-f181.google.com ([209.85.212.181])
	by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
	(Exim 4.76) id 1SfWnw-0000FT-LY
	for bitcoin-development@lists.sourceforge.net;
	Fri, 15 Jun 2012 13:43:18 +0000
Received: by wibhn14 with SMTP id hn14so488284wib.10
	for <bitcoin-development@lists.sourceforge.net>;
	Fri, 15 Jun 2012 06:43:06 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.216.226.147 with SMTP id b19mr3391227weq.210.1339767786494;
	Fri, 15 Jun 2012 06:43:06 -0700 (PDT)
Sender: mh.in.england@gmail.com
Received: by 10.216.254.232 with HTTP; Fri, 15 Jun 2012 06:43:06 -0700 (PDT)
In-Reply-To: <CA+8xBpcwhQPQRe=stYb9xksLsTbiABKLS7PZnRtvPga6AmSg4Q@mail.gmail.com>
References: <CA+8xBpecVQcTTbPxUm_3_GWC99dEd4=-VFWb+QT6jUy4rg8U4w@mail.gmail.com>
	<CANEZrP0kNZDByHpK2=UjP+ag0X1KmqHxnJdm=e_pWMitP4QvvA@mail.gmail.com>
	<CA+8xBpcwhQPQRe=stYb9xksLsTbiABKLS7PZnRtvPga6AmSg4Q@mail.gmail.com>
Date: Fri, 15 Jun 2012 15:43:06 +0200
X-Google-Sender-Auth: Fx4PycrEAoFtN8o4rUE35gp7toM
Message-ID: <CANEZrP39RHfCDX-x4ARo+oPphLv-70RxuMh3+AJzsNPxzOd=bA@mail.gmail.com>
From: Mike Hearn <mike@plan99.net>
To: Jeff Garzik <jgarzik@exmulti.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -1.3 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
	See http://spamassassin.org/tag/ for more details.
	-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
	sender-domain
	0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
	(mh.in.england[at]gmail.com)
	-0.0 SPF_PASS               SPF: sender matches SPF record
	0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
	not necessarily valid
	-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
	0.2 AWL AWL: From: address is in the auto white-list
X-Headers-End: 1SfWnw-0000FT-LY
Cc: Bitcoin Development <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] New P2P commands for diagnostics,
	SPV clients
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Fri, 15 Jun 2012 13:43:18 -0000

> Yes, the format is something that must be hashed out (no pun
> intended). =C2=A0Need input from potential users about what information
> they might need.

Matts point that a branch-per-transaction may duplicate data is well
made, that said, I suspect a format that tries to fix this would be
much more complicated.

How about see this project as a three part change?

First step - add the mempool command and make nodes sync up their
mempools on startup.

Second step - if protocol version >=3D X, the "block" message consists
of a header + num transactions + vector<hash>  instead of the full
transactions themselves.

On receiving such a block, we go look to see which transactions we're
missing from the mempool and request them with getdata. Each time we
receive a tx message we check to see if it was one we were missing
from a block. Once all transactions in the block message are in
memory, we go ahead and assemble the block, then verify as per normal.
This should speed up block propagation. Miners have an incentive to
upgrade because it should reduce wasted work.

Third step - new message, getmerkletx takes a vector<hash> and returns
a merkletx message: "merkle branch missing the root + transaction data
itself" for each requested transaction. The filtering commands are
added, so the block message now only lists transaction hashes that
match the filter which can then be requested with getmerkletx.