Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193]
	helo=mx.sourceforge.net)
	by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
	(envelope-from <andyparkins@gmail.com>) id 1V1b2S-0008Cb-24
	for bitcoin-development@lists.sourceforge.net;
	Tue, 23 Jul 2013 11:45:56 +0000
Received-SPF: pass (sog-mx-3.v43.ch3.sourceforge.com: domain of gmail.com
	designates 209.85.212.171 as permitted sender)
	client-ip=209.85.212.171; envelope-from=andyparkins@gmail.com;
	helo=mail-wi0-f171.google.com; 
Received: from mail-wi0-f171.google.com ([209.85.212.171])
	by sog-mx-3.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128)
	(Exim 4.76) id 1V1b2Q-0004Da-7J
	for bitcoin-development@lists.sourceforge.net;
	Tue, 23 Jul 2013 11:45:56 +0000
Received: by mail-wi0-f171.google.com with SMTP id hj3so2892542wib.16
	for <bitcoin-development@lists.sourceforge.net>;
	Tue, 23 Jul 2013 04:45:48 -0700 (PDT)
X-Received: by 10.194.239.225 with SMTP id vv1mr22788867wjc.63.1374579947990; 
	Tue, 23 Jul 2013 04:45:47 -0700 (PDT)
Received: from momentum.localnet ([91.84.15.31])
	by mx.google.com with ESMTPSA id b20sm5321865wiw.4.2013.07.23.04.45.46
	for <multiple recipients>
	(version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128);
	Tue, 23 Jul 2013 04:45:47 -0700 (PDT)
From: Andy Parkins <andyparkins@gmail.com>
To: Peter Todd <pete@petertodd.org>
Date: Tue, 23 Jul 2013 12:45:44 +0100
User-Agent: KMail/1.13.7 (Linux/3.9-1-686-pae; KDE/4.8.4; i686; ; )
References: <CAJHLa0Ou1xF=LeLVu_wH1-XgJ1PavDV7_NHoDevo3R9+4z-ZfQ@mail.gmail.com>
	<201307231100.24538.andyparkins@gmail.com>
	<20130723101728.GA2116@savin>
In-Reply-To: <20130723101728.GA2116@savin>
MIME-Version: 1.0
Content-Type: Text/Plain;
  charset="iso-8859-15"
Content-Transfer-Encoding: 7bit
Message-Id: <201307231245.44634.andyparkins@gmail.com>
X-Spam-Score: -1.6 (-)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
	See http://spamassassin.org/tag/ for more details.
	-1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for
	sender-domain
	0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider
	(andyparkins[at]gmail.com)
	-0.0 SPF_PASS               SPF: sender matches SPF record
	-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
	author's domain
	0.1 DKIM_SIGNED            Message has a DKIM or DK signature,
	not necessarily valid
	-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
X-Headers-End: 1V1b2Q-0004Da-7J
Cc: bitcoin-development@lists.sourceforge.net,
	Andreas Schildbach <andreas@schildbach.de>
Subject: Re: [Bitcoin-development] HTTP REST API for bitcoind
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Tue, 23 Jul 2013 11:45:56 -0000

On Tuesday 23 July 2013 11:17:28 Peter Todd wrote:
> On Tue, Jul 23, 2013 at 11:00:24AM +0100, Andy Parkins wrote:
> > > Increasing the resource usage by SPV clients on full nodes is
> > > undesirable;
> > 
> > I don't think that's thinking big enough.  What I imagine is that making
> > it easier and easier to store a partial blockchain would result in lower
> > demand on full nodes.
> 
> Read my proposal for "Partial UTXO" mode:
> http://www.mail-archive.com/bitcoin-development@lists.sourceforge.net/msg02
> 511.html

Very interesting.  I love the idea of the UTXO set being tied to a block.

> > I might run a client that has only fetched blocks that contain
> > transactions needed to verify my balances, right back to the genesis
> > block.  That will be some small subset of the block chain and will take
> > me very little resource to maintain.  I join the network and am my
> > client is willing to verify based on information I have, or supply (by
> > REST or bitcoin protocol) blocks.  Imagine then that everyone with a
> > wallet were doing this.  The blockchain would be distributed massively. 
> > Obviously the miners would still be keeping the entire chain, but we'd
> > have a lot more nodes in the network, each contributing a little bit and
> > so reducing the load on the full nodes.
> 
> Actually the really scary thing about partial UTXO mode is miners can
> get away without keeping the entire chain provided they don't (often)
> try to mine transactions spending UTXO's that they haven't verified
> themselves.

You're right.  That is scary.

> > > In any case UTXO data currently requires you to have full trust in
> > > whomever is providing you with it, and that situation will continue
> > > until UTXO commitments are implemented - if they are implemented.
> > 
> > Almost; because you can go and ask someone else the same question, it's
> > pretty
> 
> How do you know they actually are someone else?

You don't.  You can't invalidate the lie if all you have access to is lies.  
But if you have access to just one honest node; that will reveal the liars.  
I'm not claiming that headers-only nodes can ever be made as secure as a full 
node.  Just _more_ secure than they are now; and potentially able to act as 
one of those honest nodes.

> > easy to check if you're being lied to.  Also, it's far easier to maintain
> > a headers-only block chain.  When you fetch your relevant block subset,
> > you can easily see that they are real blocks in your headers-only
> > blockchain; and so it's pretty much impossible to lie to "give me the
> > block containing transaction X".
> 
> Do you think you have SPV or full security in that situation?
> Do you know the difference?

There is absolutely no need to get condescendingly shirty.  I thought this was 
a friendly list; and we were having a discussion.  If you don't want to 
respond to posts -- don't.  I also didn't realise I had to pass an exam before 
I was allowed to speak.

Yes: I know the difference between SPV and full security.  SPV is headers only 
and so has no way to verify that the transaction outputs references as inputs 
to any new as-yet-unverified transaction are valid.  Instead it relies on 
having some way of proving it's in the chain; and then looking for the number 
of blocks built on top of it as "verification".  "Full security" (which is 
itself a very poor name), is obviously just checking that every output 
referenced in the inputs is unspent; that necessarily requires full blocks.

The difference in security being that in SPV there is no way to know if the 
referenced Unspent TransaXtion Output really is unspent -- it might have been 
spent elsewhere then referenced again in this new transaction.

My suggestion was that we want to be able to fetch a block by transaction; and 
that simple nodes can all, in aggregate offer contribution to the network 
rather than just being parasitical on the full nodes.   When I ask for a block 
that contains a transaction, and I do that repeatedly, I have part of the 
block chain.  If lots of simple nodes are doing that, then the whole chain 
should be available if there are enough of them.  They would then gain the 
ability to do transaction-forwarding in some cases.  This is only possible if 
a few extra facilities are added to the protocol.  One of which is the new 
feature I suggested: block-given-transaction.  It's not enough on its own, but 
if you also add in the ability for a node to tell another about the output 
transactions (basically, what block spends it), _then_ the simple nodes are 
able to become much more secure -- not 100% of course, they're still not full 
nodes, because they have no way of knowing if they are being lied to when they 
are told (this transaction is unspent), but all it takes is one honest node to 
point them at the truth, and the lie is then exposed.

That facility is just a drain on full nodes for the most part; except if you 
start encouraging it whole-sale.  The simple node would keep cache both the 
incoming and outgoing transactions (or rather the blocks that contain them) 
for addresses to which they are paying attention.  That gives them a cache 
that contains more than just their minimal set; and then they are able to do 
just a little bit of verifying on their own.  With enough nodes of this sort, 
the verification load is reduced.

Perhaps all that effort is not worth it for the tiny reduction.  Perhaps it's 
not true that that contribution of verification adds nothing.  I can live with 
those objections.  But "do I know the difference" as a reposte?  Not so much.

Anyway; going by your post on partial UTXO's; you're well ahead of the game, 
and I'm not suggesting anything that hasn't already been thought of, and 
thought of better.  I'm not sure why you took umbridge at my idea, when it 
seems like I'm just a few steps behind what you've already thought of.  Not 
everything is an attack you know?



Andy

-- 
Dr Andy Parkins
andyparkins@gmail.com