Date: Tue, 12 Mar 2013 12:44:28 +0100
From: Pieter Wuille <pieter.wuille@gmail.com>
To: Michael Gronager <gronager@ceptacle.com>
Message-ID: <20130312114426.GA3701@vps7135.xlshosting.net>
References: <CAPg+sBip_4Jtxhq+rm-na2=RSJ_PuoZt+akGgJyo0b_Bwbr1Dw@mail.gmail.com>
	<CAPg+sBjm+e=A+edSRHXU7JSqyfSc4hou_SRdQHF48xhKQGA4zA@mail.gmail.com>
	<CANEZrP2V9uDQ-dmyaUBbsCuj5u3Mrh+jvU9RDpYkrKQV6+t0tQ@mail.gmail.com>
	<FB4ED2C4-8B65-438B-8B77-44234A644051@ceptacle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <FB4ED2C4-8B65-438B-8B77-44234A644051@ceptacle.com>
User-Agent: Mutt/1.5.21 (2010-09-15)
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] Warning: many 0.7 nodes break on large
 number of tx/block; fork risk
Precedence: list

On Tue, Mar 12, 2013 at 11:13:09AM +0100, Michael Gronager wrote:
> Yes, 0.7 (yes 0.7!) was not sufficiently tested it had an undocumented and unknown criteria for block rejection, hence the upgrade went wrong.

We're using "0.7" as a short moniker for all clients, but this was a limitation that all
BDB-based bitcoins ever had. The bug is simply a limit in the number of lock objects
that was reached.

It's ironic that 0.8 was supposed to solve all problems we had due to BDB (except the
wallet...), but now it seems it's still coming back to haunt us. I really hated telling
miners to go back to 0.7, given all efforts to make 0.8 signficantly more tolerable...

> More space in the block is needed indeed, but the real problem you are describing is actually not missing space in the block, but proper handling of mem-pool transactions. They should be pruned on two criteria:
> 
> 1. if they gets to old >24hr
> 2. if the client is running out of space, then the oldest should probably be pruned 
> 
> clients are anyway keeping, and re-relaying, their own transactions and hence it would mean only little, and only little for clients. Dropping free / old transaction is a much a better behavior than dying... Even a scheme where the client dropped all or random mempool txes would be a tolerable way of handling things (dropping all is similar to a restart, except for no user intervention).

Right now, mempools are relatively small in memory usage, but with small block sizes,
it indeed risks going up. In 0.8, conflicting (=double spending) transactions in the
chain cause clearing the mempool of conflicts, so at least the mempool is bounded by
the size of the UTXO subset being spent. Dropping transactions from the memory pool
when they run out of space seems a correct solution. I'm less convinced about a
deterministic time-based rule, as that creates a double spending incentive at that
time, and a counter incentive to spam the network with your risking-to-be-cleared
transaction as well.

Regarding the block space, we've seen the pct% of one single block chain space consumer
grow simultaneously with the introduction of larger blocks, so I'm not actually convinced
there is right now a big need for larger blocks (note: right now). The competition for
block chain space is mostly an issue for client software which doesn't deal correctly
with non-confirming transactions, and misleading users. It's mostly a usability problem
now, but increasing block sizes isn't guaranteed to fix that; it may just make more
space for spam.

However, the presence of this bug, and the fact that a full solution is available (0.8),
probably helps achieving consensus fixing it (=a hardfork) is needed, and we should take
advantage of that. But please, let's not rush things...

-- 
Piter