Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192]
	helo=mx.sourceforge.net)
	by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76)
	(envelope-from <jan@uos.de>) id 1QZroq-0005uZ-4V
	for bitcoin-development@lists.sourceforge.net;
	Thu, 23 Jun 2011 21:52:12 +0000
X-ACL-Warn: 
Received: from vm136.rz.uni-osnabrueck.de ([131.173.16.11]
	helo=smtp-auth.serv.Uni-Osnabrueck.DE)
	by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256)
	(Exim 4.76) id 1QZrom-0008BA-7J
	for bitcoin-development@lists.sourceforge.net;
	Thu, 23 Jun 2011 21:52:12 +0000
Received: from msmtp-using-host (0xbcb2ab7c.ronqu1.dynamic.dsl.tele.dk
	[188.178.171.124] (may be forged)) (authenticated bits=0)
	by smtp-auth.serv.Uni-Osnabrueck.DE (8.13.8/8.13.8) with ESMTP id
	p5NLpx8B031954
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO)
	for <bitcoin-development@lists.sourceforge.net>;
	Thu, 23 Jun 2011 23:52:00 +0200
From: jan@uos.de
Date: Thu, 23 Jun 2011 23:51:43 +0200
To: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Message-ID: <20110623215143.GA3351@dax.lan.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
X-PMX-Version: 5.6.1.2065439, Antispam-Engine: 2.7.2.376379,
	Antispam-Data: 2011.6.23.213914 (Univ. Osnabrueck)
X-PMX-Spam: Gauge=IIIIIIII, Probability=8%, Report=
	BODYTEXTP_SIZE_3000_LESS 0, BODY_SIZE_1900_1999 0, BODY_SIZE_2000_LESS 0,
	BODY_SIZE_5000_LESS 0, BODY_SIZE_7000_LESS 0, FROM_MISSING 0,
	NO_URI_FOUND 0, __CD 0, __CT 0, __CT_TEXT_PLAIN 0, __HAS_MSGID 0,
	__MIME_TEXT_ONLY 0, __MIME_VERSION 0, __PHISH_PHRASE1_B 0,
	__SANE_MSGID 0, __SUBJ_ALPHA_END 0, __TO_MALFORMED_2 0,
	__USER_AGENT 0
X-PMX-Spam-Level: IIIIIIII
X-Spam-Score: 0.3 (/)
X-Spam-Report: Spam Filtering performed by mx.sourceforge.net.
	See http://spamassassin.org/tag/ for more details.
	0.3 MAY_BE_FORGED Relay IP's reverse DNS does not resolve to IP
X-Headers-End: 1QZrom-0008BA-7J
Subject: [Bitcoin-development] Speeding up "getbalance <account>" calls
X-BeenThere: bitcoin-development@lists.sourceforge.net
X-Mailman-Version: 2.1.9
Precedence: list
List-Id: <bitcoin-development.lists.sourceforge.net>
List-Unsubscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=unsubscribe>
List-Archive: <http://sourceforge.net/mailarchive/forum.php?forum_name=bitcoin-development>
List-Post: <mailto:bitcoin-development@lists.sourceforge.net>
List-Help: <mailto:bitcoin-development-request@lists.sourceforge.net?subject=help>
List-Subscribe: <https://lists.sourceforge.net/lists/listinfo/bitcoin-development>,
	<mailto:bitcoin-development-request@lists.sourceforge.net?subject=subscribe>
X-List-Received-Date: Thu, 23 Jun 2011 21:52:12 -0000

Hi there!

Instawallet has enjoyed steady growth and I'm running into a bottleneck
now with "getbalance <someaccounthere>" taking quite some time to
complete. My understanding is, that this is because bitcoind runs
through all relevant transactions each time anew to compute the balance.
I was hoping the list could give me some pointers/ideas on how I can
improve this.

I might start to do account handling completely in my application at
some point, but for now I would like to continue letting bitcoind handle
this, so I don't have to worry about thinks like block chain reorgs.

Basically I don't have enough familiarity with the source code to feel
confident about doing fairly invasive changes. So right now I'm planning
to implement a very simple cache (directly in bitcoind), which just
caches calls to getbalance and simply invalidates the whole cache as
soon as a new block or a transaction that affects the wallet comes in.
I'm hoping this will help a little bit, although with blocks arriving
every 10 minutes on average it's not really the perfect solution.

Maybe one step better would be to only invalidate cache entries that are
affected by a new transaction or by transactions arriving in a block.
This would need to take block chain reorgs into account though, which
seems fairly complicated. Maybe I could simply invalidate the whole
cache on reorgs, which should be seldom enough that it's not a big
problem. Where would be a good place in the source code to hook into to
notice block chain reorgs?

The perfect solution would be, of course, if bitcoind could keep running
balances of all accounts and update them as new information becomes
available. But as I said, I don't feel that I have a good overview of
all the corner cases to make such an improvement. I guess an extensive
test suite testing all sorts of esoteric block chain reorgs would really
be a nice thing to have. :-)

Regards!
Jan