Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id C12229F0 for ; Wed, 29 Aug 2018 18:29:31 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-it0-f67.google.com (mail-it0-f67.google.com [209.85.214.67]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 1606C83B for ; Wed, 29 Aug 2018 18:29:31 +0000 (UTC) Received: by mail-it0-f67.google.com with SMTP id f14-v6so9040401ita.4 for ; Wed, 29 Aug 2018 11:29:30 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=08LScfZ38Vxc7vnNMTC4M9Z9Vl1AfkjfajP8YWTgkoI=; b=M0HTxx6Jry8F+ZrzE9mmzrSFL1U42AZxeB4x0e0ZrKaTeqE4nuNPk6l0S/Y0HJBZRQ ZoF1DobjjBAEF1D/375RWO5g22aNk6s5nSrue1itN1EBz6O4Bo0hzE2hz2k70hxLr1UU cmN92HYIJBkU8nJsIj5pbrA1pBa3A44fbpTeBqCkqPjt8fHFDFcAzRo4Z7Ujj1KqpCFg vlauALD2+FdxVCdrGCxh25QSVPmkR/kx7TezRX3NFeFcMhmEAshWS5Qidv8syb4iO5HE zPplkyQ4ezmYs3ZnEaycXC2w2RsehDb1YmApyH4xgUIC7axgV5DMdR0zgWe8ydYwGweX yoFw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=08LScfZ38Vxc7vnNMTC4M9Z9Vl1AfkjfajP8YWTgkoI=; b=f6ldNWVleUEvPsy9fvz15BxQrsfXJCU5i8GpHLn31UvkzgEHyBPI76LX0C3GCLxscm 8N2BQ6++bXhotHxy8vOUsW7IyK2OaWYLZvph3e9rJRU15tfbeMPwfYRmrpAia1lVhGd6 TQuQ7s6gkwHARMBFpPLInV8lyDgPP+CXViGhqNSI91e9+eLwDPiGSV8yjX+ioPNig9AY 34cNhviLE13Vq46Pi38qGEYBDDG28BfJqeqnczNTH5/nT/yZ3XFDDGt2rBG6v5ZcZ+bc ZMqLLxkAJrK4Vi5xIymWz225La0qDDgRHF0pmLPVr8gOLRTOOmpxWQRPOzv8Yq3uPy27 yDTg== X-Gm-Message-State: APzg51B8cZPzaV2D14zukGhYxu9W2In0pbDGA7Siz7O9IDGCjux8naDw RjiwgNoE9PVtCqbTFXghZQhXR0OLFSCZsTEsNbU= X-Google-Smtp-Source: ANB0VdYbAiTztvcKhKuMUtc3s98+hq9wljJU8T6LhiSLCet3AUR01yo4EP7zPulFDu0X5PxZDdC1MfJaYsdznDIUrHw= X-Received: by 2002:a24:282:: with SMTP id 124-v6mr5915590itu.151.1535567370238; Wed, 29 Aug 2018 11:29:30 -0700 (PDT) MIME-Version: 1.0 References: <8AE1517F-88FB-479D-AE89-993A5545D210@jonasschnelli.ch> <758E3CA7-295B-4B77-BFF5-9AAE959D53EA@voskuil.org> <4EC6599E-D317-4E4F-9E4D-37B7006B8C15@jonasschnelli.ch> In-Reply-To: <4EC6599E-D317-4E4F-9E4D-37B7006B8C15@jonasschnelli.ch> From: Blockchain Group Date: Wed, 29 Aug 2018 23:59:18 +0530 Message-ID: To: Jonas Schnelli Content-Type: multipart/alternative; boundary="000000000000a1bed10574972325" X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Thu, 30 Aug 2018 02:19:13 +0000 Cc: bitcoin-dev@lists.linuxfoundation.org Subject: Re: [bitcoin-dev] Building a Bitcoin API and query system. X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2018 18:29:31 -0000 --000000000000a1bed10574972325 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Awesome, thanks for the information. I will work on it and keep it in mind. On Wed, Aug 29, 2018, 11:57 PM Jonas Schnelli wrote: > > > The API implementation is not what is centralizing, nor is full > indexation non-scalable. The centralization is in not running the API fro= m > a node under your own control. This is of course implied by the comment, > =E2=80=9Cwithout the need for syncing=E2=80=9D. In other words it is the = deployment cost of > the node that is centralizing. > > IMO an API that serves non verifiable data is supporting centralised > validation. The =E2=80=9EAPI" which supports one of the most important pr= operties > in Bitcoin =E2=80=93 the ability to self-validate =E2=80=93 is the data a= vailable via the > p2p network. > > > > > Yet if people relied only on bitcoind and never centralized services > there would be *no* block explorers (and no secure light wallets), becaus= e > it does not provide remote query and does not fully index. > > > > Block explorers and light wallets are pretty useful, so presumably some > API must provide these features (ideally with reduced deployment cost). > That will either be centralized or decentralized services. As such it see= ms > wise to encourage the latter, as opposed to questioning whether there is > any valid block explorer use case. > > Bitcoin-Core has all required features to partially =E2=80=9Eindex=E2=80= =9C data (called > the wallet) and provides them via the RPC API. If you don=E2=80=99t need = to serve > thousands of wallets (which smells after centralised validation), selecti= ve > indexing (wallets) are the right choice. Also, if you have a proper light > client architecture, you can use Bitcoin Core in pruned mode (<10GB of > data) to serve an endless amount of wallets (client/server mode, I guess > that is what you are referring to with "light clients"). > > I fail to see the use-cases where a fully index blockchain makes sense > (the only one I can come up with is instant backup recovery where the > transaction history needs to be preserved rather then recovering the UTXO= s > only). > > Also, the p2p protocol has built in light client support with BIP37 (bloo= m > filters) and soon BIP158 will be available on the network which does allo= w > privacy-preserving "light clients" in a way where no trusted layer is > required (client <-> p2p network rather then client <-> API provider <-> > p2p network). > > I don=E2=80=99t want to advocate against a full-index blockexplorer-like = API. I > just think its important to define the use case and be aware of the > consequences and downsides. > > /jonas > --000000000000a1bed10574972325 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Awesome, thanks for the information. I will work on it an= d keep it in mind.=C2=A0

On Wed, Aug 29, 2018, 11:57 PM Jonas Schnelli <dev@jonasschnelli.ch> wrote:

> The API implementation is not what is centralizing, nor is full indexa= tion non-scalable. The centralization is in not running the API from a node= under your own control. This is of course implied by the comment, =E2=80= =9Cwithout the need for syncing=E2=80=9D. In other words it is the deployme= nt cost of the node that is centralizing.

IMO an API that serves non verifiable data is supporting centralised valida= tion. The =E2=80=9EAPI" which supports one of the most important prope= rties in Bitcoin =E2=80=93 the ability to self-validate =E2=80=93 is the da= ta available via the p2p network.

>
> Yet if people relied only on bitcoind and never centralized services t= here would be *no* block explorers (and no secure light wallets), because i= t does not provide remote query and does not fully index.
>
> Block explorers and light wallets are pretty useful, so presumably som= e API must provide these features (ideally with reduced deployment cost). T= hat will either be centralized or decentralized services. As such it seems = wise to encourage the latter, as opposed to questioning whether there is an= y valid block explorer use case.

Bitcoin-Core has all required features to partially =E2=80=9Eindex=E2=80=9C= data (called the wallet) and provides them via the RPC API. If you don=E2= =80=99t need to serve thousands of wallets (which smells after centralised = validation), selective indexing (wallets) are the right choice. Also, if yo= u have a proper light client architecture, you can use Bitcoin Core in prun= ed mode (<10GB of data) to serve an endless amount of wallets (client/se= rver mode, I guess that is what you are referring to with "light clien= ts").

I fail to see the use-cases where a fully index blockchain makes sense (the= only one I can come up with is instant backup recovery where the transacti= on history needs to be preserved rather then recovering the UTXOs only).
Also, the p2p protocol has built in light client support with BIP37 (bloom = filters) and soon BIP158 will be available on the network which does allow = privacy-preserving "light clients" in a way where no trusted laye= r is required (client <-> p2p network rather then client <-> AP= I provider <-> p2p network).

I don=E2=80=99t want to advocate against a full-index blockexplorer-like AP= I. I just think its important to define the use case and be aware of the co= nsequences and downsides.

/jonas
--000000000000a1bed10574972325--