Return-Path: Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id 19021C000A for ; Mon, 29 Nov 2021 14:10:19 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 01C4B6065F for ; Mon, 29 Nov 2021 14:10:19 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -1.901 X-Spam-Level: X-Spam-Status: No, score=-1.901 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Xn-TJOejaORK for ; Mon, 29 Nov 2021 14:10:17 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 Received: from KOR01-PS2-obe.outbound.protection.outlook.com (mail-ps2kor01olkn0800.outbound.protection.outlook.com [IPv6:2a01:111:f400:fead::800]) by smtp3.osuosl.org (Postfix) with ESMTPS id DB5E06062B for ; Mon, 29 Nov 2021 14:10:15 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ORhxs44/L6K+XseCTyAQYGcVo8Gb90fkAr+sTEmOVgI1jE2Jfmql9BvVnredg92V2wpHdPffGdrtS7UpCOKMgPNqj2otvL9j7uC/6DwsmDRh9rkKMoMPNmHjzK609G4+skRXEJYMY9z2c69ZJarzroRFsNJYZI7GnQ9EuPbGhUM+GuKfHjpXbSt6UxAUpFmC015B1NsvO6TrDX/bqVFxoQ/X1fBlhaWnfRwDsjHg8Dtg6JX6/LspOa193lF+EZdkVMFr3HBaktDiEfy4xJN/3nuIvSaVIHWKLTIe2hkF2qmx9HnW4BCwinGNgV95rJ6vZNzw2cGwxNPHOR5uQ6xb6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=pW9473JqNfSH4WgbjNI/diGpkdt05P0sPmApsC/T3GY=; b=eqXx6+Hzig183zFc/DoluhAUWwGeaDbBZCsw4sfe68OcpAUU1vkrPaRlslUTOmnhShLV8drR07gC3mCZXYmOxfpZb6gRhDzLa+2A/OME9f32yCIxmUhICVuBucXTWlGX65i5VnPKG/GKY6TMSmvPvth4od/99nutipv372Dq3mMC4+XqzkFjm90xrR1eGYvBE071MjOYj+x54f2hByRjnn8DAFLIMIeif9oXQBVGnFRfTjab2cDCnirs4nShhmghkgy5ywxr84bbQwxEVzWZFeArOZUalu7gJSiL7dw5jt6/aB2oLjUTDvUWjmL3HA/+zsw+1e/S3sE5PyvHtVdDPw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none Received: from PS2P216MB1089.KORP216.PROD.OUTLOOK.COM (2603:1096:301:52::11) by PU4P216MB1356.KORP216.PROD.OUTLOOK.COM (2603:1096:301:a2::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4734.23; Mon, 29 Nov 2021 14:10:11 +0000 Received: from PS2P216MB1089.KORP216.PROD.OUTLOOK.COM ([fe80::4c6c:a1dc:c8c0:fed2]) by PS2P216MB1089.KORP216.PROD.OUTLOOK.COM ([fe80::4c6c:a1dc:c8c0:fed2%3]) with mapi id 15.20.4734.024; Mon, 29 Nov 2021 14:10:11 +0000 From: LORD HIS EXCELLENCY JAMES HRMH To: Ali Sherief , Bitcoin Protocol Discussion , 0xB10C <0xb10c@gmail.com> Thread-Topic: [bitcoin-dev] Trying to patch Core ZMQ "rawtx" topic to only publish unconfirmed transactions: How? Thread-Index: AQHX5QYL3AFqJ1NPY0C53B0aDLTo/awaiuIc Date: Mon, 29 Nov 2021 14:10:11 +0000 Message-ID: References: <6dbaa89c-b658-e239-9f28-5f1609b8e746@gmail.com> In-Reply-To: <6dbaa89c-b658-e239-9f28-5f1609b8e746@gmail.com> Accept-Language: en-AU, en-US Content-Language: en-AU X-MS-Has-Attach: X-MS-TNEF-Correlator: suggested_attachment_session_id: c02dd0a9-2f23-d713-203e-ece5cef35278 x-ms-exchange-messagesentrepresentingtype: 1 x-tmn: [j4yRbNZXDbkjlwLji6YsQcQituB5YbvV] x-ms-publictraffictype: Email x-ms-office365-filtering-correlation-id: dfad70af-950e-4813-7614-08d9b341f4d7 x-ms-traffictypediagnostic: PU4P216MB1356: x-microsoft-antispam: BCL:0; x-microsoft-antispam-message-info: wCC1289guYx1QLgy7y6E12qiynyq6CqRqvUkPMtxuruxK41qXAD1/KMtdkb+Mpww7oPib/HUma2RcVOIuuBpgig3a/v2O+qmoqcZMz2ZQLGpMl8hxlvLNJjobpzWsNTFfLQP/4uS1pki/+iyTtO9SjJWeG/wJ74OAPMKULAh6wTytZ29eq1ZXfGeLSNo69NJNOojoguIay3fGtQiWya/iKHpUWkXH6yydwc2iFk74z1DA2F429jRtFLuWTee5uiVKhIELvgvkppHM2VP5UOqFPEBfmsIJ+wtyxyCMJ8alG6MlCuRpnCY6eiwzpH/aDP7bTlnhHJbfhsW0Kquk2iLTAulYAsEmilCqeKPAhIbFdIZFzS+b49LgXKmETXbYX6hlmRWOLgxjuwS3JeHi0POJ3+FkCoQXPyxVZE1pAF8JKbzwrA98DQSOn8+wS36RwDbU43CYAxYL1zjI48hBlZHvIq1xJlmsnJPUqp8jwEUGAIDOwLktFk1UINfabv1aPd7Y6Quxwe2SYXFOQBpw4DpbgAFhqUsgtMrOSfG6lvmzn0sNlnlVHvsqmWQ1mTgyG4NvVTHt3aTQS7wPOf++yddKruzTHDQ8nKmzO8tTKTWtebNiS6ht9jOD56H8TbHDv0QgxkgfsJobeRC2O0eJd/xx9fvA9kCjyyaY28+iN7HC4Y= x-ms-exchange-antispam-messagedata-chunkcount: 1 x-ms-exchange-antispam-messagedata-0: j1iyVPARQa8uDIYkDPE/SfKXBMdhwmnGeSTOffhjvKz4IaqdLB6ZFlJcJFgwmAcy5IjlgAICVLZijNlOAQ7JyYTN57NZD5jhKufLIwHQz1DbBz4keEAGDy8PS4dA6mgZCKvQLkEFVt6osq3HpRQ5IA5aV3DRxbDD/zGSwZigsldrp0VREkUn83qfUptg3ioNK9bWnfvdckhJmp6J+Gbspi/26Vy9AlzS3Tvr17VrAUXrhkj4r8Qo5r5P/vkSZvBgJj+RY+a2QsBbBuG4xTVA+1wpwBph3g/roEQtJreW/zbGqULgw4KkQ6V3P+aTncxXpyp5x0yCN1VUbpVAi1foN7YaZ31scaEOCSjr+7U8Z3rTRKSk5M3+Sa1/NUw/xk1ygimcheoU0b55zcvvm8mdjsQSppEswNrlQkz9REJX6LBGJ0eqrKx6IsUlHgd5y05lYJlJ/aFiCMVEpEvWotvAvdY+fbps3KiOzL5jCIWa58H4D6HjgDFpRtdYq/qbbvpM+EzZ4MOtdYgu4ccoPERFvtL4J87iqpd4GCjvFNsdiZ2ucWx9oBQe3MGJe7LVZuHrPDPlo2ctm+PM8Rgf1gvZ3LiZP0NopHCLINuYeIFKfbG0vvRdLHX7Gk3Ug8VZql7yxKFXSlY+MA4Iza/ktQ2JrR8bzNFQWbyxlYvDm6czKB4nXLltcKedf+9MNVDe7/FpHm1ZeHasp9F7Ly8iNndEWA== Content-Type: multipart/alternative; boundary="_000_PS2P216MB108925E1A218737486323FA19D669PS2P216MB1089KORP_" MIME-Version: 1.0 X-OriginatorOrg: sct-15-20-3174-20-msonline-outlook-af45a.templateTenant X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-AuthSource: PS2P216MB1089.KORP216.PROD.OUTLOOK.COM X-MS-Exchange-CrossTenant-RMS-PersistedConsumerOrg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-CrossTenant-Network-Message-Id: dfad70af-950e-4813-7614-08d9b341f4d7 X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Nov 2021 14:10:11.0767 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 84df9e7f-e9f6-40af-b435-aaaaaaaaaaaa X-MS-Exchange-CrossTenant-rms-persistedconsumerorg: 00000000-0000-0000-0000-000000000000 X-MS-Exchange-Transport-CrossTenantHeadersStamped: PU4P216MB1356 X-Mailman-Approved-At: Mon, 29 Nov 2021 14:21:38 +0000 Subject: Re: [bitcoin-dev] Trying to patch Core ZMQ "rawtx" topic to only publish unconfirmed transactions: How? X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Nov 2021 14:10:19 -0000 --_000_PS2P216MB108925E1A218737486323FA19D669PS2P216MB1089KORP_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Wasn't this already not a problem because you can check if it was confirmed= ? The transaction is not finalised in the mempool it is just speculation of= a transaction, so it makes sense to emit when the transaction is confirmed= . Just already check.. > It appears that the ZeroMQ topic I'm listening to, "rawtx", not only > emits a raw transaction when it appears on the mempool, but once it's > already confirmed too. KING JAMES HRMH Great British Empire Regards, The Australian LORD HIS EXCELLENCY JAMES HRMH (& HMRH) of Hougun Manor & Glencoe & British Empire MR. Damian A. James Williamson Wills et al. Willtech www.willtech.com.au www.go-overt.com duigco.org DUIGCO API and other projects m. 0487135719 f. +61261470192 This email does not constitute a general advice. Please disregard this emai= l if misdelivered. ________________________________ From: bitcoin-dev on behalf= of 0xB10C via bitcoin-dev Sent: Monday, 29 November 2021 8:32 PM To: Ali Sherief ; Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] Trying to patch Core ZMQ "rawtx" topic to only p= ublish unconfirmed transactions: How? Hi Ali, I've run into this multiple times myself. I've opened a draft PR [0] adding a rawmempooltx publisher. You're right. In zmq/zmqnotificationinterface.cpp the CZMQNotificationInterface is notified about TransactionAddedToMempool. Currently, this calls NotifyTransaction() (the publisher with the rawtx topic) and NotifyTransactionAcceptance() (the publisher with the sequence topic)[1]. I've added a call to a new NotifyMempoolTransaction() function (the publisher with the rawmempooltx topic). I'd find a mempool transaction publisher with both the raw transaction and transaction fee useful too. However, this requires changes to the chain notifications in interfaces/chain.h. [0]: https://github.com/bitcoin/bitcoin/pull/23624 [1]: https://github.com/bitcoin/bitcoin/pull/23624/files#diff-ac4b2d3a8de2c4dd41= ad9d75505ea6ce4dc87a476710a9ebee8acf9bebf5cca2L146-L148 Best, 0xB10C On 11/26/21 5:56 PM, Ali Sherief via bitcoin-dev wrote: > > This has also been posted on Bitcointalk > forum: https://bitcointalk.org/index.php?topic=3D5373341.msg58539261#msg5= 8539261 > I > have republished it here hoping someone more knowledgeable can post > some insight about this. > ---- > It appears that the ZeroMQ topic I'm listening to, "rawtx", not only > emits a raw transaction when it appears on the mempool, but once it's > already confirmed too. > > This messes with my software, causing it to add txids, addresses, etc. > a second time inside arrays (this means that the same transaction is > received twice in total). > > Array de-duping is not a viable solution long-term (because the array > will quickly grow to be big eventually and then this has to happen > every time a new element is added), so I'm trying to nip the problem > from the source by instructing Core to only publish unconfirmed > bitcoin transactions. > > According to > https://bitcoin.stackexchange.com/questions/52848/is-it-possible-to-confi= gure-the-bitcoin-daemon-to-only-broadcast-unconfirmed-tra > > , it is not possible to configure this from a configuration or > command-line option. The source code must directly be edited. But > since the codebase has changed greatly, the proposed solution no > longer works. > > ---- > > So basically, I know that something inside > src/zmq/zmqnotificationinterface.cpp needs to be patched, but I'm not > sure which function, or how to do it. Because I only need unconfirmed > transactions to be published on ZeroMQ rawtx and not confirmed ones, > it's one of the following functions that I need to patch for my own build= : > > CZMQNotificationInterface::TransactionRemovedFromMempool > void CZMQNotificationInterface::BlockDisconnected > > Both of these call NotifyTransaction() method which I assume fires a > message on "rawtx" channel. > > In the Stack Exchange question I linked above, Jonas Schnelli > suggested adding an `if (!pblock)` check, but that was several years > ago and the function he was referencing no longer exists. > > But I still wonder if the pblock check is still applicable in the > present day (i.e. if it's indicating a block the transaction is inside). > > - Ali Sherief > > _______________________________________________ > bitcoin-dev mailing list > bitcoin-dev@lists.linuxfoundation.org > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev _______________________________________________ bitcoin-dev mailing list bitcoin-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev --_000_PS2P216MB108925E1A218737486323FA19D669PS2P216MB1089KORP_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Wasn't this already not a problem because you can check if it was confirmed= ? The transaction is not finalised in the mempool it is just speculation of= a transaction, so it makes sense to emit when the transaction is confirmed= .  Just already check..

> It appears that the Ze= roMQ topic I'm listening to, "rawtx", not only
> emits a raw transaction when it appears on the mempool, but once it's<= br> > already confirmed too.

KING JAMES HRMH
Great British Empire

Regards,
The Australian
LORD HIS EXCELLENCY JAMES HRMH (& HMRH)
of Hougun Manor & Glencoe & British Empire
MR. Damian A. James Williamson
Wills

et al.

 
Willtech
www.willtech.com.au
www.go-overt.com
duigco.org DUIGCO API
and other projects
 

m. 0487135719
f. +61261470192


This email does not constitute a general advice. Please disregard this emai= l if misdelivered.
From: bitcoin-dev <bit= coin-dev-bounces@lists.linuxfoundation.org> on behalf of 0xB10C via bitc= oin-dev <bitcoin-dev@lists.linuxfoundation.org>
Sent: Monday, 29 November 2021 8:32 PM
To: Ali Sherief <ali@notatether.com>; Bitcoin Protocol Discuss= ion <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] Trying to patch Core ZMQ "rawtx"= ; topic to only publish unconfirmed transactions: How?
 
Hi Ali,

I've run into this multiple times myself. I've opened a draft PR [0]
adding a rawmempooltx publisher.

You're right. In zmq/zmqnotificationinterface.cpp the
CZMQNotificationInterface is notified about TransactionAddedToMempool.
Currently, this calls NotifyTransaction() (the publisher with the rawtx
topic) and NotifyTransactionAcceptance() (the publisher with the
sequence topic)[1]. I've added a call to a new
NotifyMempoolTransaction() function (the publisher with the rawmempooltx topic).

I'd find a mempool transaction publisher with both the raw transaction
and transaction fee useful too. However, this requires changes to the
chain notifications in interfaces/chain.h. 

[0]: https://gith= ub.com/bitcoin/bitcoin/pull/23624
[1]:
https:/= /github.com/bitcoin/bitcoin/pull/23624/files#diff-ac4b2d3a8de2c4dd41ad9d755= 05ea6ce4dc87a476710a9ebee8acf9bebf5cca2L146-L148 


Best,
0xB10C



On 11/26/21 5:56 PM, Ali Sherief via bitcoin-dev wrote:
>
> This has also been posted on Bitcointalk
> forum: https://bitcointalk.org/index.php?topic=3D53733= 41.msg58539261#msg58539261
> <https://bitcointalk.org/index.php?topic=3D5373341.msg58= 539261#msg58539261> I
> have republished it here hoping someone more knowledgeable can post > some insight about this.
> ----
> It appears that the ZeroMQ topic I'm listening to, "rawtx", = not only
> emits a raw transaction when it appears on the mempool, but once it's<= br> > already confirmed too.
>
> This messes with my software, causing it to add txids, addresses, etc.=
> a second time inside arrays (this means that the same transaction is > received twice in total).
>
> Array de-duping is not a viable solution long-term (because the array<= br> > will quickly grow to be big eventually and then this has to happen
> every time a new element is added), so I'm trying to nip the problem > from the source by instructing Core to only publish unconfirmed
> bitcoin transactions.
>
> According to
> https://bitcoin.stackexchange.com/questions/52848/is-it-possible-to-configu= re-the-bitcoin-daemon-to-only-broadcast-unconfirmed-tra
> <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev


_______________________________________________
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
= https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
--_000_PS2P216MB108925E1A218737486323FA19D669PS2P216MB1089KORP_--