## Get transaction data from full node

1

Background Information

I am building an application that requires the ability to query any bitcoin full node (from a python script using raw tcp sockets) in order to read the OP_Return value listed in the following transaction (https://live.blockcypher.com/btc-testnet/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1/)

What I have done to date

I have successfully managed to establish the version + verack, however I was faced with two issues.

1. When I send a getdata for the necessary transaction Idit does not return the result. I have identified this is because my initial assumptions that getdata returns the necessary data was incorrect and instead it will only return transaction data for transactions still in mempool.
2. This lead me to my main issue, I was hoping to replicate the initial block download on my python script, starting from the block containing the transaction I'm interested in. However when I do getblocks/getheaders no matter what I seem to put in the hash filter I get a dump of 500 (or 2000) blocks/headers. Below is a hex dump of the requests I am making (just the bitcoin protocol frame):

I am looking for the following help:

1. How can I query for transaction data without using bitcoin RPC if I can't use the getdata message?
2. How do I get my block or headers response to only contain 1 inv item instead of 500/5000

EDIT (4/5/20) The implied requirement which wasn't made clear here was that we needed to do so in a decentralised way using the blockchain RPC api or equivalent.

You want to download 250GB of blocks from a remote peer in order to find a single output? – Anonymous – 2019-12-14T17:45:45.057

No I don't want to do that, but there doesn't seem like there are too many alternatives. I'm trying to pretend like I am a bitcoin full node trying to sync (but attempting to sync from the necessary block that I need, this way I would only be syncing 1 block worth of data) I just don't think I'm using the API properly – Dimitree – 2019-12-14T22:38:20.007

1If your best choice is to do something absurd, maybe reconsider your strategy. The p2p network isn’t designed for arbitrary data access like you’re attempting. – Anonymous – 2019-12-14T22:41:55.423

I've gone through a few alternatives but the main goal here is to be able to look up the data in a decentralised censorship resistant manner. So I can't rely on looking at blockchain explorer apis. Also if I use bitcoin RPC I would need to be connecting to a specific bitcoin full node which could get taken down. – Dimitree – 2019-12-15T00:11:50.440

2If you want something censorship resistant, your only choice is running your own node, and indexing the blocks in its data yourself, and then querying that. – Pieter Wuille – 2019-12-16T00:50:36.277

is there no way for me to mock what a full node does (but only for part of the sync process)? Like surely I can say, hey I'm a full node and I'd like to start initial block download from X block and then validate the transactions in there. Is this not handled through bitcoinP2P ? Sorry in advanced if I have made any errors here – Dimitree – 2019-12-17T01:05:53.203

@PieterWuille I might not fully understand your comment from December, but in actual fact I managed to do what I was hoping, in theory you can 'mock' a full nodes functionality perfectly fine if you simulate the initial ver-ack messages. You can view the full source code here: https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py

– Dimitree – 2020-05-04T13:02:34.120

0

It turns out the main issue with the above code was that the the endianness in python was defaulting to my hex payload being mixed up. I simply needed to correct the ordering of the payload. The correct code can be viewed here: https://github.com/dummytree/blockchain-botnet-poc/blob/master/MalwareManager.py#L284-L299

0

I don't see any endpoint to get the whole transaction in Blockcypher API documentation, so I used blockstream.info API:

>https://blockstream.info/testnet/api/tx/2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1
{
"txid":"2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1",
"version":2,
"locktime":0,
"vin":[
{
"txid":"382bda6ee2bac22afd051104af16146436895ecca382a76c2d66535a837254bc",
"vout":1,
"prevout":{
"scriptpubkey_type":"v0_p2wpkh",
"value":2948920
},
"scriptsig":"",
"scriptsig_asm":"",
"witness":[
"023532fa6b866f37729719a3615ef3a776407048989736ac2b7fafa5678a519df3"
],
"is_coinbase":false,
"sequence":4294967295
}
],
"vout":[
{
"scriptpubkey":"6a0e3132372e302e302e313a38313831",
"scriptpubkey_asm":"OP_RETURN OP_PUSHBYTES_14 3132372e302e302e313a38313831",
"scriptpubkey_type":"op_return",
"value":0
},
{
"scriptpubkey":"a914171f697fe358f7d7238b6128930bb1fa7363b44d87",
"scriptpubkey_asm":"OP_HASH160 OP_PUSHBYTES_20 171f697fe358f7d7238b6128930bb1fa7363b44d OP_EQUAL",
"scriptpubkey_type":"p2sh",
"value":948920
},
{
"scriptpubkey_type":"v0_p2wpkh",
"value":1500000
}
],
"size":248,
"weight":665,
"fee":500000,
"status":{
"confirmed":true,
"block_height":1612110,
"block_hash":"000000000000592589e55cda6e8a093998e8356ea770d4aaeb7c0f5439b147d7",
"block_time":1576024567
}
}


You can see here that the first output has the data is 3132372e302e302e313a38313831, which decoding to ASCII characters seems like a local URL:

hex = '3132372e302e302e313a38313831'
decoded = binascii.unhexlify(hex).decode('ascii')
print(decoded)


Prints: 127.0.0.1:8181

EDIT:

To get the transaction that interests you without using any external API you will need to run a full bitcoin node with the txindex=1 option. If this transaction is tracked by your wallet, a pruned node would suffice. Then you can get the information about the transaction with this command:

>bitcoin-cli getrawtransaction 2599dbe540a583ede3512fef9a0f26be718c039ffd4d04d85ff3b339f40e73b1 true


You should get something very similar to the answer from blockstream's API I pasted above

sorry if it wasn't clear, but I needed to do this in a decentralised way without relying on stored data from an API server (for example if they just respond with the incorrect data). I'm trying to read this data straight off the blockchain and can't find any way to do this – Dimitree – 2019-12-16T00:14:02.050

Check the edit in the final part of my answer, it explains how to do that without an external API – Pedro – 2019-12-16T00:25:11.613

0

The bitcoin p2p protocol does not directly permit transaction lookups this way. Maybe some bloom filtering would minimize the footprint, but it's still relatively complex to implement.

Electrum servers are maybe the next best thing - they do support the lookups you want, but they're less decentralized, and if your application is doing naughty things they could collude and manipulate only your results.

How do I get my block or headers response to only contain 1 inv item instead of 500/5000

I don't think you can limit the headers results (you have to know all the headers to do so) But once you have the header chain, you can send a block locator with start:HASH(n) end:HASH(n+1) and you can download a single block.

Re bloom filtering, if you find nodes with the BLOOM_SERVICE bit you can tell them your filter, and the block data only contains your txs and the false positives allowed by your filter.