[ot][spam][crazy][random][crazy]

[Undescribed Horrific Abuse, One Victim & Survivor of Many]

Ok, the bug is indeed on my end.

At https://viewblock.io/arweave/tx/GK_ffFFjA39-FuvzXoJ47CQvon6uWYRYix55LGcj0tk
one can see that this transaction is not a bundle. It's gzipped data
from KYVE, a data streaming project that I haven't looked into
extensively.

1935

So, for some reason my code thinks this transaction is a bundle.

I've restarted pdb and stepped into the code that lists the bundles.

(Pdb) n
> /shared/src/log/download.py(141)block_bundles()
-> self._cache_block(block)

Guess I'd better check that function out

(Pdb) l
136                     if block == current_height + 1:
137                         pending = self.peer.tx_pending()
138                         return self._txs2bundles(pending,
f'{len(pending)} pending txs', unconfirmed = True)
139                     else:
140                         raise KeyError(f'block {block} does not exist yet')
141  ->             self._cache_block(block)
142                 bundles = self.bundle_cache[block]
143             return bundles
144         def block_height(self, block):
145             if type(block) is list:
146                 for block in block:

Looks like the bundles are indeed enumerated in the _cache_block function.

(Pdb) list
119                     tags = self.peer.tx_tags(txid)
120                 if
any((ar.utils.b64dec_if_not_bytes(tag['name']).startswith(b'Bundle')
for tag in tags)):
121                     bundles.append(txid)
122             return bundles
123         def _cache_block(self, block):
124  ->         block = self.fetch_block(block)
125             bundles = self._txs2bundles(block.txs, f'Caching
{block.height}')
126             self.bundle_cache[block.height] = bundles
127             self.height_cache[block.indep_hash] = block.height
128             return block.height, bundles
129         def block_bundles(self, block):

Now I'm in txs2bundles and I found where it selects bundles and placed
a breakpoint.

(Pdb) list
116                 if unconfirmed:
117                     tags =
Transaction.frombytes(self.peer.unconfirmed_tx2(txid)).tags
118                 else:
119                     tags = self.peer.tx_tags(txid)
120                 if
any((ar.utils.b64dec_if_not_bytes(tag['name']).startswith(b'Bundle')
for tag in tags)):
121 B->                 bundles.append(txid)
122             return bundles
123         def _cache_block(self, block):
124             block = self.fetch_block(block)
125             bundles = self._txs2bundles(block.txs, f'Caching
{block.height}')
126             self.bundle_cache[block.height] = bundles
(Pdb) p txid
'gF9DQa99YKsNKeDHQLOT2inuQuBGvLUtTsGD9NMeVM8'

That's the first bundle it thinks it found in this block.
The filter for what is a bundle is visible here too. It looks for tags
with names that start with the string "Bundle". Maybe the KYVE
transaction matches that.

BundleSize
100
FromKey
3638932
ToKey
3639031
BundleSummary
0x4463f4562edc86bc82521cae492ea54a6793efa3d41f654e3a60348e4abaf6ab

Yeah. This tx isn't a bundle.

I can check the ANS-104 and ANS-102 specs to see what is a bundle.
I could also add checking code to my implementation so it is more
user-friendly in the future :S this would be a good idea, but since
this isn't actually my current task I might not this time :S which is
maybe not the best idea, unsure.

Arweave bundle specs:
https://github.com/ArweaveTeam/arweave-standards/blob/master/ans/ANS-102.md
https://github.com/ArweaveTeam/arweave-standards/blob/master/ans/ANS-104.md

Their tags are Bundle-Format and Bundle-Version.
I guess I'll just be more strict like that.

[user@ log]$ git diff download.py
diff --git a/download.py b/download.py
index 448addb..23721a9 100644
--- a/download.py
+++ b/download.py
@@ -117,7 +117,7 @@ class Stream:
                 tags =
Transaction.frombytes(self.peer.unconfirmed_tx2(txid)).tags
             else:
                 tags = self.peer.tx_tags(txid)
-            if
any((ar.utils.b64dec_if_not_bytes(tag['name']).startswith(b'Bundle')
for tag in tags)):
+            if any((ar.utils.b64dec_if_not_bytes(tag['name']) in
(b'Bundle-Format', b'Bundle-Version') for tag in tags)):
                 bundles.append(txid)
         return bundles
     def _cache_block(self, block):

[user@ log]$ python3 download.py hellouniverse.json
Caching 1056142: 100%|██████████████████████████████████████████| 5/5
[00:00<00:00, 10.92tx/s]
Caching 1056143: 100%|████████████████████████████████████████| 40/40
[00:06<00:00,  6.22tx/s]
Caching 1056144: 100%|████████████████████████████████████████| 24/24
[00:02<00:00,  8.81tx/s]
Caching 1056145: 100%|██████████████████████████████████████████| 5/5
[00:00<00:00,  7.01tx/s]
Caching 1056146: 100%|████████████████████████████████████████| 72/72
[00:08<00:00,  8.66tx/s]
yielding capture @ 0
channel data: capture: 4
yielding capture @ 4
channel data: capture: 4
yielding capture @ 8
channel data: capture: 4
yielding capture @ 12
channel data: capture: 4
yielding capture @ 16
channel data: capture: 1
Hello, Universe!

It works :D