[ot][spam][crazy][random][crazy]

[Undescribed Horrific Abuse, One Victim & Survivor of Many]

I'm trying to download it.

It still has [legacy?] code that checks the pending txs.

Arweave doesn't provide access to data that's in pending txs, as far
as I know. I think you need to listen on ports to get the data ... it
might even need to come from users who upload it directly, I'm not
sure. At least Arweave exists, even if it's full of compromises.

My usual strategy is to repeat the download, waiting for blocks to mine.

It's downloading very slowly, which implies (i guess it's still using
a gateway?) the thing it is contacting is responding slowly for some
request or another.

I kind of use HTTP things that aren't fully documented to read only
parts of the data. The situation of how those work may have changed.
But it seems more likely the data is just still propagating. Not sure.

I'll step into it and see where the holdup is.

1924

I hit ctrl-C when it was paused, and it was actually in CPU execution
rather than network blocking.

it's parsing a _huge_ bundle. It's on bundled item # 253681 . That is
a lot of items.

Due to a bug in pdb it's not showing me the total item count.
1926

it looks like this large bundle is
GK_ffFFjA39-FuvzXoJ47CQvon6uWYRYix55LGcj0tk that it is slowly parsing.
for each entry, it seeks to the area in the bundle and parses the
header to read the tags.
this generally means an http request for partial data, or to download
the data chunk that contains the header.

1927
It's at 2150646 . That's an order of magnitude larger than before.
I'm wondering if it could have overflowed something.

(Pdb) p stream.tell()
91133
(Pdb) p idx
2150646

That doesn't look right.

Here's the code it's executing:
 82             length_id_pairs = (
 83                 (int.from_bytes(stream.read(32), 'little'),
b64enc(stream.read(32)))
 84  ->             for idx in range(entryct)
 85             )

I stepped into the stream.read function:
 44         def readinto(self, b):
 45  ->         if self.offset >= self.end:
 46                 return 0
 47
 48             if (
 49                     self.chunk is None or
 50                     self.chunk.start_offset > self.offset or
(Pdb) p self.offset
91133
(Pdb) p self.end
91133

Oops.