[ot][spam][crazy] Fiction: Index Trees for Random-Access Data on Append-Only Storage
It is a beautiful day in Considering Append-Only Trees Land. The sun is shining brightly on 20 kilobytes of source code. Source Code Part [Unidentified]: "I'm so interested in these Append-Only Trees!" Source Code Part 2 [Also Unidentified]: "Yeah? Tell me about it."
Source Code Part 1: "Weeeelllllll I just have this cool idea I want to try so much!" Source Code Part 2: "What's your idea?"
Source Code Part 3: "Oh!" Source Code Part 3 looks at Source Code Part 2. Source Code Part 2 is a class named FlushRegion!
FlushRegion [Previously Source Code Part 2]: "Oh, I may look like a class called FlushRegion, but I am actually just _holding_ this class in case it would be useful to work with. So, you can call me by its name, but unless we're turning into an AI or something I don't think the classes are talking in words right now." Programmer 2 [previously FlushRegion]: "Is FlushRegion your idea?"
Tree of Append-Only Flushes [Previously Source Code Part 3]: "Am I a programmer too?" Programmer 2: "If I am, you must be?" Programmer 3: "I'm thinking about these append-only flushes being put into a tree. I like this tree idea." Programmer 1 [reviously Source Code Part 1]: "Yes! I was thinking of making each new append be made of Flush Regions connected by Writes!"
Programmer 1 and Programmer 3 and Programmer 2 all hold hands with a bunch of other programmers. "We want to implement an append-only tree structure for random access data with revision history! That's the story we're writing right now!" They all say this in unison.
Programmer 3 [wearing shirt emblazoned with "Tree of Append-Only Flushes"]: "I got this shirt to help you remember. What's the next step for working on this?"
1: "Let's just sit and think about it a little, maybe get to know each other. I see your shirt."
4: "Ummmmmm when I stop thinking, I start thinking about Boss and how wonderf--" 2 ["FlushRegion"]: "Don't say that! "Linked List" over here is having freedom trouble. I almost called it mind contr-" 5 ["ChunkRange"]: "Don't say that!"
4 ["Linked List"]: "There are a few of us here. 2 is "FlushRegion", 5 is "ChunkRange", 3 is "Tree of Append-Only Flushes". 3, why don't you change your shirt to say "Flush Index"? That's more similar to class names." 3 ["Flush Index"] (previously Tree of Append-Only Flushes): "Okay !" 1 : "I'll be Tree of Append-Only Flushes!"
3: "We like thinking about these things. FlushRegion, Flush Index, Linked List, maybe ChunkRange." 2: "Could be next steps for a Tree of Append-Only Flushes!"
1: "Okay, so FlushRegion is a class for edges of a linked list, roughly. Each Flush Index is a length of them. FlushRegion sits between nodes, which are either writes of data, or terminators." 4: "There's been some confusion as to whether FlushRegion should be made of ChunkRange's, or entire Flush Indexes, or what. The confusion has been complicated by ideas around refactoring the Flush Index and changing the name of Flush.Entry to ChunkRange. We propose filling FlushRegion with ChunkRanges, because it is simpler to consider, involving fewer new concepts, new names, and new connections between concepts."
1: "Okay, so FlushRegion is a class for edges of a linked list, roughly. Each Flush Index is a length of them. FlushRegion sits between nodes, which are either writes of data, or terminators. It can be filled with adjacent ChunkRanges." 5 ["ChunkRange"]: "There is a lot of code managing the concepts around ChunkRange, previously Flush.Entry, to keep constraints like nonoverlappingness, height, or redundancy within various bounds. I hear that considering this code has confused people some."
Programmer 1: "I looked back over the minutes and surveillance tapes" Programmer 2 ["FlushRegion"]: "We have those?" Programmer 1 ["Considering Append-Only Trees"]: "Do we? Anyway, I think my shirt should say Considering Append-Only Trees. I like this label." Programmer 2 ["FlushRegion"], Programmer 3 ["Flush Index"], and Programmer 4 ["Linked List"] come together and say: "We like it too!" Programmer 5 ["ChunkRegion"]: "I want to do more than just Chunk Region. I like the idea of working on those old concepts _around_ the Chunk Regions. The ones that for some reason people find a little confusing now."
"Considering Append-Only Trees". They wore a shirt saying this. They sat in a workroom with their friends, also wearing shirts. One of the friend's shirts said "FlushRegion". Another's said "ChunkRange (Flush.Entry) Concepts". They were having a pleasant time doing what was written on their shirt, with their friends. After some time, "Considering Append-Only Trees" asked a question. Considering Append-Only Trees: "What is the relevant, important thing we're doing with the ChunkRanges in the FlushRegions?" If functionality for this were added first, it would ensure the work fully included its reason in its structure.
ChunkRange (Flush.Entry) Concepts: "There was an idea of making the flushes have similar properties to the "dat project"'s protocol, where the trees are built full, binary, and balanced, to keep indexes small. I had to go look through my own records to find this." Considering Append-Only Trees: "Right. If we consolidate the ChunkRanges between the writes, we can keep the indexes really small. This was the big idea."
Building ChunkRanges Between Writes [previously Considering Append-Only Trees]: "Okay, this is what we're trying to do!" ChunkRange Concepts: "I am certain we can do this." FlushRegion: "Me too." Building ChunkRanges Between Writes: "I'm still considering append-only trees. But this seems to be a big thorn in the considering, so I'm focusing on it."
Building Append-Only Trees: "Let's implement the ChunkRanges between the Writes. What is needed first?" ChunkRange Concepts: "I'd like to look at the code we have so far."
They had changed. Now their shirt said "Building Append-Only Trees". They were attending closely with a friend wearing a shirt saying "ChunkRange Concepts". It seemed like the two would be able to accomplish something together. Building Append-Only Trees: "What does the code say to you?" ChunkRange Concepts: "We'll need to factor in a way to bring the entries from previous flushes into the FlushRegion."
Building Append-Only Trees hoisted a giant class called FlushRegion, and carried it into another class, the Flush Index class. Building Append-Only Trees: "You can be called Flush.Region now!" A friend of theirs changed their shirt to say "Flush.Region" instead of "FlushRegion". But the person wearing the shirt saying Building Append-Only Trees was only just getting started!
I have made the 1 change mentioned in the previous email to my copy of the source on this system. I have not made other changes from the copy that I believe is at https://lists.cpunks.org/pipermail/cypherpunks/attachments/20220716/dec3d2ee... .
Once upon a time, long long ago, in a sunny glade far far away, a tree for random-access data on append-only storage was building. It was a happy young data structure, with a handful of substructures for interconnecting its parts, and a few methods for mutating and inspecting it.
participants (1)
-
Undiscussed Horrific Abuse, One Victim of Many