[ot][spam][crazy] crazylog: append-only random-access data
it's nice to think about this algorithm part i'm indoors right now and where i was working on it was outdoors. i'll just open up an unsync'd copy here that system is offline; i wonder how in the world i got my code onto it O_O maybe i copied it over by hand? maybe i started dev on it? i dont' think ihave the camera sorted out for qr codes, or a ready usb key
so things are split between chunk regions and flush regions makes for a stronger cognitive challenge, to change the source in multiple concepts at once i guess it's just inspiring to add in names that seem a little more intuitive. it is of course confusing that both are 'regions', when juggling memory fragments
i think where i left off, i was trying to kind of rewrite the add() function to just plain use the new flush regions. without regard to all the old techniques. simplification.
each flush region has a left write and a right write. these are passed to the constructor. a flush can happen two ways: 1 - new flush. starts with nothing in it. adds only writes. 2 - old flush. starts with a previous flush and no writes. new writes can break up the reference to the previous flush.
i'm a little confused around the difference in code paths between new flushes and old flushes. but, noting that adding writes to an empty flush is the same as adding writes that are sparsely distant from the bounds of a previous flush.
it seems it would be good to get something that functions, and uses flush regions, just like how i started the old approach. even if it doesn't have most of the things planned to include.
it seems the core behavior is adding a write. this branches various conditions depending on whether it intersects a previous flush, a previous write, opens rebalancing, etc.
so very basic code would simply be to add writes as edges between flush regions and not do any rebalancing at all. simply use the chunk region class to subdivide them. then we would move straight to the core inhibited part of retaining binary-like trees that are distant from write areas.
so a write has two ends, a start and an end, and each one can either be within an old write, within an old flush, or in a new area. then similarly, they can span all or parts of writes and flushes, as well. the old code uses start_idx and end_idx and then excises that range. the ends are treated specially. the most important part seems to be adjusting their bounds. if a write or a flush is partially cut, its bounds can be adjusted, and if it is a flush then it can be further optimised as it will have lost leaves.
the current written bounds are earliest where old_end >= new_start, and earliest where old_start > new_end earliest where old_end >= new_start : if this isn't off the right end, then it intersects or touches the new data. if end == start, it touches. if end > start, it overlaps, unless it is off the right end. earliest where old_start > new_end : this looks like an upper bound. it would be the first one that _doesn't_ intersect. so it produces a half-open range of mutated regions.
what happens if a write does not overlap a region ...? maybe it would make sense to think of the region of note as being the one prior to the upper index
I'm a making a Write class to balance the concept of writes with flushes. This helps make concepts around the design have less complexity for me.
class Region: def __init__(self, *entries, left_write = None, right_write = None): self.entries = entries self.left_write = left_write self.right_write = right_write class Write(Chunk): def __init__(self, offset, data, flush, left_region = None, right_region = None) super().__init__(offset, offset + len(data), data, age=flush.age) self.left_region = left_region self.right_region = right_region def __init__(self, prev_flush = None): self.regions = [] self.writes = []
there are many different kinds of complexity i'm noting how the kinds i am handling are somewhat indicated by the parts of the code i have improved vs left
so there are two factoring concepts here one is the linked lists, for organising of functionality the other is the semibalanced trees.
i have so much semibalancing inhibition, that the linked lists get inhibited too. i am already weak around linked lists due to graph inhibition i experience. so i'm mostly working on the linked lists now, and am presently planning to balance the trees after i have a workable framework to do so in.
the linked lists swap back and forth with regard to the types of data linked it makes sense to have two different classes for these, so they can have separate methods.
it also makes sense to put them in one list, so one can quickly find what is overlapped
a nice thing here is that i have existing stable code so as i conceptualise and lose things more, a workable approach may trend toward changes to stable code, that keep the code stable.
it seems it would make sense to give the flush regions and the writes a common base class of being a node in a linked list. i think there are also some parts that could optionally be removed or simplified, since writes are a special thing used only in the latest flush, they don't need to have chunkregions.
maybe Chunk can be the linked list node, since everything that is a chunk is a member of a linked list. noting also that the list index is available when accessing the nodes, so the link members might be mostly for simplification, or a nonverbal possible plan i'm not thinking of much yet
having the two separate classes seems it would be helpful for design. putting methods on them.
next step i think: find recent stable code, such as whatever was last posted regarding thinking/spamming, could use more stabilisation around idea of separating writes from flush regions into peer classes possibly with different methods. this plan could be tuned more by planning the semibalancing.
here's a recent stable code. this was before the 'fiction' approach i believe; classes could still be hoisted to be subclasses. https://lists.cpunks.org/pipermail/cypherpunks/attachments/20220716/dec3d2ee...
regarding thinking/spamming, could use more stabilisation around idea of separating writes from flush regions into peer classes possibly with different methods. this plan could be tuned more by planning the semibalancing.
This was the latest-dated reference I found in the July archives with a relevant subject line. On July 22, 2022 3:09:36 AM EDT, "Undiscussed Horrific Abuse, One Victim of Many" <gmkarl@gmail.com> wrote:
here's a recent stable code. this was before the 'fiction' approach i believe; classes could still be hoisted to be subclasses. https://lists.cpunks.org/pipermail/cypherpunks/attachments/20220716/dec3d2ee...
participants (2)
-
Undiscussed Horrific Abuse, One Victim of Many
-
Undiscussed Past Horrific Abuse, One Victim Of Many