karl3＠writeme.com wrote:
karl3＠writeme.com wrote:
( >( >( (Pdb) p input.shape torch.Size([1, 6, 16384]) (Pdb) p weight[0:16].T.shape torch.Size([16384, 16]) input @ weight rows @ cols so one row of input is [0,0,:] then one col of weight.T is [:,0] these are dotted. now, weight.T is dense on the first dimension. so ideally we'd make stripes across the other dimension. uhhhhhhhhhhhhhhhhhhhhhhhhh [_,_] @ [_,_] [_,_] @ [_,_] rows x cols rows of left times cols of right so the output is just a concatenation differently on each side. the rows of the left can be treated independently. the cols of the right can be treated independently. [_,_] @ [_,_] [_,_] @ [_,_] rows x cols the dense portion of the second operand is the dimension that is broadcast. so, since it's rows @ cols, the dense portion of the second operation would be the columns. the columns are dense. well that's frustrating. this would work better if they were stored sideways. if i get all of one col and none of the next it doesn't really help me. it's not summed inside the dot product. this may still be an appropriate thing to do, but it would involve math elsewise in the operator graph rather than right here looks like http does actually support sparse requests: https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Range_requests#mult... i wonder what a server would do if i sent a huge sparsity document for a range header. there must be some kind of maximum (there might not be) ... likely other issues too
i think the underlying store operates in sizes of mmap pages. so ideally that would backpropagate somehow to the slicing operator and i could also use the surrounding data that is in the same page. maybe a separate function to fetch the smallest encompassing slice--[someday