Re: Pre-allocating key segments
-----BEGIN PGP SIGNED MESSAGE----- Piete Brooks <Piete.Brooks@cl.cam.ac.uk> writes:
I wrote:
Not only that, but the client ought to allocate some keyspace before it needs it, as I think one other cpunk suggested.
I'd prefer to keep the number of segments "lost" if a brloop ceases.
Keep down, I guess you meant. Regarding the local farm software:
... if any keys were allocated to machines which failed to ask for another segment -- you should assume that that segment was not searched.
I agree that is the best policy -- it fails safe -- but I still think the prefetching of some more segments would be useful. The goal is to suck up as many idle cycles as is practical.
For instance, if it has four segments allocated and it's done three of them, it should fork a process to begin requesting four more segments *while* it is scanning the last segment, rather than waiting until after it is done and leaving the machine idle until it can alloc more keys.
That means that if it crashes, 8 segments are left unACKed :-(
And if it had grabbed 8 segments to begin with and crashed, it still would have been 8 segments left unACKed. Plus, it's only 8 segments unACKed if it crashes before it finished that last segment, since it will start trying to ACK the first four segments when it finishes the fourth -- at the same time starting on the next four segments. Would you see it any differently if I had said, "For instance, if it has two segments allocated and it is halfway through the second segment, it should request two more segments *while* it is scanning the last segment"? Keeping in mind that it will still ACK the first bunch of segments when it finishes them. -----BEGIN PGP SIGNATURE----- Version: 2.6.2 iQCVAwUBMEIOnBEcrOJethBVAQHUfQP+OUA+iC7sTp2CVCZ5YqtM7ouNykhyx7Nm agcTHN6FFZUOxDmAogiY/Op/SLBZbgtmACC3RSG0cEHwzCQJZ6jeUrTe9g3qU/Vm jHRn8PurOUYE188QnZSGEj0qcZbeoYJoLE4qOcrd7SbizIcZoWk/WVA4STZwEHuH wHHusza6Un4= =UOqi -----END PGP SIGNATURE----- -- David R. Conrad, ab411@detroit.freenet.org, http://www.grfn.org/~conrad Finger conrad@grfn.org for PGP 2.6 public key; it's also on my home page Key fingerprint = 33 12 BC 77 48 81 99 A5 D8 9C 43 16 3C 37 0B 50 Jerry Garcia, August 1, 1942 - August 9, 1995. Requiescat in pace.
I'd prefer to keep the number of segments "lost" if a brloop ceases. Keep down, I guess you meant.
Indeed -- Ta.
... if any keys were allocated to machines which failed to ask for another segment -- you should assume that that segment was not searched. I agree that is the best policy -- it fails safe -- but I still think the prefetching of some more segments would be useful.
I'm seeing calls from calpoly.edu and albany.net taking less than a second. Are you **REALLY** worried about wasting that sort of time, when even a single segment usually takes a quarter of an hour even on the faster machines ?
The goal is to suck up as many idle cycles as is practical.
I don't think a second's overhead (practical with local cache) is significant.
That means that if it crashes, 8 segments are left unACKed :-( And if it had grabbed 8 segments to begin with and crashed, it still would have been 8 segments left unACKed.
Sure, but I'd prefer you allocate single segments .....
Plus, it's only 8 segments unACKed if it crashes before it finished that last segment, since it will start trying to ACK the first four segments when it finishes the fourth -- at the same time starting on the next four segments.
Sure. [ Getting down to the implementation details 1) it would be hard for brloop to know that brutessl is 3/4s done. 2) I can't think how to do prefetching in a safe way, and without disc use ]
Would you see it any differently if I had said, "For instance, if it has two segments allocated and it is halfway through the second segment, it should request two more segments *while* it is scanning the last segment"?
No. If you gave me code which would guess how long the request for the next segment will take, and then know when brutessl is that many milliseconds from completion, and can tell that brloop isn't going to die within that time, sure :-)) If someone supplies info for my "PS3:", I can generate central stats on what %age of Hal3 was wasted waiting on the server. Otherwise, brloop users will have to scan their own logs (if enabled) and work it out (latest brloop happens to log when brutessl starts and finishes).
participants (2)
-
ab411＠detroit.freenet.org -
Piete Brooks