Anyone can access it, but only from NARA II at College Park, MD. I'm there on a semi-regular basis and want to figure out a way to efficiently liberate the docs that are only accessible from there....because that's some weird-ass censorship. On Sat, Jan 16, 2016 at 7:05 PM, Martin Becze wrote:

Can anyone access the CREST db? Do I need any specail permission?

On Sat, Jan 16, 2016 at 2:55 PM, Michael Best wrote:

...

I'm spending a fair amount of time at NARA II. Any thoughts on how I might be able to automate retrieval of documents from the CREST database? There are about 10 million pages of CIA docs that haven't even been accessed, much less made their way online.

Since 2000, CIA has installed and maintained an electronic full-text searchable system, which it has named CREST (the CIA Records Search Tool), at NARA II in College Park, Maryland. *Over 11 million pages have been released in electronic format and reside on the CREST database, from which researchers have printed about 1.1 million pages. **In order to directly access CREST, a researcher must visit the National Archives at College Park, Maryland.* http://www.foia.cia.gov/collection/crest-25-year-program-archive

I'll take notes on the access and interface when I'm there next, hopefully next week. On Sat, Jan 16, 2016 at 7:16 PM, Martin Becze wrote:

How exactly is the accesses setup? Can we arbitrary software that accesses DB?

On Sat, Jan 16, 2016 at 7:07 PM, Michael Best wrote:

...

Anyone can access it, but only from NARA II at College Park, MD. I'm there on a semi-regular basis and want to figure out a way to efficiently liberate the docs that are only accessible from there....because that's some weird-ass censorship.

On Sat, Jan 16, 2016 at 7:05 PM, Martin Becze wrote:

...

Can anyone access the CREST db? Do I need any specail permission?

On Sat, Jan 16, 2016 at 2:55 PM, Michael Best wrote:

...

I'm spending a fair amount of time at NARA II. Any thoughts on how I might be able to automate retrieval of documents from the CREST database? There are about 10 million pages of CIA docs that haven't even been accessed, much less made their way online.

Since 2000, CIA has installed and maintained an electronic full-text searchable system, which it has named CREST (the CIA Records Search Tool), at NARA II in College Park, Maryland. *Over 11 million pages have been released in electronic format and reside on the CREST database, from which researchers have printed about 1.1 million pages. **In order to directly access CREST, a researcher must visit the National Archives at College Park, Maryland.* http://www.foia.cia.gov/collection/crest-25-year-program-archive

it looks like the files can't be downloaded or saved electronically, at all, period. They have to be printed instead

This is such bullshit On Sat, Jan 16, 2016 at 11:03 PM, Michael Best wrote:

I spoke to someone from the list privately and did a little digging - it looks like the files can't be downloaded or saved electronically, at all, period. They have to be printed instead, from one of four computers which are located in College Park, MD (transparency!). So now I'm working on a plan to print and scan files from CREST. Given the scale (11+ million pages)...

I plan to be down there on Tuesday; I'll get more information and solid numbers (document counts, print speeds, etc.) and see where things are from there. Keep your fingers crossed, I may actually be able to get something.

--Mike

On Sat, Jan 16, 2016 at 7:18 PM, Michael Best wrote:

...

I'll take notes on the access and interface when I'm there next, hopefully next week.

On Sat, Jan 16, 2016 at 7:16 PM, Martin Becze wrote:

...

How exactly is the accesses setup? Can we arbitrary software that accesses DB?

On Sat, Jan 16, 2016 at 7:07 PM, Michael Best wrote:

...

Anyone can access it, but only from NARA II at College Park, MD. I'm there on a semi-regular basis and want to figure out a way to efficiently liberate the docs that are only accessible from there....because that's some weird-ass censorship.

On Sat, Jan 16, 2016 at 7:05 PM, Martin Becze wrote:

...

Can anyone access the CREST db? Do I need any specail permission?

On Sat, Jan 16, 2016 at 2:55 PM, Michael Best wrote:

...

I'm spending a fair amount of time at NARA II. Any thoughts on how I might be able to automate retrieval of documents from the CREST database? There are about 10 million pages of CIA docs that haven't even been accessed, much less made their way online.

Since 2000, CIA has installed and maintained an electronic full-text searchable system, which it has named CREST (the CIA Records Search Tool), at NARA II in College Park, Maryland. *Over 11 million pages have been released in electronic format and reside on the CREST database, from which researchers have printed about 1.1 million pages. **In order to directly access CREST, a researcher must visit the National Archives at College Park, Maryland.* http://www.foia.cia.gov/collection/crest-25-year-program-archive

On 1/17/16, Michael Best wrote:

I spoke to someone from the list privately and did a little digging - it looks like the files can't be downloaded or saved electronically, at all, period. They have to be printed instead, from one of four computers which are located in College Park, MD (transparency!). So now I'm working on a plan to print and scan files from CREST. Given the scale (11+ million pages)...

it's almost like they want to claim transparency in empty words, and frustrate it at all points through deterrent actions... hmm!

I plan to be down there on Tuesday; I'll get more information and solid numbers (document counts, print speeds, etc.) and see where things are from there. Keep your fingers crossed, I may actually be able to get something.

can you bring in a raspberry pi with battery pack to act as a printer on the local network and then "print" to a 200G microSD slotted in it? :P #OpCIADox

Not at all weird; it seems pretty effective to me. Over 11 million pages "released" and only 10% _printed_ (as in someone would have to scan the page to get it back into electronic format) sounds like a pretty good access control system for something you don't really want to make public.

Unfortunately you're right. The weird part to me is the way it's justified and presented as a transparency tool! Double speak never dies, I suppose. =( On Sun, Jan 17, 2016 at 2:50 PM, Lee wrote:

On 1/16/16, Michael Best wrote:

...

Anyone can access it, but only from NARA II at College Park, MD.

If anyone can access the database from a laptop on the guest wireless then something like https://www.att.com/devices/netgear/beam.html would give the rest of the world access. But if access is allowed from only a NARA computer you're pretty much SOL..

...

I'm there on a semi-regular basis and want to figure out a way to efficiently liberate the docs that are only accessible from there....because that's some weird-ass censorship.

Not at all weird; it seems pretty effective to me. Over 11 million pages "released" and only 10% _printed_ (as in someone would have to scan the page to get it back into electronic format) sounds like a pretty good access control system for something you don't really want to make public.

Regards, Lee

...

On Sat, Jan 16, 2016 at 7:05 PM, Martin Becze wrote:

...

Can anyone access the CREST db? Do I need any specail permission?

On Sat, Jan 16, 2016 at 2:55 PM, Michael Best wrote:

...

I'm spending a fair amount of time at NARA II. Any thoughts on how I might be able to automate retrieval of documents from the CREST database? There are about 10 million pages of CIA docs that haven't even been accessed, much less made their way online.

Since 2000, CIA has installed and maintained an electronic full-text searchable system, which it has named CREST (the CIA Records Search Tool), at NARA II in College Park, Maryland. *Over 11 million pages have been released in electronic format and reside on the CREST database, from which researchers have printed about 1.1 million pages. **In order to

directly

...

...

...

access CREST, a researcher must visit the National Archives at College Park, Maryland.* http://www.foia.cia.gov/collection/crest-25-year-program-archive

On 1/17/16, coderman wrote:

On 1/17/16, Michael Best wrote:

...

... Unfortunately you're right. The weird part to me is the way it's justified and presented as a transparency tool! Double speak never dies, I suppose. =(

"Better than nuthin', see?"

We're obeying the law, see?

On 1/17/16, Michael Best wrote:

...

Not at all weird; it seems pretty effective to me. Over 11 million pages "released" and only 10% _printed_ (as in someone would have to scan the page to get it back into electronic format) sounds like a pretty good access control system for something you don't really want to make public.

Unfortunately you're right. The weird part to me is the way it's justified and presented as a transparency tool! Double speak never dies, I suppose. =(

Where is it presented as a transparency tool? They seem to be real clear about what they're doing: "In order to directly access CREST, a researcher must visit the National Archives at College Park, Maryland. CIA recognizes that such visits may be inconvienent and present an obstacle to many researchers." Regards, Lee

On Mon, Jan 18, 2016 at 11:58 AM, Michael Best wrote:

I have the beginnings of a plan for this; I'll share more as soon as I double check a few things and run some numbers.

I shall call it... Project OPERATION.

Mail them an 8TB drive in your foia for a digital copy of the dataset, they're only $225.