Are the interwebz rather big for google to index?
I strongly suspect at least one of the following holds: 1. The interwebz are rather big for google to index 2. google doesn't return in searches all indexed content on purpose Partial evidence: this list and my blog don't appear in searches.
Google stopped indexing most blogs years ago. They used to show a blog search in the dropdown. It's long gone too. Rr -------- Original message --------From: Georgi Guninski <guninski@guninski.com> Date: 6/13/18 8:08 AM (GMT-08:00) To: cypherpunks@lists.cpunks.org Subject: Are the interwebz rather big for google to index? I strongly suspect at least one of the following holds: 1. The interwebz are rather big for google to index 2. google doesn't return in searches all indexed content on purpose Partial evidence: this list and my blog don't appear in searches.
On Wed, 13 Jun 2018 18:08:16 +0300 Georgi Guninski <guninski@guninski.com> wrote:
I strongly suspect at least one of the following holds:
1. The interwebz are rather big for google to index 2. google doesn't return in searches all indexed content on purpose
google never granted lowly lusers (like us) access to everything they 'indexed'. And of course given their exponential increase in joo-christian virtues and goodness, the amount of stuff they censor must have gone from, say, 10% to 60% or so now, and counting.
Partial evidence: this list and my blog don't appear in searches.
On 06/13/2018 04:08 AM, Georgi Guninski wrote:
I strongly suspect at least one of the following holds:
1. The interwebz are rather big for google to index 2. google doesn't return in searches all indexed content on purpose
Google search has become much less comprehensive over that past few years. It's a lot harder to find old stuff. It seems like they've become far more focused on delivering personalized clickbait. And if you don't give them any history, you get random crap.
Partial evidence: this list and my blog don't appear in searches.
I hadn't noticed, but "site:lists.cpunks.org" doesn't return any results after November 2017. There are 9670 results in total. And even that is not much more than the total for 2016 alone (8123 messages). However, "grarpamp" does return results from this list through at least May 2018. About blogs, I don't know generally. But searching "mirimir vpn" shows my old VPN testing site as the second result. Google also clearly indexes Wilders Security Forums, HN, and the IVPN website. So hey, I'm not complaining. About that, anyway.
On Wed, Jun 13, 2018 at 12:45:39PM -1100, Mirimir wrote:
I hadn't noticed, but "site:lists.cpunks.org" doesn't return any results after November 2017. There are 9670 results in total. And even that is not much more than the total for 2016 alone (8123 messages). However, "grarpamp" does return results from this list through at least May 2018.
Hm, my mistake I didn't notice this. Searching for "grarpamp site:lists.cpunks.org" and then from left selecting "since last year" doesn't return any results for me.
About blogs, I don't know generally. But searching "mirimir vpn" shows
google definitely indexes some blogs @Razer, probably only the elite ones.
-------- Original message --------From: Georgi Guninski <guninski@guninski.com> Date: 6/14/18 7:03 AM (GMT-08:00) To: cypherpunks@lists.cpunks.org Subject: Re: Are the interwebz rather big for google to index? On Wed, Jun 13, 2018 at 12:45:39PM -1100, Mirimir wrote:
I hadn't noticed, but "site:lists.cpunks.org" doesn't return any results after November 2017. There are 9670 results in total. And even that is not much more than the total for 2016 alone (8123 messages). However, "grarpamp" does return results from this list through at least May 2018.
Hm, my mistake I didn't notice this. Searching for "grarpamp site:lists.cpunks.org" and then from left selecting "since last year" doesn't return any results for me.
About blogs, I don't know generally. But searching "mirimir vpn" shows
google definitely indexes some blogs @Razer, probably only the elite ones.
If its really popular or you do a direct search for a unique post. This https://www.google.com/search?q=razedbywolves+Talk+Dirty+To+Me...+Obama%27s+... turned up the post and a link to the blog. Admittedly a blogger blog, google.
On 06/13/2018 11:08 AM, Georgi Guninski wrote:
I strongly suspect at least one of the following holds:
1. The interwebz are rather big for google to index 2. google doesn't return in searches all indexed content on purpose
Partial evidence: this list and my blog don't appear in searches.
Back around 2000 or so, when Google was under initial development, I read about other search providers complaining because the NSA loaned engineers to Google to help them develop their server farm architecture. Since then I have often wished I had access to the Google search utilities NSA analysts have. Other search providers' complaints had nothing to do with the NSA being "evil" but rather, with Uncle Sam interfering in commerce by playing favorites: 'Why don't we get this kind of free handout?' I don't recall Google disputing the NSA's role in building the foundation of their physical plant and/or algorithms. In light of this paper from brookings.edu, not only do I suspect that Google can index the whole of the publicly accessible Internet, but cache all of the Internet's content as well: https://www.brookings.edu/research/recording-everything-digital-storage-as-a... Professor Villasenor indicated that the storage density now exists to record and retain /all/ surveillance data. The redundancy of network traffic (same web page to 10,000 users) cuts the size of the archive down to a tiny fraction of the network's gross traffic volume. The NSA's Bluffdale facility may greatly reduce Google's role in indexing and content storage for Uncle Sam, but once established, relationships like that only expire as/when the organizations themselves and their relevant staff members expire. Google strongly biases search results to reinforce individual "user preferences" and consumer profiles, both to provide a more addictive "user experience" and to herd users toward Google's real customers, their advertising clients. Seek and ye shall find plenty of information on that subject; a whole industry (SEO) struggles to sell services based on their analysis of Google bias, not to mention in-house efforts by other "public relations" providers to crack Google's algorithms. Search is a battlefield. The best example I remember of Google censorship involved a YouTube video based on an article about The Facebook's funding sources - describing how the said company was capitalized to go national by DARPA financiers, and The Facebook's shocking abusive TOS. After about 15 minutes of digging I gave up and concluded that the video had been taken down and Google's memory of it expunged. Later I did find a saved copy of the video in an old archive of mine. Searching at Google using the video's exact title and file name did turn it up on the first try. Previously, the same words in a different order and numerous very similar strings returned nothing related to the video in question. Searches for "approximate" versions of the title still returned nothing useful. I consider that a fine example of how Google itself engages in "negative SEO" to suppress distribution of selected content while avoiding charges of "censorship" - in a very narrow technical sense, the suppressed content does remain available to the public. "Don't Be Evil - That's OUR Job" BTW, here's the thing: https://www.youtube.com/watch?v=DIGdWsxHJlM A "loose search" now turns up a different copy someone else uploaded, probably after failing to find the original... :o)
participants (5)
-
Georgi Guninski
-
juan
-
Mirimir
-
Razer
-
Steve Kinney