--- begin forwarded text Delivered-To: ignition-point@majordomo.pobox.com X-Sender: believer@telepath.com Date: Mon, 14 Sep 1998 21:18:32 -0500 To: believer@telepath.com From: believer@telepath.com Subject: IP: Critics Pick Apart Study on Internet and Depression Mime-Version: 1.0 Sender: owner-ignition-point@majordomo.pobox.com Precedence: list Reply-To: believer@telepath.com Source: New York Times http://www.nytimes.com/library/tech/98/09/biztech/articles/14digicom.html September 14, 1998 TECHNOLOGY COLUMN Critics Pick Apart Study on Internet and Depression By DENISE CARUSO Robert Kraut, co-author of a new study linking depression with Internet use, sounded a bit depressed himself last week. "I thought I was finished with this," Kraut, a professor of social psychology and human-computer interaction at Carnegie Mellon University, said with a sigh. He was alluding to the flood of attention -- and criticism -- that his study, titled "Home Net," had received since it was published two weeks ago. Starting in 1995, "Home Net" researchers gave PCs and free Internet accounts to 169 people in 73 families in the Pittsburgh area. After monitoring their online behavior, in some cases for more than two years, the researchers concluded that spending time on the Internet was associated with statistically significant increases in depression and loneliness. Critics assert that the study has fatal flaws that neutralize its findings and that they are appalled at the authors' far-reaching conclusions about the impact the findings might have on Internet policy and technology development. Donna Hoffman, a Vanderbilt University professor and outspoken critic of Internet research design, was unequivocal about the "Home Net" study. "Speaking as an editor, if this had crossed my desk, I would have rejected it," said Ms. Hoffman, who edits the journal Marketing Science. "The mistakes are so bad that they render the results fairly close to meaningless." Among those mistakes, she said, were the absence of two standard safeguards: a control group and random selection of subjects. "With 'Home Net,' we don't know for sure what led to their results," Ms. Hoffman said of the lack of a control group, "because we don't know what happened to people who weren't using the Internet." In addition, the study recruited people from high schools and community service organizations, instead of selecting people randomly from a large area. Random selection is crucial to building a truly representative sample of a population -- in this case, residents of the United States. The study found that one hour a week online led to small but measurable increases in depression and loneliness and loss of friendships. While those measurements might well be statistically significant, critics assert that without a random sample, they are meaningless outside the group that was studied. "The assertions have no statistical relevance to any population of Internet users beyond those in the study population -- even in principle," declared Charles Brownstein, a former director at the National Science Foundation, now an executive director at the Corporation for National Research Initiatives in Reston, Va. Although research studies do not always have to use control groups or randomly chosen participants to be valid, Ms. Hoffman said, those safeguards become imperative "when you're doing a study that claims causal relationships and that these relationships hold in the larger population." The "Home Net" team clearly made such claims. For example, the news release stated that "Carnegie Mellon Study Reveals Negative Potential of Heavy Internet Use on Emotional Well Being," and even suggested that parents move PCs out of teen-agers' bedrooms and into shared family rooms. Kraut was quoted extensively in the release, with such statements as, "We were surprised to find that what is a social technology has such anti-social consequences." Also: "Our results have clear implications for further research on personal Internet use. As we understand the reasons for the declines in social involvement, there will be implications for social policies and for the design of Internet technology." Last week, Kraut wearily defended his study. "In 1995, we did start with a control group, but it was very hard to keep it, with little in the way of incentives for them to continue to fill out questionnaires," he said. "And we couldn't use a random sample because of the nature of the study's design -- we wanted to be able to include groups who already had social connections with each other so we could observe some shifting, if it was going to occur, between existing social relationships." And despite criticism of the researchers' methods, he said that the study was widely applicable. "We have changes big enough that they aren't likely to have occurred on the basis of chance," Kraut said. "There is something here to explain." But critics like Ms. Hoffman look askance at such results, given her experience debunking other Internet studies, including one from Neilsen Media Research in which she participated in 1996 and another in 1995 in which Marty Rimm, a graduate student in Carnegie Mellon's College of Engineering, published a study purporting to show that the Internet was overrun with pornography. The impact of Rimm's study, though based on false premises and quickly discredited, was profound. The resulting furor in the media and in Congress helped bring about the Communications Decency Act, which the Supreme Court ruled was a violation of the First Amendment. Critics fear that the "Home Net" study might end up having a similar effect. "It's easy to imagine the results of this study being used to influence policy decisions about Internet access, especially in controversial funding decisions for schools and libraries," Ms. Hoffman said. But in the end, Ms. Hoffman said, protecting the Internet itself is not the point. "We're trying to protect the standard of research," she said. "This isn't obscure ivory-tower stuff that never sees the light of day. It has an impact on people's lives. If we're going to the trouble to study the Internet, at least we should make sure we're doing it right." Copyright 1998 The New York Times Company ----------------------- NOTE: In accordance with Title 17 U.S.C. section 107, this material is distributed without profit or payment to those who have expressed a prior interest in receiving this information for non-profit research and educational purposes only. For more information go to: http://www.law.cornell.edu/uscode/17/107.shtml ----------------------- ********************************************** To subscribe or unsubscribe, email: majordomo@majordomo.pobox.com with the message: (un)subscribe ignition-point email@address ********************************************** www.telepath.com/believer ********************************************** --- end forwarded text ----------------- Robert A. Hettinga <mailto: rah@philodox.com> Philodox Financial Technology Evangelism <http://www.philodox.com/> 44 Farquhar Street, Boston, MA 02131 USA "... however it may deserve respect for its usefulness and antiquity, [predicting the end of the world] has not been found agreeable to experience." -- Edward Gibbon, 'Decline and Fall of the Roman Empire'