A Preliminary Discussion of Methodological Peculiarities in the Rimm Study of Pornography on the "Information Superhighway"

Please Distribute Freely

The Georgetown Law Journal is about to publish the results of a study by Marty Rimm of Carnegie Mellon University on "Marketing Pornography on the Information Superhighway: A Survey of 917,410 Images, Description, Short Stories, and Animations Downloaded 8.5 Million Times by Consumers in over 2000 Cities in Forty Countries, Provinces, and Territories." The study has recently been the subject of a cover story in Time magazine ("Cyberporn," July 3, 1995). Rimm has claimed that the methodology and results were extensively reviewed by Carnegie Mellon faculty (see "Cybersensitivity," Washington Post, page C1, June 28, 1995); whether or not that was the case, it appears that the Georgetown Law Journal did not similarly make the study available to outside reviewers (other than the three commentators -- Anne Wells Branscomb, Catherine MacKinnon, and Carlin Meyer) prior to publication. As a member of the Georgetown University faculty with research interests in this area, I was approached in March, 1995, to help several of the student editors with questions that they had arising out of the study; they would not, however, show me a copy of the study itself, and they asserted that they were unable to do so because of a secrecy arrangement they had made with Mr. Rimm.

One would have, perhaps, more confidence in the results of the Rimm study had it been subjected to more vigorous peer review. What follows is a preliminary list of some of the methodological oddities that I have uncovered after review of a pre-publication copy of the study that the Law Journal editors made available to me on June 26 (the publication date for the Time story). THIS LIST IS NOT, NOR IS IT INTENDED TO BE, EXHAUSTIVE; I anticipate that other such oddities will emerge as the interested community takes a more careful look at these results in the coming weeks and months.

1. Usenet Groups. Rimm's study of Usenet groups was confined to those groups with the "alt.binaries" prefix (p. 1865).
   
   The researchers determined that "[s]eventeen of the thirty-two alt.binaries newsgroups located on the Usenet contained pornographic images" (p. 1867). During a single seven-day period (9/21/94 to 9/27/94), the researchers logged 827 image postings to the "non-pornographic" newsgroups (Rimm's descriptor), and 4206 image postings to the "pornographic newsgroups." Thus, of the 827+4206=5033 images posted, 83.5% (4206) were to newsgroups that contain pornographic material.
   
   Preposterously, in his "Summary of Significant Results of the Carnegie Mellon Study, Rimm writes that "83.5% of all images posted on the Usenet are pornographic." The correct conclusion, of course, is that 83.5% of the images posted to a subset of newsgroups (the alt.binaries newsgroups) are to newsgroups that contain pornographic images. Rimm's conclusion is the precise methodological equivalent to the following: (a) restricting a study of printed pornography to magazines located in the "adult" area of a bookstore, (b) finding that 83.5% of the reader submissions during a one-week period were to magazines that contained "pornographic" material, and concluding (c) that 83.5% of all reader submissions to all magazines are pornographic.
   
   
2. Usenet, II. Rimm writes:
   The best data concerning network pornography consumption comes from the Usenet, which itself constitutes only 11.5% of Internet traffic. Of this 11.5%, approximately 3% by message count, but 22% by byte count (e.g., 2.5% of total Internet backbone traffic) is associated with Usenet newsgroups containing pornographic imagery" (p. 1869).
   Thus, by Rimm's own figures (which he chooses not to highlight), then, fewer than one-half of 1% of the messages on the Internet (3% of 11.5%) are "associated with" newsgroups that contain pornographic imagery; since some (many? most?) of those messages are, presumably, not themselves pornographic, the actual proportion of pornographic messages is therefore even smaller than that.
   
   
3. "However, as this study makes clear, studying pornography according to consumption, as opposed to availability, provides a much more revealing picture of the marketplace" (p. 1869). Although Rimm's figures show that of the forty most popular newsgroups worldwide "only one -- alt.binaries.pictures.erotica -- contained encoded pornographic images," (p. 1871), he claims that "when the data is (sic) classified by percent of news readers who subscribe to the newsgroups, three of the five most popular newsgroups are pornographic. Moreover, 20,644 of the 101,211 monthly Usenet posts in the top forty newsgroups, or 20.4%, are pornographic" (p. 1873).
   
   Oddly, no data are presented to support this claim, i.e., no data classify newsgroups by "percent of newsreaders who subscribe to the newsgroups." Nor is it clear whether Rimm, as he appears to claim, actually looked at 101,211 Usenet posts in the top forty newsgroups in order to determine that 20.4% of the postings "are pornographic."
   
   
4. World Wide Web. In his Summary of Significant Results, Rimm reports that "[p]edophilic and paraphilic pornography are widely available through various computer networks and protocols such as the Usenet, World Wide Web, and commercial 'adult' BBS" (p. 1849). No evidence is presented to demonstrate that such material is available anywhere on the Web. Indeed, in the Appendix dealing with the results of a March 1995 Web Survey (Appendix C), Rimm reports locating only 123 Web sites containing any "sexually explicit imagery or materials," (p. 1923), only 9 of which had any "pornographic material" at all. Rimm provides no information that any of these sites -- which constitute, in any event, far less than one-tenth of 1% of all Web sites -- contain pedophilic or paraphilic material.