Chasing Zero:
The Quest for Standards of Validity
By Steve N. Jackson
Hilbert College
February, 2005
The Rimm Report: Why Structures of Validity are Important.
In 1995, Carnegie-Mellon University student Marty Rimm published a report entitled Marketing Pornography on the Information Superhighway: A Survey of 917,410 Images, Descriptions, Short Stories, and Animations Downloaded 8.5 Million Times by Consumers in Over 2000 Cities in Forty Countries, Provinces, and Territories in the Georgetown Law Review (Rimm, 1995a). The report was a scientific study of pornography on the Internet, claimed a worldwide research scope, and was directed at the marketing, consumption, and trading of pornography.
Rimm’s survey of pornographic material studied how people accessed erotic images on private adult BBS (Bulletin board system, an electronic dial-in billboard containing computer information) and on Usenet news site. To conduct this study, he convinced owners of these adult BBS to let him look at the log files (1) of all of their customers during a specified time period. Using a computer program that could read and parse text he did an analysis of the content of the image files on the BBS. He used the text to divide images into soft and hard core, and to decide if they involved coercion, illegal acts, or showed signs of being violent or degrading.
Rimm also did a content analysis of images found on Usenet binary discussion groups. To study Usenet, Rimm downloaded every image from “five popular Usenet boards over a four month period”. He also collected information on the Internet use habits at an undisclosed “private, mid-sized university in the northeast” as related to pornography, apparently taken from computer networking logs covering 4227 internet users (Rimm 1995a). He used this to build a research-based picture of the Internet and the World Wide Web.
As recounted from his “Summary of Significant Results of the Carnegie Mellon Study,” “71% of Usenet pornography comes from commercial "adult" bulletin board services (BBS)”, while “83.5% of all images posted on the Usenet are pornographic.” His report also claimed that “Paraphilic, hebephilic, and pedophilic imagery…” accounted for more than half of the downloads from billboards, while “At the university studied, Usenet newsgroups containing pornography account for thirteen of the top forty groups.” The language of Rimm’s conclusion was very critical of the Internet as a whole, implying that the Internet was a highway of “free pornography”. In particular, Rimm described the business practices of BBS managers and worldwide nature of pornography.
Rimm’s report did not stop with sharing his findings on Internet pornography. He commented on the lack of government action connection to a single convicted pornographer, saying, “…law enforcement efforts have done little to discourage consumers or indeed the market leader, Robert Thomas from discussing, trading, and redistributing these images,” calling Thomas the “The Marquis de Cyberspace.” He also raised issues of Privacy, when he disclosed that pornographic Internet sites kept track of usage, saying that, “Unscrupulous network administrators are in a position to compile and sell detailed online information about each of their users to third party vendors.” In the process Rimm directed some harsh criticism at Internet culture, discussing the “Phenomenology of a Porngeist,” and calling the images “Digital Cultural Artifacts” (Rimm, 1995a).
At the same time the Rimm study was introduced, Senator Jim Exon (D- Nebraska) was proposing an amendment to the 1934 Telecomunication Act known as the Exon-Coats amendment (Exon-Coats, 1995) that would eventually become the Telecommunication Decency Act. With debate on the floor of congress peaking, but chances that a bill regulating speech on the Internet fading due to constitutional concerns, news of the studies pending release gave it national attention that few academic studies get. Time magazine, which had been covering an ongoing debate on Internet censorship at Carnegie-Mellon signed Rimm to an exclusive contract and featured his findings on the cover of their July 3 rd , 1995 issue. Adding to the article’s prestige was a set of legal commentaries by three law professor, Anne Branscomb, Catherine MacKinnon, and Carlin Meyer that was included after the Rimm article in the Georgetown Law Review (Godwin, 1997).
The Rimm report, embraced by Time , loudly discussed in the halls of congress, and widely quoted by public figures of every description, from feminist theorists to conservative religious activists, produced by a prestigious research university with a reputation for serious research, turned out to have a number of fatal flaws. Brian Reid, a Digital Equipment Corporation Internet researcher said, “Normally, when I am sent a publication for review, if I find a flaw in it I can identify it and say ‘here, in this paragraph, you are making some unwarranted assumptions’. In this report (Rimm’s study) I have trouble finding measurement techniques that are not flawed (Reid, 1995).” In response to the Time magazine article Donna Hoffman and Thomas Novak of Vanderbilt University’s Owen Graduate School of Management released a Detailed Critique of the Time Article “On a Screen Near You: Cyberporn ” (Hoffman, 1995). Mike Godwin, the Electronic Frontier Foundation staff counsel, whom Rimm had contacted for critique of a small section of his article, also released a critical piece on the Rimm report (Godwin, 1995).
The first theoretical flaw was in how the study was conceived. A study of the Internet should look at the Internet. If the whole Internet is not looked at, then some part of it has to be looked at, and evidence has to be presented that would convince the reader that the part looked at was demonstrative of the Internet as a whole. Rimm looked at the content of adult Billboards and found out that they contained ‘shocking’ erotic images, but the adult BBS he looked at were not on the Internet, they were private dial-up services (Hoffman, 1995). Rimm’s look at a few binary image groups were also suspect because he did not say that they represented only a small number of the interest groups on Usenet. The unnamed university that Rimm used to study the habits of Internet users turned out to be his own Carnegie Mellon University. Carnegie Mellon is dominated by its technology programs, and has a significant majority of male students. Assuming that the sexual habits of University students, a majority of whom are male, young, and unmarried mirror that of the United States as a whole requires a stretch of imagination that most scholars were unwilling to make (Reid, 1995).
Rimm didn’t take the criticism lightly. He immediately responded to the flood of claims that his report was flawed. In reply to Reid’s claim that Rimm’s statistical methods and even his understanding of the Internet were flawed, Rimm said that he had “consulted with a faculty member in the department of statistics,” in writing the report, assuring their validity. The letter also said that Rimm had, “hoped that the statistics we inserted would be taken in a general sense” (Rimm, 1995b). He also decried many of Reid’s criticism, bringing up issues of “intellectually honest discussions,” as if Reid’s critique was less than honest. His reply to Hoffman and Novak was even harsher. Hoffman and Novak had criticized the Time article because it had not been peer reviewed, to which Rimm pointed to the faculty members that had critiqued his paper, to the three law professors that had written commentaries on his study, and to the reputation of the Georgetown Law Review . He also spent a great deal of time discussing his credentials and impartial position on the subject. Rimm also claimed that Hoffman and Novak were not qualified to judge the research project as professor of management trained in psychology (Rimm, 1995c). In both the Reid reply and the Hoffman and Novak reply, Rimm did not really defend his methods (except for pointing the critics back to his article) as much as he defended the prestige of the organizations and peoples that supporting his findings.
The study was only embraced by Time for less than three weeks. By July 24 th , Time partially retracted its story on Internet pornography, claiming that Rimm had developed “credibility problems.” It turned out that Rimm was author of a book called The Pornographer’s Handbook: How to Exploit Women, Dupe Men, and Make Lots of Money , offering practical advise on how to market pornographic images. Despite the retraction and the firestorm of evidence against Rimm, Time ended the article by saying “It would be a shame, however, if the damaging flaws in Rimm’s study obscured the larger and more important debate about hard-core porn in the Internet.”
This lukewarm retraction is important in understanding why structures of validity are important for scholars. Time Magazine , for its own reasons, wanted a “larger and more important issue of hard-core porn in the Internet.” Rimm’s report, which they called the Carnegie Mellon study, was believed because it was done by Carnegie Mellon and printed in the Georgetown Law Review . The Rimm report was news, and it sold magazines. From the point of view of the journalists at Time , the Rimm report seems to be saying what they already believed. Since they already believed the Internet was rife with porn, an article that said it was scientifically would seem to have been an easy sell to that magazine.
In fact, Rimm’s study was never peer-reviewed. The Georgetown Law Journal was not a journal that uses peer review. Instead, student editors choose stories. The students that put together the Georgetown Law Review are learning to be lawyers, not social scientists, and would have no background in Internet technology, social science, or statistical methodology. The law journal may even have had an unusual bias that led it to publish the article based upon its findings and not its quality. According to Mike Godwin the law journal’s editor at the time Rimm’s report was published was Deen Kaplan, who was also the vice-president of the National Coalition for the Protection of Families and Children (Gadwin). His opinions on pornography on the Internet are fully spelled out in a letter, written the month before the Time article was published, to Senator Exon in which he states, “We believe that such legislation (The Exon-Coats bill) is vital to the well being of our nation's most important resource, its children” (Kaplan, 1995).
The Politics of Validity
The entire case of the Rimm study is an example of cognitive acceptance and cognitive dissidence. First put forward by Leon Festinger in his book The Theory of Cognitive Dissonance , Cognitive Dissonance theorizes that people build world models based upon their previous experience, and data to the contrary to their world experience is likely to be rejected because it conflicts with that inner template. On the flip side, data that agrees with their world concept is likely to accepted, some times uncritically (Festinger, 1957). Without knowing Deen Kaplan we cannot really know why the Georgetown Law Review accepted the Rimm article, but since the article affirmed Kaplan’s expectations, and supported a point he considered important, he was obviously blinded to the articles flaws.
The three scholars that wrote supporting pieces to the article may have also been led into that support by cognitive acceptance. Catharine MacKinnon, a professor of law at the University of Michigan, sees pornography as a civil rights issue with serious consequences to how women live their lives (MacKinnon, 1988). A report that seemed to support her work on the effect of pornography on women would naturally be looked at favorably, if in fact the research was valid. In fact, she may never have seen an entire copy of the report she report and article supporting. Mike Godwin, in an article for Hotwired magazine claims that each of the researchers where only shown small pieces of the study. Only Rimm, his faculty advisor, and the staff at the Georgetown Law Review , itself with a hidden agenda, had seen the entire piece before its publication. Godwin also claims that the pieces shown in each case presented a different sent of assumptions that played along with each scholar’s previous work. When they wrote their supporting articles, they were ignorant of the entire scope of the work (Godwin, 1994).
Rimm’s research struck a cord with the outside for the same reason. A careful reading of the Time magazine piece shows that while Time certainly had a copy of the same report that went into the law review (they quote from it extensively), they didn’t read it very critically. Comments from scholars such as University of Chicago sociologist Edward Laumann show they did not have an understanding of what they were commenting on, and were likely only shown the results, not the methods that led to the results. Rimm studied dial-up adult billboards that check age of user and confirm it with a credit card number, then wrote a whole section on the ease of access to pornography on the World Wide Web, a service he hadn’t even looked at. This is an “apples and oranges argument”, study apples (and find out that the group you are looking at are red in color) and then claiming oranges are also red because you were actually looked at all fruit.
Defining the Metric of Zero
The a priori assumptions of the people who wrote commentaries in the Georgetown Law Review represent the zero in the formation of their understanding of information. Numbers are a human created construct used to pass precise descriptions of information from one person to another. The modern number line contains zero as a place of balance. Zero is the ultimate neutral. If I take the equation 1+1+1=3 and add +0, I still get 3 as an answer. The balance has not changed. When a scholar comments on an article, or a journalist writes a story, the reader assumed that the data they present represents some form of balanced consideration. That the journalistic zero or the scientific zero has been set in such a way that the data presented can be compared with all of the other data the reader is presented with. Validity, and intersubjective agreement that arises from two sides understanding and agreeing that conclusions are valid, demands that
Rimm’s study was a case of playing on the weakness of our systems of validity. Rimm, by manipulating the zero point he showed to people, got them to support his research. He then used the validity of their work to mask the problems that his own work had. In the mass society, political groups with a political agenda embraced Rimm’s work not because it was valid, but because it agreed with their own concepts of zero. In an odd way, people constructed a reality that would have made any existentialist proud. The issue of validity is tied up with the concept of the position of zero point.
The neutral or zero point is the assumed standard by which we build a system of understanding. Our understanding of the neutral resting-place of our ideas colors our understanding of the world and allows us to create and judge information. This zero point is reached by a reading of previous people’s work.
Thomas Kuhn is perhaps the most famous of the people who have questioned the methods of scientific study. In his 1962 book, The Structure of Scientific Revolutions , Kuhn proposes a historical model of scientific exploration that is at odds with the classical model of progressive gathering of knowledge. Kuhn sees the process of theory building and testing not as one of continuos refinement, but of dramatic shifts of understanding, which he calls paradigm shifts. Most research, according to Kuhn, occurs to flesh out theoretical understanding, with the questions asked and answered safely within the bounds of the theory. Some of that research leads to unexplainable problems in the theory, which is usually overlooked. These problems can build though, until it becomes obvious that the current theoretical model can no longer be stretched to explain them, at which a new model (paradigm) is invented and the old one discarded (Kuhn, 1961).
Kuhn’s model is a simple but powerful extension to the understanding of scientific theory based upon circumstantial historical evidence. By looking back on how scientific breakthroughs happened, he was able to formulate a predictive model of how scientific understandings come about and are debunked. He also brought humility to scientific study that has touched many scholars. A quick jaunt to a college bookstore shows his influence. If a college has a graduate program, its bookstore probably stocks a copy of The Structure of Scientific Revolutions . Many graduate level survey textbooks have sections devoted to Kuhnian understanding of scientific progress, and few research scholars in any field will be completely ignorant of his work.
Kuhnian thinking though assumes that everyone in the field agrees on zero, so that errors in theory can be exposed during testing. This assumes that terms are narrowly enough defined that we can be sure that the zero point chosen for one subject is appropriate for another subject, and that a zero point is possible at all.
The assumption of a valid metric and agreed zero are essential to Kuhnian science. A metric is a measurement tool that allows the establishment of comparisons such as big versus small, long versus short, or even same versus different. Communicating information assumes that a metric can be converted into a language. Many of the “Hard sciences” use mathematics to communicate metric. In speaking this language a scientist isusing a precise code to transmit information. The precision of this information transmission is critical to the answers that scientists get (Kramer, 1986).
An example of this is an attempt to use straight numbers in a historical setting. On July 3, 1863, at the battle of Gettysburg, 7,000 American soldiers died during a single charge, now called “Pickett’s Charge”. We can discover that number, and that they were born in the “United States of America” by looking at records kept by people of the. This is a classical fact. The number 7,000 can be counted and statistical measures applied to it (for example, if 12,000 people were involved in the charge and 7,000 died, then 7 in every 12 or 58.3% died). As a classical fact, I can also dispute that number with another number. If my research uncovers a casualty list from Gettysburg with less than 7,000 people listed as having fallen in that charge, I can suggest that the number currently held by scholars is in error. We can embrace the number 7,000 and say with certainty that “this is a fact”.
Knowing a simple number, such as the number of casualties in a civil war battle, is not the main purpose of science. These facts are actually used as building blocks for theoretical understanding. A theory is a template that a scientist uses to understand the why an event proceeds in the way that it does. That template of theory both predicts what tests should be performed to check validity, and what answers can be expected on the tests. Classically, theory is the continuity that connects current understanding with past and future understanding. Work done in the past forms the basis for current theory, and work done in testing current theory will be used to form theoretical perspectives for the future. A simple fact is only a small part of a mosaic of information that forms the templates through which a scientist understands a subject.
Assigning numbers to human action requires careful attention to not only the measurability of a subject but the foundation of theory that underlies our assumptions. In Rimm’s case, he assumed that assigning a number to perversion, in his case based upon viewing pornography, that he could assess a social ill. His theoretical assumptions said that pornography comes in two types, socially dysfunctional and socially unacceptable, and he looked at how often the later was made available to the former. His work struck at a problem with human studies: it is very difficult to make theories around human behavior. In fact, even the “true effects” of pornography are difficult to understand. According to a report by Edward Mulrey and Jeffrey Haugaard, “Surgeon General Koop's survey concluded that only two reliable generalizations could be made about the impact of exposure to "degrading" sexual material on its viewers: it caused them to think that a variety of sexual practices were more common than they had previously believed, and it caused them to more accurately estimate the prevalance of varied sexual practices” (Mulray, 1986). This sort of ambiguity where the number may not lie, but we don’t have a clue if they are valid measure, is a basic difficulty of human research.
We can measure an object like a plant and come up with a number in a metric. Metrics are measurement tools that establish big versus small, long versus short, or even same versus different. We assume that the absence of the plant would give us a height of zero. If the plant is there, and assuming we measure it correctly, we will get a height, which we will express in meters. A meter is an agreed upon meter. If another researcher asks how long a meter is, they can look through the literature and discover that a meter is 1,650,763.73 wavelengths in vacuum of the radiation corresponding to the transition between levels 2p10 and 5d5 of the krypton 86 atom. A meter is a consensual illusion that works because its definition is widely known and precise.
Now, assume that I measure that plant and get a measure of a meter tall. A fellow researcher measures it and discovers it is actually 3 kilograms. We are using different metrics so we cannot possibly compare them in any useful fashion. Scientists that primarily use numbers to communicate can solve this problem by looking at the metrics in use. The measures that quantitative scientist’s use are tightly defined, differing metrics are easily caught by a trained researcher even if they pass inspection by a casual reader. In the case of the Rimm report, only three days elapsed between the article being discussed in Time , and being debunked by a number of scientists.
The problem with quantitative research is that the more complex a system, the harder to define it in numbers, and the more likely those numbers will suffer from a defective understanding of the metric. Studies of human culture and experience suffer from this problem. Humans are so complex, and their actions are controlled by so many cultural variables, that human studies are often forced to rely on other systems of understanding for description. Quantitative studies use numbers to establish metric, qualitative studies use textual language (2). Language allows a much broader understanding of a subject.
Qualitative Metrics
Qualitative studies, with their reliance on textual language, are susceptible to the same problems of metric and appropriate zero point as quantitative studies. The difficulty is not the validity of qualitative studies, but the difficulty of hiding the assumed zero. Although credited with bringing classical scientific methods to Anthropology, Franz Boas seems to have done an even greater service the other direction, he brought classical science the anthropological method. With the release of Charles Darwin’s Evolution of Species , scientists of every description seemed intent on applying evolution to their own works. Evolution made a nice neat package of chaotic systems, and it was simple enough to test through observation. The study of very complex systems, which includes the way human’s interact with each other, embraced evolution with paradigms such as “Social Darwinism” to explain the perceived superiority or inferiority of groups of peoples and their social customs.
Franz Boas’s work was very different from previous attempts to place human systems into neat categories. Instead of being reductionistic, looking for one clear truth in complex systems, Boas approached his study from in a holistic manner. This holistic manner of study meant that an anthropologist was not intent merely on isolating variables of culture in the human biome and studying them as discrete elements, but in studying culture in situ and as a comprehensive whole. Boas believed that to study the systems (culture) under which humans live, a researcher had to look at their language, religion, consumption patterns, and even their dance. In reaching conclusions about a group of people, Boas learned as much about every facet of the culture as he could (Stocking, 1974). In some ways, Boas’s descriptive approach to the scientific study was similar in style to the writer of travelogues, with two exceptions. Boas’s work created and used theoretical models to aid in understanding complex systems, just as the traditional physical scientists did, and his writings were backed by systematic and in-depth notes from his study. A second researcher could, if they wished, judge Boas’s work based upon his theoretical perspective and based upon notes from his observations, just as they could any other scientist. The only difference was that a cultural finding was more complex that a finding in a field like chemistry, and since variables weren’t isolated, harder to reproduce.
Boas’s work left an unintended legacy. While Boas was able to redefine how scientists defined knowledge in relation to complex human systems, his “holistic” system of looking at information in depth was not easy to apply. Scholars such as Margaret Mead and Derek Freeman had to move into a culture and observe it from within. That description from within was like no other area of science. In traditional science, the scientist assumes a detached persona, peering into a microcosm of a problem. This relationship plays itself out like a visit to a psychotherapist. The observed sits on a couch open to the eyes of the observer, who uses training to cast off his or her positional shackles and discover the “real” causes of a patient’s discomfort. Traditional science, as known and accepted by physical scientists, usually prefers data that can be classified as right or wrong.
The problem with simplistic dualistic thinking in regards to a complex issue is demonstrated by Karl Heider’s article The Rashomon Effect (1987). Published in response to the so called “Mead-Freeman” debate, the article describes the Rashomon Effect, named after Akira Kurosawa’s hit 1959 movie where accounts of a rape and murder are told differently by four different witness-participants, each of whom saw events through a different filter of reality. Heider’s work is ground breaking because it sees the researcher as an element effecting the data collected and the interpretation of that datum. It also poses a number of problems in understanding academic research of complex subjects. This recognition that classical definitions of right and wrong are not very useful in studying complex systems causes a number of problems for academics. When a classical fact could be pointed to and declared a truth, science no longer leaves us easy interpretations.
Boas, in proposing that inferior / superior labels could not be used in ethnography, predicted Heider’s Rashomon Effect, but also left the door open to the question can information on culture be judged? The Mead-Freeman debate showed the worse case scenario for Boas’s ethnographic tradition. Two separate ethnographers, visiting the same group of people, come up with two opposing answers that are difficult to combine, especially since one of the participants, Margaret Mead, was dead by the time Derek Freeman presented his views.
The Rashomon Effect creates a tool that scholars can use to judge differing research. Instead of dualistic acceptance / rejection of a hypothesis, it recognizes that complex data sets, such as generated by field ethnography, should not be judged as complete answers to any question, but as positional judgements of the data present. Positional reality is a simple term that means each person, due to their place in a event, and the filters which their world knowledge places on information, will come up with a unique viewpoint. To understand an event or a culture with more than one positional account as evidence a scholars has to decode things like researcher bias to arrive at a view that can be understood as reliable. Researcher bias, in a traditional scientific model, is a slander of a researcher’s ethics and ability. Heider’s work sees researcher bias as inevitable and just another factor in judging research.
This makes a lot of sense for research of complex systems. With a huge number of variables, the weight given to any particular variable as it relates to the other will depend on the bias of the researcher. This bias can be expressed in where we place the zero in our research equations, and how we apply metrics. When judging academic work, many scholars are forced to look at factors outside of the research at hand to judges validity.
The Credibility Game
Source credibility, according to some of the classical research by Karl Hovland, is how much we trust the source of the data we are receiving, which effects how much weight we give it (Hovland, 1951). If the canon (or body of information that is “accepted” as “true” by the “recognized” leading practitioners of a field) says 7,000 people fell in one particular charge at Gettysburg, and I present a number of 6,000 as more accurate, my source’s credibility will effect how my datum is received. In a study that disputes a simple number like a casualty figure, the question of the authenticity of my claim will normally rest on the perceived credibility of the source that I am basing my claim on. Academic arguments seem to rest on the credibility of methodology, or “did I go about collecting my data correctly”. This credibility of source is the hinge which traditional scientific rests.
Source credibility though points to another way in which academics seem to judge others work, and that is the credibility of the person or people giving the work. In the Mead-Freeman debate-which spurred Karl Heider to write his groundbreaking article, Derek Freeman attacks not only the findings of Margaret Mead, but her credibility as a researcher. In Freeman’s research, Mead’s work is characterized as sloppy, and representative of wishful thinking on the part of an unprofessional researcher. He sees her as a naïve young women away from home and family for the first time, trying to do an ethnographic study at a time when she was not prepared for the rigors of the work (Freeman, 1998). Margaret Mead’s high reputation in the field of anthropology means that a reader must be convinced of her professional failings in order to accept Freeman’s thesis.
Thus, my claim for a revisiting of the casualties from Pickett’s charge hinges not only on my ability to bring up a credible source, but on my own personal credibility. In a communication class I taught several years ago, I used the case of the Tasaday Tribe to demonstrate this. The Tasaday tribe, consisting of about two dozen souls, supposedly wandered out of the jungles on the island of Mindano and were found by Manuel Elizalde, a cabinet minister for national minorities. Dafal, a local islander, had claimed to known of the tribe, having found them in the jungle during a hunting expedition. The story was treated with skepticism, until American journalist John Nance arrived and spoke (through an interpreter) to the Tasaday. Nance’s book, “The Gentle Tasaday”, led to an outpouring of interest in the Mindano tribe. Soon CBS, NBC, and National Geographic would be featuring the tribe in video productions and on magazine covers.
Within a couple of years though, professional anthropologists were questioning the “finding” of the tribe. After a Swedish and two German journalists visited the tribe and presented evidence that the tribe was a hoax, the Anthropology community mostly dropped study of the Tasaday, despite persisting claims by Nance and Elizalde (Eichenberger, 1998).
Are a small group of people calling themselves the Tasaday really a “lost tribe”, or was it a hoax created for political reasons? Unless we visit Mindano, learn the dialect spoken by the Tasaday and ask them ourselves, it really comes down to a question of source credibility. Do you believe the truth as transmitted by Nance and Elizalde, of that as transmitted by the three European journalists?
A counter example is the case of Ishi, last of the Yahi, a tribe of Native Americans that lived in what is today California. If a tribe in 1971 could be made up out of whole cloth, could a single person in much the same situation also be a myth? Many would find the suggestion troubling, since Ishi was described in anthropological literature by no less a person than A. L. Kroeber (Kroeber, 1976). Theodora Kroeber wrote a popular book called Ishi of the Two Worlds that popularized the last of the Yahi (Kroeber, 1988). Pictures of Ishi and Kroeber exist, and I have heard an authentic sounding recording of his voice that would have had to be made on a wax cylinder. Pictures of Ishi and Kroeber exist, and Ishi is looks the part of a Native American freshly descended from the hills, down to his shoeless feet and crooked tie (as if he was wearing his first suit for the picture). Dozens of boxes of information collected on Ishi supposedly exist, stored at the University of California, San Francisco, including the doctor’s report of an exam the showed him to be suffering from Tuberculosis, a disease that would take his life in 1916. Of course there is no evidence that Ishi was anything other than what he was reported to be, the last member of a tribe of Native Americans, but it is to easy to take the Tasaday as an obvious hoax without exploring how we really know about it.
Chasing Zero: Rethinking Human Validity Structures
Some anthropologist’s see the Tasaday as an example of another error of our scientific system. In The Artificial Savage: Modern Myths of the Wild Man by Roger Bartra, Bartra puts forward a theory that the Tasaday were accepted uncritically because they fit a stereotype of the “savage” (Bartra, 1997). The metric that researchers apply to the world may include a zero point that defines a concept of “savage” on a metric scale of civilization. When the Tasaday were described they fit the assumed metric and were accepted as a fact by the academic community.
This assumption of a metric of modernity can color how qualitative and quantitative scientists present information. The question that faces someone searching for validity is at what point Bronislaw Malinowski, in his book Argonauts of the Western Pacific , used a metric of economics when he discussed the benefits derived by the Kula ring. For Malinowski, proving that the Trobrianders had an economic dimension placed them onto a scale of modernity to which they could be judged against western culture (Malinowski,1984). This works both ways. Margaret Mead in her popular book Coming of Age in Somoa: A Psychological Study of Primitive Youth for Western Society presented a Somoan female metric of sexuality to the western peoples (Mead, 1988).
Mead’s work in particular is a target of the search for validity. In the book The Fateful Hoaxing of Margaret Mead: A Historical Analysis of Her Samoan Research , Derek Freeman discusses how Margaret Mead was trapped by her own desires and misconceptions into creating an idyllic land of female sexual choice and freedom that did not really exist. Using a style of writing I call “mouse in the pocket,” he discusses how his own experiences in Somoa proved Mead’s analysis invalid. Debunking Mead has become almost the life’s goal of Derek Freeman, and he created an insightful work that completely decodes the reasons why Mead was the wrong person at the wrong time to be studying the Somoans.
Freeman’s metric argues that Mead’s zero-point was flawed. She could not have seen and done what she claims. The mouse in Freeman’s pocket allows him to see clearly than Mead, despite what we know about the Roshamon Effect (Heider, 1984). His metric though may be equally flawed. Carol Tavris argues in Mismeasure of Woman that the metrics we apply to human studies may be biased by our assumptions of a male norm (Tarvis, 1993). Applied to Freeman’s critique, it would support Heider’s contention that position is important when information from qualitative research is decoded.
Anthropologist’s have been challenging many of the assumed zero points that can create direct both the academic and popular understanding of human studies. Boas deconstructed race as a biological concept, but left only culture in its place. In an absence of discourse on race people with hidden agendas have made popular in roads at redefining race around politically convenient definition. Hernstein’s Bell Curve , supported by Pioneer Foundation (3), is an example of a hidden eugenics agenda disguised as research. This is an activist model of research, that of a tool for transformation, much in the vein of Faye Harrison’s arguments about decolonising anthropology (Harrison, 1997).
These activist movements are very important, but they are also a danger to finding a structure of validity. Instead of chasing a fleeing zero, and fencing with meter sticks, scientists instead need to accept that metrics are not easily transferable without becoming vague generalities, and that our popular expressions of research may create unintended images. Scholars need to accept that arguments like the Freeman – Mead debate are part of the academic process. Without them we would drown in Bell Curves and Rimm reports . That said, we should be more open to accepting that research can be right for a moment but not completely and forever correct through out time. This means we have to become more tolerant of revisits to subjects, and be willing to understand that research can have political dimensions that need to be discussed. (4)
Notes
A log file refers to lists of machines that requested data from a server. A log file normally consists of the time of request, what information was being requested, and where it was being sent. This (usually) only consists of the IP address of the users computer, which in the case of dial-up Internet connections such as AOL is different each time the customer connects. Content that needs a credit card number to access, or is contained in a private database such as the adult BBS mentioned by Rimm in his study, may have much more detailed log files. They can include the name and address of the user and even the user’s credit card number.
I should perhaps note that this does not assume that either qualitative or quantitative research is superior, just like I would not assume that an apple or an orange is superior without careful definition of what I mean by superior. Most quantitative studies require qualitative thinking in the formation of the study’s basic assumption, and in the description of the conclusion, while many qualitative studies will use quantitative methodology in forming their basic assumptions. Both the Rimm study (Rimm, 1995a) and the Bell Curve (Hernstein, 1996) make qualitative mistakes in their basic assumptions that bring their validity into question before the first statistical proof is trotted out. In fact, you would have a hard time calling Bell Curve a quantitative study, it is really a qualitative study that uses statistics to validate a conclusion.
Rajiv Rawat, in his article The Return of Determinism (Rawat, 1995), traces the works cited by the Bell Curve and discovers some interesting connections. Available on the Internet, it is a good critique to read after tackling the 800 page Bell Curve .
Literature Cited
Bartra, Roger. (1997). The Artificial Savage: Modern Myths of the Wild Man. Michigan: The University of Michigan Press.
Beauvois, Jean-Leon. (1996). A Radical Dissonance Theory. London: Taylor & Francis Publishers.
Berlo, D.K., and J.B. Lemert and R.J.Mertz. (1969). Dimensions for Evaluating the Acceptability of Message Sources. Public Opinion Quarterly. 33(4):563-576.
Bernal, Martin. (1994). The image of Ancient Greece as a tool for colonialism and European hegemony. In Bond, George, and Angela Gilliam, Social Construction of the Past: Representation as power. (1994). Routledge: London.
Boas, Franz. (1896). The Limitations of the Comparative Method in Anthropology. Science, n.s., 4:901-908. In Anthropological Theory: An Introductory History. R. Jon MacGee and Richard L. Warms. Mayfield Publishing: Mountain View.
Catton, Bruce. (1992). America Goes to War: The Civil War and Its Meaning in American Culture. Wesleyan University Press: Middletown CT.
Cohen, Bernard. (1963). The Press and Foreign Policy. Princeton: Princeton University Press.
Cohen, Jeff, and Norman Solomon. (1995). How Time Magazine Promoted A Cyberhoax. Media Beat. July 19. Online: http://www.fair.org/media-beat/950719.html
Christiano, Paul. (1995). Research Project on Internet Pornography. E-Mail. Online: http://www.cs.cmu.edu/~jch/rimm/inquiry-report.txt
Darwin, Charles. (1993). The Origin of Species: By Means of Natural Selection or the Preservation of Favored Races in the Struggle for Life. New York: Modern Library.
Eichenberger, Bill. (1998). Writer still has more questions than answers about Tasaday tribe. The Columbus Dispatch, July 22. Online: http://www.dispatch.com/news/newsfea98/july98/nance0722.html
Elmer-Dewitt, Phillip. (1995a). On A Screen Near You: Cyberporn. Time Magazine. July 3 rd . 146(1). Online: http://cgi.pathfinder.com/ Time /magazine/archive/1995/950703/950703.cover.html
Elmer-Dewitt, Phillip. (1995b). Fire Storm On The Computer Nets: A new study of cyberporn, reported in a TIME cover story, sparks Controversy. Time Magazine. July 24 th . 146 (4) Online: http://cgi.pathfinder.com/Time/magazine/archive/1995/950724/950724.internet.html
Exon, Jim. (1995). Amendment No. 1362 by [Senators Jim J.] Exon and [Daniel R.] Coats To Amdt No. 1288 S. 652. Online: http://www.eff.org/pub/Alerts/s652_95_a1362_exon_coats.amend
Festinger, Leon. (1957). A Theory of Cognitive Dissonance. Stanford, CA: Stanford University Press.
Freeman, Derek. (1998). The Fateful Hoaxing of Margaret Mead: A Historical Analysis of Her Samoan Research. Boulder: Westview Press.
Geertz, Clifford. (1973). Interpretations of Cultures. New York: Basic Books.
Gero, Joan. (1994). Gender divisions of labour in the construction of archeological knowledge in the United States. In Bond, George, and Angela Gilliam, Social Construction of the Past: Representation as power. (1994). Routledge: London.
Godwin, Mike. (1997). JournoPorn: Dissection of the Time Scandal – Rimm’s Fairy Tales. Hotwired. Online: http://www.hotwired.com/special/pornscare/marty.html
Gordon, Edmund T. (1991). Anthropology and Liberation. In Decolonizing Anthropology: Moving Further toward an Anthropology for Liberation. Faye V. Harrison, ed. pp. 149-167. Washington, DC: American Anthropological Association.
Gould, Stephen J. (1996). The Mismeasure of Man. New York: W.W. Norton & Company.
Harris, Marvin. (1968). The Rise of Anthropological Theory: A Histories (sic) of Theories of Culture. New York: Crowell. In Historicism by Jonathon Berry. http://www.as.ua.edu/ant/murphy/histor.htm
Harrison, Faye. (1997). Decolonizing Anthropology : Moving Further Toward an Anthropology of Liberation. Washington D.C: American Anthropological Association.
Headland, Thomas. (1992). Washington D.C.: The Tasaday Controversy : Assessing the EvidenceAmerican Anthropological Association.
Hernstein, Richard, and Charles Murray. (1996). The Bell Curve: Intelligence and Class Structure in American Life. New York: Free Press.
Hoffman, Donna, and Thomas P. Novak. (1995). A Detailed Analysis of the Conceptual, Logical, and Methodological Flaws in the Article: "Marketing Pornography on the Information Superhighway". Online: http://www2000.ogsm.vanderbilt.edu/novak/rimm.review.html
Hovland Carl, and W. Weiss. (1951). The Influence of Source Credibility on Communication Effectiveness. Public Opinion Quarterly. 15:4:635-50.
Kaplan, Deen. (1995). “Letter of support to Sen. James Exon, D-Neb., read into the Congressional Record.” Online: http://www.pathfinder.com/@@vEcLeQUApiVYGAzT/pathfinder/politics/netpol/famletter.html
Kingsolver, Ann. (1998). More Than Class : Studying Power in U.S. Workplaces. Albany: State University of New York Press.
Kramer, Stephen, and Felicia Bond. (1987). How to Think Like a Scientist : Answering Questions by the Scientific Method. New York: Ty Crowell Co.
Kroeber, Alfred. (1976). Handbook of the Indians of California. New York: Dover Publications.
Kroeber, Theodora. (1988). Ishi in Two Worlds: A Biography of the Last Wild Indian in North America. Berkley: University of California Press.
Kuhn, Thomas. (1962). The Structure of Scientific Revolutions. Chicago: University of Chicago Press.
Lazarsfeld, Paul F. and Bernard Berelson and Helen Gaudet. (1968). The People’s Choice: How the Voter Makes Up His Mind in a Presidential Campaign. New York: Duell, Sloan, and Pearce. Originally published 1948.
McKinnon, Catherine, and Andrea Dworkin. (1988). Pornography and Civil Rights: A New Day for Women's Equality. Organizing Against Pornography.
McKinnon, Catherine. (1995). Vindication and Resistance: A Response to the Carnegie Mellon Study of Pornography in Cyberspace. The Georgetown Law Journal. 83: 1959.
Miles, Mathew and A. Micheal Huberman. (1994). Qualitative Data Analysis, A Sourcebook of New Methods. Beverly Hills: Sage Publications.
Malinowski, Bronislaw. (1984). Argonauts of the Western Pacific. Prospect Heights, IL: Waveland Press. Originally published in 1922.
Mead, Margaret (1988). Coming of Age in Somoa: A Psychological Study of Primitive Youth for Western Society. New York: William Morrow. Originally published in 1928.
Mulrey, Edward and Jeffrey Haugaard. (1986). Surgeon General's Workshop on Pornography and Public Health. Arlington, Virginia: U.S. Department of Health and Human Services.
Mukhopadhyay, Carol, and Yolanda Moses. (1997). Reestablishing “Race” in Anthropological Discourse. American Anthropologist. 99(3), 517-533.
Nance, John. (1975). The Gentle Tassday: A Stone Age People in the Phillipine Rain Forest. New York: Harcourt Brace Jovanovich.
Orans, Martin. (1996). Not Even Wrong: Margaret Mead, Derek Freeman, and the Samoans. Novato, CA: Chandler & Sharp Publications.
Patton, Michael. (1990). Qualitative Evaluation and Research Methods. Beverly Hills: Sage Publications.
Purdy, Barbara. (1996). How to Do Archaeology the Right Way. Gainesville: University of Florida Press.
Rawat, Rajiv. (1995). The Return of Determinism. Online: http://www.englib.cornell.edu.SciTech/u95/bell.html
Reid, Brian. (1995). Critique of the Rimm Study. Digital Equipment Corporation. http://www2000ogsm.vanderbilt.edu/novak/brian.reid.critique.html0
Rimm, Marty. (1995a). Marketing Pornography on the Information Superhighway: A Survey of 917,410 Images, Descriptions, Short Stories, and Animations Downloaded 8.5 Million Time s by Consumers in Over 2000 Cities in Forty Countries, Provinces,and Territories. Online: http://TRFN.pgh.pa.us/guest/mrtext.html Originally published in the Georgetown Law Journal, Volume 83, Issue 5.
Rimm, Marty. (1995b). Response to Letter from Brian Ried. Online:http://TRFN.pgh.pa.us/guest/mrbr.html
Rimm, Marty. (1995c). A Detailed Critique A Detailed Critique of the TIME Article "On a Screen Near You: Cyberporn" Online: http://TRFN.pgh.pa.us/guest/mrcc.html
Sternthal, B. and L. W. Phillips and R. Dholakia. (1978). The Persuasive Effect of Source Credibility: A Situational Analysis. Public Opinion Quarterly. 42(3):285-314 .
Stocking, George. (1974). The Shaping of American Anthropology, 1883-1911: A Franz Boas Reader. New York: Basic Books.
Strauss, Anselm. (1987). Qualitative Analysis for Social Sciences. Cambridge: Cambridge University Press.
Symposium: Agenda Setting Revisited. (1993). Journal of Communication. 43:58-127.
Toulmin, Stephen. (1992). Cosmopolis : The Hidden Agenda of Modernity. Chicago: University of Chicago Press.
Travis, Carol. (1993). The Mismeasure of Woman. New York: Touchstone Books.
Winant , Howard. (1994). Racial Conditions: Politics, Theory, Comparisons. Minneapolis: University of Minnesota Press.
Wolf, Eric. (1982). Europe and the People without History. Berkeley: University of California Press.