This paper aims to outline methods of connecting statements made in computer communications with the sources of information which underlie those assertions. This process has three objectives:
We receive information from computers as a series of assertions. Some of these assertions come directly from other people, with minimal mediation from the computer systems; other assertions come to us mediated, sometimes extensively, by programs that implement rules for logical or statistical transformations. Much of the time, we lack a critical ingredient for turning these assertions into usable knowledge: we have no way to verify the assertions offered.
Verifying an assertion traditionally involves two actions: checking the assertion against other available information, and holding the people who make the assertion accountable for the accuracy of what they say. Academic communities have evolved sophisticated mechanisms, mainly through references and authorities, for determining the sources of assertions. Courts of law have probably the most stringent methods for holding people responsible for the accuracy of their assertions, but other communities have developed effective methods of holding their members accountable for the quality of the information they offer. Dr. N. A. M. Rodger's book The Wooden World provides a excellent insight into the way the Royal Navy of the Seven Years' War developed the "information economy" required to sustain a meritocratic system.
Computer technology would seem to lend itself well to the development of a knowledge exchange system that supports references and holds people accountable for the accuracy of their assertions. We still use the old conferencing systems such as Usenet, which carry noise as well as accurate information. In this paper, I will propose some technical standards, based mainly on xml, which will address the following questions:
Based on my own past research and that of other people, I propose a computer-based architecture for implementing these processes, which can potentially increase the value of knowledge exchanged on internal and external networks.
In the abstract of this paper, I have made many statements. Most of those statements express my own conclusions; you can tell by the use of phrases such as "seems to" that I intend only to express my own views. I have made other statements, which I intend to define the problem as I see it, and the way I approach it. I have made other statements, however, which make claims about the real world. I can justify some of these statements based on my own personal experience. I have to base other statements on the work, and the ideas, of other people.
We have very effective methods of referring to the sources of ideas; in fact, the language of traditional scholarship has inspired much of the language of computer technology. We speak of references, both in computer programs and on the World Wide Web, in the same spirit as scholars have made "references" in their work for centuries. For example, since I have made mention of Dr. Rodger's book The Wooden World, you can expect me to tell you what edition of it I have (Fontana 1990), and to provide the numbers of the pages on which I find the ideas I tell you about. A reference tells you how to find something; it provides information about a text, which may refer to other information, which (with a little diligence) you can follow to the original sources: the artist's painting, the soldier's letters home, the statesman's treaty, or the scientist's notes. Thus, however much interpretative scholarship between the facts and the reader, a serious work of non-fiction will always remain rooted in the truth.
Add the awesome power of the Internet to this tradition, and the many sites offering objective, often immediate economic, demographic, and scientific data of all kinds, and you would expect an explosion of well thought out arguments grounded in facts. The hypertext linking features of the web seem perfectly suited to automate the process of linking arguments and conclusions to facts. On some web sites, and in some discussion forums, you find exactly that. In many other discussion forums, neither the available technology, nor the prevailing culture, support attempts to integrate information with discussion. In some cases, this reflects the conscious choice of the participant to simply argue as a game, without putting any effort into references or facts. However, software designers have many design and standards issues to address before we may confidently claim that, when network users discuss issues without reference to information, they do so purely out of their own choice.
Specifically, we have three issues to address: making it easier to find reliable information, making it easier to reference and evaluate the information, and making it easier to engage in discussions using information.
Most of the discussions on the Internet consist of pure opinion, usually expressed in a deliberately provocative way. Since many of these exchanges have no purpose but to amuse the participants, demanding facts and references gets in the way of the fun: at best, some participants see it as another debating tactic. A standard for facts and references has nothing to offer this kind of recreational noisemaking, except another distraction. However, the Internet as a whole has a great deal more to offer than an exchange of unsubstantiated opinions interspersed with insults and arcane acronyms (ROTFL, for the uninitiated, means rolling on the floor laughing). By creating tools for information-based discussion we will not silence the verbal scrums which provide amusement to many people. But new tools may permit a different, more thoughtful, and perhaps more convincing voice to make itself heard.
To build an argument based on the facts, you must first obtain the facts in question. Complaining about a shortage of facts on the internet seems a lot like complaining about a shortage of toys at FAO Schwartz, but despite the huge amount of raw data on the internet, extracting information to support an argument faces two barriers: one cultural, and one technical. The cultural barriers consist of the tendency of web site operators to charge for access to archival information, and the reluctance of governments to put important information on-line. On the technical side, the lack of search standards makes it difficult to search for information, and the lack of presentation standards, particularly for on-line databases, makes it more difficult to integrate them with an argument.
A solution to the cultural problem probably depends on a solution to the technical problem. Until we have solutions in place, and widely accepted, which allow people to base their discussions on objective information, few people will pay for it, and without customers or standards, information providers cannot lower the barriers.
The proposed standard must therefore accomplish three significant tasks: it must allow the identification of arguments based on facts; it must provide a framework for evaluating the credibility of the information, and for evaluating the relevance of facts to arguments. Above all, it must provide a framework for encouraging an appropriate use of facts and references.
On its face, the potential for automating the evaluation of facts and arguments seems very limited, and attempts to expand and encourage it may offend many people. But while computers cannot themselves evaluate a complex argument, they can do a superb job of mustering and presenting information.
In the World Wide Web, we already have an excellent selection of reference tools, and various xml projects of the World Wide Web consortium have added to these. In xml rdf, we already have an excellent structure for describing information, and in particular, distinguishing between reports of assertion and verified data. However, to present information for assessing an argument, we need more than a structure for describing connections; we also need a structure for describing a context.
Hok Lin Leung of Queen's University, Kingston has developed a method of evaluating problems and making cases that lends itself to computer implementation. In his book Policy Planning and Evaluation: Common-Sense Structured, he describes the importance of understanding information according to the point of view of the participants. Dr. Leung also rightly insists on the primacy of values in the process of discussion and understanding. Without a framework in which participants explain their viewpoints, information remains a set of facts without context: easy to misunderstand, and easy to dispute. Without a framework to describe the values behind a viewpoint, a simple set of links to facts will neither convince anyone, nor will they provide the necessary context for any actor, human or machine, to analyse an argument.
A person's subjectivity begins with their values, both implicit and explicit. Those values inform assumptions, which they may or may not state clearly in the course of a discussion. For example, consider the form of Pascal's wager often used to justify capital punishment:
"If we execute murderers and there is in fact no deterrent effect, we have killed a bunch of murderers. If we fail to execute murderers, and doing so would in fact have deterred other murders, we have allowed the killing of a bunch of innocent victims." John McAdams - Marquette University/Department of Political Science
This oft-quoted summation makes a number of assumptions about facts: for example, that the courts do not convict innocent people, but above all, it makes assumptions about values, in this case, the value of different people's lives. It involves not merely issues of fact, such as the effect of capital punishment and the fallibility (or otherwise) of the courts, but issues of value, such as what powers courts, and by extension the communities they serve, ought to have over people's lives.
The issues, then, do not simply involve facts. They involve the facts assessed in the context of a structure of values. If the frameworks of values have nothing in common, then we have no basis for discussion. The parties can only present their cases, complete with facts, and leave the issue at that, for they will never succeed in convincing the other party. But providing a framework of values together with the facts allows the participants in a discussion to at least consider the values they have in common, and encourages people to make explicit the way their values affect their perception of the facts.
Dr. Leung defines consistency as his first test for a policy as seen in this subjective light: consistency with the actor's values, goals, and specific intentions. In the case of a framework for evaluating an argument in light of the facts, we must determine the consistency of a statement with the facts as seen through the participant's values.
For example, to a supporter of capital punishment, the importance of a strong community stand against murder may appear so great, that inevitable errors by the courts have lesser importance. Therefore evidence that the courts have convicted some people in error will not sway such people. For me, as someone who opposes capital punishment, the community must not make stands its institutions cannot justify; therefore, I view the possibility of one wrongful execution with the deepest concern.
Both of us can justify our beliefs. For a discussion on this subject, a framework for evaluating facts through values has this advantage: once we have defined our values, we must apply them consistently to other facts. To continue the example, if a supporter of capital punishment gives the importance of a strong stand by the community against murder to downplay evidence of innocent people convicted in error, they must take account of that value when addressing other questions about capital punishment in the United States. American jurisdictions execute only about two in a thousand killers, and do not even close about 20% of murder cases. According to the Bureau of Justice Statistics of the US department of Justice, the remaining offenders serve an average of about 12 years in prison . Does this really make a thundering community statement against murder?
Likewise the retentionist can expect me to square my insistence on the absolute importance that the courts not convict innocent people with the reality that murderers often kill innocent people. Both of us can amend our values and muster other facts to support our case, but we must either make our arguments compatible with the facts as seen through our values, or else pronounce ourselves willing to live with the contradiction.
The xml mark-up language provides an excellent format for evaluating information. Using special tags to link to external sources of information, users can enter the source of the information, the questions the source addresses, the answer the external source provides, the credibility they assign to the answer, and, most mportantly, the relevant values. Since each participant defines and ranks the values they consider important, The system can evaluate facts in light of each participant's stated values. Other participants may review these sources, consider the information available, and provide their own assessments. Assuming all participants in the discussion use the same scale, they can assess each other's positions using their own perceptions and values, the stated perceptions and values of the person whose arguments they wish to assess, or a group consensus view.
A computer can process these statements and determine whether or not, and to what degree, a participant has made statements consistent with the facts, as evaluated through the lens of their own values.
The pessimistic view of human nature holds that the overall value of conduct depends, in a high degree, on the consequences following that conduct. In an environment without penalties for misbehaviour and rewards for good behaviour, more people will misbehave than would if rewards or penalties applied. The world of Internet discussions provides perhaps an extreme example, because no consequences, even the informal one of a really cutting denunciation, flow from bad behaviour. It seems likely that at least part of the nonsense that clutters up nearly all the open discussion forums gets there because posting nonsense in most forums gets you at least as far as posting a solid case.
Nicholas Rodger, in The Wooden World, provides a simple and very effective analysis of the information economy that allowed the British Admiralty of the mid-eighteenth century to administer a wartime service of over eighty thousand men with thirty clerks. Dr. Rodger describes the way officers attracted able subordinates by recommending men of ability for promotion, and how their ability to help (and thus attract) good men depended on the reliability of their recommendations. Dr. Rodger describes the Royal Navy of the period as the antitheses of the Internet today; a highly successful institution, with communication limited and to the point, where the high stakes facing a naval officer at war encouraged accuracy and care. We can create a framework that will identify claims made without regard to the facts, whether participants did this maliciously or simply in error. It also seems clear that allowing users to choose to discuss issues in an environment that filters out false, malicious, or simply pointless arguments can increase the usefulness, and thus the reach, of a computer-mediated communication system.
If, then, we define a successful communication system as a system that encourages the communication of something besides unsubstantiated assertions and invective, facilitating the integration of information into the participants' comments should succeed. At least, it should succeed better than the current system, which accepts all messages as having the same value. Any system designed to encourage the use of references in discussions will allow other participants to consider their attitude to those who will not back up their opinions with facts. It thus disperses the power of a moderator, from one person to all the participants in a forum, while ensuring each participant's contribution, whether well considered or not, leaves a trail the other users can evaluate. It may help integrate the huge store of information on the Internet into the huge volume of opinion. I hope that in at least some cases, we will see the opinions informed, and the information made intelligible and humanised, by thoughtful analysis.John Spragge has built and maintained information access systems for sixteen years, starting with one of the first integrated multiple database access systems, which he built for Queen's University in the 1980s. He has also worked on search engine design and maintenance and design for SilverPlatter Information.
He has studied the process of computer-mediated information exchange extensively, participating in many conferencing systems and discussions, and writing about the effects of computer- mediated interactions.
His publications include a paper for the Guelph University Conference on computer-mediated communications in 1987, in which he analysed the operation of a computer conferencing system installed at Queen's University. He presented another paper in 1990 at Guelph on the integrated database system at Queen's, and papers on computer-mediated communications and data access for the Conference on Computing for the Social Sciences in 1992, 1996, and 1997. He also presented at Developer's Day, World Wide Web 8 in Toronto, in 1999.
Mr. Spragge currently works as president of Dancing Cat Software, a company developing advanced Web-based data integration systems.