Argumentation in Research

There are many ways of supporting a thesis. Formal proofs and empirical evidence gathered from systematic experimentation are two of the most common in computer science. A third one that I find equally attractive is by presenting sound arguments for the thesis. However, in computer science, especially software engineering, it is a technique that is seldom-used and little-respected.

Advancing a thesis by argumentation is an analytical technique. Doing it well is not easy.

One must setup criteria, that is, the values, by which to judge the thesis. Further, the criteria better have broad appeal or else you will have to argue forcefully that they ought to be broadly accepted. If you fail to make a case for the criteria, your evaluation of the thesis against them would have little appeal to others.

One must have a clear grasp of the area that the thesis concerns, which includes being aware of the minute details of other people's approaches in the area: knowing exactly what they mean by the terms they use, knowing the details of their approach, and how what they are doing relates to your thesis. If you want to show the novelty of your thesis, you must show that others fail some of the criteria. Further, no one must be able to reasonably argue that they meet your criteria. Anticipating and addressing the arguments that others will put forward to reject your thesis is where a significant portion of your energy will be spent. You have to close potential all loopholes, at least in your mind if not in the paper itself.

It is a big mistake to put a well-argued paper in the same category as vision papers, or position papers, or "nice to read, but lacks evaluation". A vision paper would chart out the world an author would like to see, perhaps listing specific challenges and promising approaches. A position paper just says what the author thinks. Sometimes the author may take a contrarian stance in a position paper; however, no deep analysis of the strength of his position and the weakness of others need be involved. And it is patently wrong to say that paper that relies solely on arguments to make its points lacks evaluation: doesn't it evaluate the thesis against the criteria (and the criteria themselves).

A vision paper: Berners-Lee et al.: Scientific publishing on the semantic web

A position paper: Chopra et al.: Research Directions in Multiagent Systems

A well-argued technical paper: Singh.: Agent Communication Languages: Rethinking the Principles

Sometimes a good argument is all it needs to show limitations in a whole class of approaches that have been adequately formalized and "proved", implemented, and tested empirically. If instances of the latter were found worthy of publication, isn't the argument that shows their limitations at least as worthy?

Argumentation is a first-class scientific technique of validating claims. Let us not judge a good argument by the criteria of others techniques. Let us not outright consider a well-argued paper as unscientific or unsubstantiated. Let us give them a fair evaluation.

There is indescribable beauty in a well-argued paper. Dijkstra said that a formula is worth a thousand pictures. An argument is worth a thousand formulas.