Special Issue: When Learning and Mining Methods Yield Unexpected Results 
Guest Editors: Margaret Dunham and Christophe Giraud-Carrier

SIGKDD Explorations (Volume 12, Issue 2) will be available December 2010.

IMPORTANT DATES

Submission Deadline: September 15, 2010
Acceptance notice: October 15, 2010
Camera-ready due: November 1, 2010
Publication: December 2010 issue

DESCRIPTION

ACM SIGKDD Explorations seeks original manuscripts for a Special Issue on Unexpected Results in Data mining projects, scheduled to appear in December 2010.

We believe that the empirical study of machine learning and data mining methods often falls prey to the effects of publication bias that favors positive results over negative ones. Most, if not all, articles in conferences and journals report only positive results. This does not reflect the practice of a field where failures happen regularly. As in real life, we often learn more from negative results than we do from positive results. It is time we, as a community, start to regard failures as being as informative as successes. After all, we do know the difficulty of learning from positive only examples; so how can we expect to learn about our field if all we ever see are successes?

We think that a special issue concentrating on interesting and enlightening failures will be a good starting point. The special issue intends to provide a forum for papers that would otherwise likely be rejected because of the lack of a positive result. We will accept papers that describe clear, and somehow surprising, failures that stand in need of an explanation. Clear, or interesting, failures happen in situations where the learning or mining method is not only sub-optimal, but performs far worse than expected. To be of value, however, the description of such failures must be accompanied by some discussion of lessons learned and at least a partial explanation of their causes.

The importance of negative results has been recognized in other disciplines. In Bioinformatics there are three journals dedicated to negative results: Journal of Negative Results in Biomedicine (http://www.jnrbm.com/articles/browse.asp), The All Results Journals (http://www.arjournals.com/ojs/index.php?journal=Biol&page=about&op=editorialPolicies#focusAndScope), and Journal of Negative Results (http://www.jnr-eeb.org/index.php/jnr).

CALL FOR PAPERS

The main purpose of the special issue therefore is to collect short papers (2-5 pages) describing exemplary failures, with the goals of:

  1. making these experiences accessible to fellow researchers who may otherwise waste their time on the same or similar idea, and

  2. providing the first few negative data points necessary to gain additional insights into our methods (e.g., what method is applicable where).

The special issue should be of value to the broadest possible audience. Consequently, we are mainly interested in papers that report failures of learning and mining strategies that are already popular and well-known in the community, or of novel ideas that are easy to comprehend and do not require extensive prior knowledge in a special niche area of machine learning or data mining. Furthermore, both the scenario leading to and the actual failure should be explained in a way that would be understandable to the average machine learning and data mining practitioner. The ideal paper will describe an experiment or an idea that is likely to be repeated by other people, and whose expected outcome is clear to everybody. The result should fail this expectation, and possibly lead to an (at least sketched out) explanation, that allows to turn the negative experience into a piece of useful machine learning and data mining knowledge. While this is ideal, we will also consider papers that describe common failures that could have been foreseen, but where the respective researchers or practitioners simply did not know the relevant facts or did not understand their relevance. The main selection criterion will be whether it appears worthwhile to record the failure for the community. That is, whether the lessons learned will be beneficial to a broader audience.

The special issue will include both invited and contributed papers.

SUBMISSION GUIDELINES

Submissions should be emailed to mhd at lyle.smu.edu and cgc at cs.byu.edu, preferably in PDF format. In addition, please email the authors, title, abstract and contact information for the corresponding author in ascii text to the same address.

Submissions will be reviewed externally.

Detailed formatting instructions and templates are available from:http://www.acm.org/sigs/sigkdd/explorations/submissions.php 
Submissions will be reviewed externally.

Please address the correspondence regarding this special issue to the Guest Editors: Margaret Dunham and Christophe Giraud-Carrier