View this PageEdit this PageUploads to this PageHistory of this PageHomeRecent ChangesSearchHelp Guide

DW

Title: Using the Experience Sampling Method to Evaluate Ubicomp Applications

Submission type: Research paper
/Research paper
Retrospective
Survey/Tutorial
Project overview
Other/


[All submission types]
Relevance to magazine: 4
/5 = Burning issue of the day; 4 = Timely and relevant;
3 = Worth reporting;
2 = Yawn; 1 = Much ado about nothing/


[All submission types]
Presentation and clarity: 3
/5 = Lucid and eloquent; 4 = Easily understandable;
3 = Could be improved;
2 = Hard to understand; 1 = Dense and impenetrable/


[All submission types]
Reviewer expertise: 4
/5 = I'm an authority on this topic; 4 = My work overlaps this topic;
3 = I'm know this stuff well, but don't work on it;
2 = I'm somewhat familiar with this stuff; 1 = This is new to me/


[Research papers only]
Depth of contribution: 3
/5 = Deep and Impressive; 4 = Good, but could be improved;
3 = Useful contribution, but not earthshaking;
2 = Ho Hum; 1 = Seriously flawed/


[All submission types]
Overall recommendation: 2
/5 = Award quality; 4 = Accept with minimal revision;
3 = Accept with major revision but no re-reviewing;
2 = Accept with major revision and re-reviewing; 1 = Reject/


Detailed comments for the authors:

/This paper is a useful description of an evaluation for ubiquitous-computing applications, but it would be useful if the authors earlier in the paper the specific technology they are evaluating. It is not until the bottom of the fourth page – halfway through the paper – that the authors reveal that they are attempting to define usage scenarios for the Intel Personal Server. The term “ubicomp applications,” used in the title, encompasses a broad class of technologies. Knowing that this is an evaluation of the Personal Server would help the reader form a situate the evaluation technique in terms of the technology. This is especially important with this particular technique and device because of the apparent likelihood that the reader may not be familiar with either of them.

The description of the Experience Sampling Method is informative, but the reader would benefit from more detailed information about how the method is used in the papers cited. Also, the paragraph headed “Delivering” lists four questions that the researcher must address in deciding on a mode for delivering questionnaires, but the issues that must be considered in choosing a delivery mode are not given. This is obviously a very important consideration and merits more discussion.

There are other examples of information that need further elaboration, particularly in the description of the case study. There are only two recruiting criteria given: It is understandable that the researchers would want only subjects who are at least 18 years old, but no reason is given for the stipulation that they must use a cellphone regularly. Considering that the questionnaires were delivered on PDAs, wouldn’t regular PDA usage be a more valid criterion? We are also told that the subjects were given Intel Pocket PC cameras, but we aren’t told whether the subjects were asked to take pictures simply to situate their responses visually or for some other reason. I wondered if the usage scenarios being developed included visual input. Finally, the results should be described more thoroughly./