Predicting Searcher Frustration was written out of the University of Massachesetts by Feild, Allan, in conjunction with Jones from Yahoo!. It can be found for free online here.
Life
Problem
Search engines, like all businesses, are striving to be better in order to claim more market shares, ad revenue, and viewers. As part of this effort, one of the things that they (or at least Yahoo!) are looking into is predicting when users are getting tired/frustrated in looking for data. If they detect that a specific user is frustrated, then presumably they could make the interface better, give different results, give a different category of results, or simply mark it as an area for future improvement.Experiment
What we are going to do here is make a bunch of users go on a scavenger hunt for information, and report how they feel about it. This will be measured in a couple of different ways:
- Query Logs - including page focuses, clicks, navigation, mouse movements, etc. (47 features in total)
- Sensor Data - including a mental state camera, pressure sensitive mouse, and pressure sensitive chair
- mental state camera has 6 states - agree disagree, unsure, interested, thinking, confident
- mouse has 6 pressure sensors - 2 on top, 2 on each side
- chair has six sensors - 3 on back, 3 on seat
Results
As you can see from the right, there are a few important conclusions:
- The addition of sensors has a relatively small impact on predicting frustration
- The feature set of White and Dumais (Characterizing and Predicting Search Engine Switching Behavior) is a reasonably accurate predictor of frustration
- The addition of a Markov Model Likelihood (used by Hassan in User Behavior as a Predictor of a Successful Search) doesn't yield a significant advantage.
- Markov Models, with time, have respectable results, and may be able to be boosted above W&D
- More work needs to be done by search engines before the users switch.
Why do you care?
1 - Search engines are getting better, and user modeling is likely to play a role in this in the future
2 - Direct sensor data is not required in order to predict how you are feeling (your webpage views alone are more accurate)
2 - Direct sensor data is not required in order to predict how you are feeling (your webpage views alone are more accurate)
No comments:
Post a Comment