Scaling Synthesis

Search IconIcon to open search

Interactive Intent Modeling for Exploratory Search

Last updated March 17, 2023

author: T. Ruotsalo et al. Note Authored by:: Brendan Langen

# Core Questions

How do we enable users to find what they’re looking for when they are exploring a new area?

How do we design search systems that evolve with user knowledge of a topic?

Current search engines are effective for specific informational searches.

The current generation of information retrieval systems, such as the major Web search engines, is effective at identifying a small set of the most relevant documents given a well specified information need. However, it is easy to identify many situations where more complex exploratory search support is required and increasing real-world evidence suggests that users are struggling with exploratory search [81].

Brendan Notes How can we retroactively provide users with new information related to their past searches?

New findings can be pushed to a user based on their search history.

In what situations does having more information decrease user confidence?

# Methodology

Their process differs from past attempts because:

  1. They actually allow the user to visualize alongside their search, instead of looking at queries or documentation
  2. They implement the experiment in a practical manner alongside a real-life data collection.
  3. They empircally validate the performance in situ, instead of against log files or artificial sessions.
# Experiment Design + Results

Experiment 1 - Exploratory Search

The task was defined as a scientific writing scenario, i.e., the participants were asked to prepare materials and an outline for writing an essay on a given topic. The assignments were: > (1) Search for relevant articles to be used as references in the essay. > (2) Search for relevant keywords representing topics to be used to structure the essay

An interesting insight is that for the IntentRadar system, precision is slightly increasing toward the end of the session for novel documents. This suggests that richer interaction becomes crucial to discover novel information, in particular for these exploratory tasks that were studied in the experiment. This suggests that even though the participants were shown more keywords in the IntentRadar system, they were able to select relevant keywords from the display. Interestingly, the participants who used the IntentRadar system were significantly less convinced that they had found the right articles during the task. Given that the retrieval effectiveness was found to be significantly better for the IntentRadar system, and therefore the responses from the system were of better quality, a possible explanation is that because of the visualization the participants became more aware of other potentially relevant directions that they could not explore in the given time, and therefore might have been more informed about potentially relevant, but not yet explored, topics.

Experiment 2 - Information Comprehension

the focus of the study was threefold. First, to study if the visualization would assist users in the comprehension process. Second, to study if the users preferred interaction with the visualization. Third, if the visualization would improve the output of the comprehension process. RQ5 Comprehension process: Do participants in the visualization condition inspect the search result space using the visualization more often than using the result list? RQ6 Interaction support: Do participants in the visualization condition select keywords from the visualization more often than from the result list? RQ7 Comprehension outcome: Does the visualization result in improved information comprehension outcome? RQ8 User Experience: Does the result presentation using the visualization result in improved user experience?