Ratings, Questionnaire design
We prepared 5 different questions to assess user’s cognitive input. These questions cover the 5 different dimensions mentioned in the previous section in order to assess user’s experience with the dashboards. This is a sample case study done in an imaginary situation with the Tableau sample dashboard “Superstores”. The table below is an extract of ratings given by different users for a dashboard. The table was prepared by joining the “_users” table which stores user ID and the schema created to store ratings “insights from data”.
The questions we included in the form with respect to each section are the following:
The above histogram shows user response, to questions related to one of the dimensions. The median value indicates the most common response, which is neutral (rating 3). Also, the most frequent response – is neutral(rating :3), which indicates the dashboard has adequate performance on engagement dimension. How ever there is also room for development . The above diagram was prepared with responses from a particular time period and does not indicate the change in satisfaction level over time.
Similarly, we did plot the responses (whisker plot) from a 6-month period, also belong the same dashboard and same dimension. It showed considerable improvement in the dashboard performance in a specific dimension. In addition to that the median, value changed to 4. So, it looks like the dashboard designer took some time to analyze the responses and make changes to the layout and the content of the dashboard. A rating scale aimed at different characteristics like the color, font etc. can bring detailed insights on the aesthetics part.
We did a similar analysis on the remaining dimensions. The following is the questionnaire that we used to collect information in a similar fashion.
2. Do you feel the dashboard was engaging enough to get more insights?
Include an ordinal scale data with 1 = Not engaging 2 = Somewhat not Engaging 3 = neutral 4 = somewhat engaging 5 = Engaging
3. How useful do you think this dashboard is?
Nominal like and dislike like = it was confusing
dislike = it was useful
4. How clear are the visualisations?
Star rating – 0 to 5. 0 = it is not clear at all, 5 = it is very clear and can interpret data very easily.
5. How efficient the dashboard is in terms of performance?
Ratio scale with 0 to 100.
Before jumping to the next sections, it is important to note that the inferences and the methodology followed are primarily indented to emphasize a simple use case of the new WriteBackExtreme features, so the inferences may be different from what is real. A real study requires a well described methodology and statistically significant inferences.