Changing Scales and Creating Disappointment

I got an email recently that notified me that session feedback from the Summit was available for my talk. I’d had a lot of people in the room, and was curious how things went. I think the session was OK, a little off on time, a few too many questions I tried to answer, and perhaps a bumpy flow.

However when I got my scores, I had a 2.85 for the session overall, with various aspects of the talk being rated from 2.5 to 2.9.

Well, I sucked.

That was my first thought. I’ve been getting rated, and evaluating speakers on a 5 point scale for quite a few years at PASS events. I was surprised, and disappointed, and then a bit embarrassed that I hadn’t delivered a good talk at the Summit. Since I hadn’t delivered that talk in public anywhere prior to the conference, I thought I had made a big mistake. Apparently my practice that week in my hotel room had been for naught.

However then I saw this note in another email: One of the changes this year was to move from a five point rating scale to a three-point scale.

Hmmm, I missed that in my email somewhere, and didn’t notice this as I filled out a couple of session evaluations.

I don’t think there’s anything wrong with changing the scale. Personally I like the 3 point scale, but it wasn’t a change I noticed. The first communication with speaker feedback didn’t mention this.

Scale matters. Many of us know that by manipulating scales, we can make data look different, We can prove a point that might not be supported by a different presentation of the data on another scale.

Our clients and business users come to know and expect the various ways we present data. They will start to internalize scales and interpret data based on their expectations. We can change scales, but we need to make it clear and visible that we have changed scales.

Personally I would have appreciated the results being reported as:

Overall Session Score: 2.85/3.00

instead of

Overall Session Score: 2.85

That little extra information can mean a lot. Keep this in mind as you make fundamental changes to the way you present data.

About way0utwest

Editor, SQLServerCentral
This entry was posted in Blog and tagged , , . Bookmark the permalink.

One Response to Changing Scales and Creating Disappointment

  1. papynormand says:

    I am supporting you about this “stupid” way to evaluate a session .
    The first reason ( and own ) is that I am living in a country which is the “father” of the metric system.
    The second one is that I am considering that it is impolite not to warn the speakers about this kind of change ( maybe it was written in too small size in the notes at the end of the a document , the part so difficult to read that most of people are skipping them ) as soon as the speaker proposes his/her session.
    I have other reasons but I have difficulties to “translate” my thoughts from my mother tongue towards the English language.

Comments are closed.