Choosing Content for the Summit

This is a series of posts on the PASS Summit and SQL Saturdays. I’m outlining some thoughts here, sometimes for the first time, but on topics that I think make better events. These are opinions and thoughts, not mandates or demands. I’ve written on choosing speakers and debuting content.

Update: added a comment from another set of feedback

Tl;dr: We need to review and examine comments made on abstracts as a community and a professional organization. Ultimately the program committee has a tough job, but I think we can better prepare people to both critically review abstracts and choose great content that makes the Summit an incredible event.

As the PASS Summit selections were released in June, a few of the speakers posted their submissions and the comments they received from the program committee. I think this is great, and I’d really encourage every speaker to do this. Whether you were selected or not, this is good data to better understand how the committee works and how sessions get evaluated.

This will change year to year, as committee members change. It’s good to understand the viewpoints this year, along with the results. Debate and discussion is good here, and will help shape the views of new volunteers in the future, so I’d encourage it.

In fact, if speakers want to publish on their blogs and send links, I’ll compile a list. Here’s what I’ve seen so far:

Not a big list. I don’t have anything because I didn’t submit this year.

But I’m getting distracted. What I wanted to talk about is…

How Should We Choose Content?

It’s always interested to hear various people involved in the process at events talk about this. For the most part it comes across like the strategy many business people employ. They think a little, make a guess, and hope it works well.

If anyone tells you they know what people want to see, then go look at the rooms at the event and see how well the audience matches up with the size of the room. There are plenty of mistakes that showcase themselves with too few, or too many, people in a room.

And how could they know? When the schedule is picked, no one really has much of an idea on who is actually attending. Most of the registrations will come as the event gets closer. In addition, we’re all fickle and subject to the changing requirements and demands of our jobs. We might think PowerBI is a joke (or incredibly necessary) right now, but our view could change 180 degrees by October.

There isn’t a perfect system. And I’m not saying that I think the program committee did a bad job. On the contrary, it’s a hard, thankless job, and someone will always complain about the process. My intention isn’t to do that, but rather discuss what is good feedback and what isn’t.

I hope the volunteers next year are better prepared and have some idea of how attendees and speakers view their thoughts. I also hope we publish better guidelines, both for reviewers to better choose sessions, but also for attendee comment. I’d like to know what attendees actually want to see or what they think is a good session.

Lastly, for reviewers, you need to pick sessions for the attendees. I know  many of you are attendees, but there are many, many more attendees that you need to consider. It’s easy to get trapped into picking what you like, which may not make a great conference schedule.

The reason I think this is important is that none of us necessarily has great skills as a critic, which is what reviewers do here. Many of us learned about how to examine writing in school and provide an analysis. Those skills wane, and should be both practiced and critiqued.

Comments on Comments

You can read the various posts above and look at comment, but I have a few thoughts on what items were added to the abstracts.

Note, this section is long, and I’ve mixed up comments from the various people that published items above.

Comments I Like

“Goals align w/ abstract description. Demonstration of Brent Ozar tools is a must for all SQL Administrators. Topic: Interesting for attendees and would gain an audience. Title reflects content.Interesting session for those seeking performance solutions. Objective: Level and prerequisites match goals. Material matches subject and could be presented in 75 minutes.”Excellent feedback and detailed. I like the disclosure and notes about how the abstract/goals/title relate.

“The outline seems well developed. The outline seems to clearly describe the contents of the presentation. The topic and goals should be compelling to attendees. The topic appears to be timely, new and relevant. There appears to be a reasonable amount of live demonstrations in relation to the topic being presented. The topic and goals appear to deliver an appropriate amount of material for the time allotted.” – Again, good feedback with detail that shows the reviewer thought through this and attempted to clarify what they saw.

“Whilst I get why the title might have chartaphobia in it – it still didn’t sound right. It’s not an actual word. The abstract is good, it does tell me that I may have charts but they’re “hideous and disgusting”. This didn’t sound nice and might turn some audience members away. The topic of Power BI is a good topic, the delivery just needs some work for a 100 level – explain earlier in the abstract how Power BI will make the charts better or exist at all.” – I like this feedback. It’s detailed with pointers to why the reviewer drew conclusions.

“Well written outline with clear goals and an outline that lets the attendee know exactly what will be achieved from the session. The topic is one that I think appeals to any data professional who has …” – This is good feedback. The goals match the abstract and inform the attendee. Why is the topic good? There’s more, but I left it off.

“Basic and 300 don’t coincide. Need to decide is it an advanced topic or not.” – This is a good comment that explains why this abstract needs work.

“I like the Shakespearean twist to the abstract, but I feel like it is really lacking in content to let an attendee know what they would really be coming into (outside of learning about whether to get certified or not).” – A good list of what is wrong. I agree. There isn’t quite enough to entice someone and let them know what’s coming. It’s not bad, but could be better.

Comments I Don’t Like

“…should be 200 level – the title sounds like a deep dive, not entry level.” – OK, so not entry level, but deep dive is 200? Levels are somewhat silly, but clearly we have a typo (meant 300/400) or a big mismatch in what 200 is. Maybe we need better guidelines on what 100/200/300/400/500 are?

“detailed but not compelling” – I really dislike very general comments like this. Why isn’t this compelling? Is it that you don’t like the topic? You know it already? I’ve made comments like this (for abstracts and in VCS), and this is decidedly unhelpful. As a reviewer, stop and think about what you don’t like and express that. If you can’t, you need to work on that skill.

Personally, I feel this is more suited to an SQL Saturday”What? I’ve given quite a few sessions at a SQL Saturday that I’ve delivered at PASS. Same for sessions I’ve seen. Who thinks we do better or worse sessions at one event or the other?

“Also session has no indicated real examples!” – I thought this was funny because the abstract mentions volunteering, speaking, blogging, and organizing events. What “real” examples are needed?

“Abstract is a bit rough but good enough to capture attention, describe the topic, and provides reasons why someone should attend.” – I like the detail, but what does “rough” mean? I have my impression, but I’d rather see this articulated as “difficult to read and understand the content” or “the formatting is offputting.” Note, this is why we should have some re-review and perhaps copy edit of sessions accepted. At least let the author fix things.

“Very focused topic – great for a lightning talk. Very good prerequisites and goals. The abstract is entertaining yet clear.” – Good comments, but not great. What does “good” mean? The pre-reqs should match the goals, is that what happened?

A two-fer here: These are two comments:

  • Well written abstract, but it doesn’t appear appealing with only 25% demo and no real examples
  • This seems like the right length to really kick start learning R.

OK, is length the amount of material or something else? I don’t like that word, but we have two reviewers with what seems to be opposite views. I tend to agree with the first one, there need to be examples and for this, 25% seems low. But I think we should debate when comments are vastly different and someone should perhaps think about more training or declining a repeat volunteer request.

A seven-fer:

  • Sounds like it could be a good session but the abstract seemed all over the place. Hard to follow.
  • abstract seems to ramble. grammatical anomalies. punctuation misuse.
  • Fantastic topic, and extremely well-constructed abstract!
  • Well written abstract, sold me in easily. Seems to be on level. Clear goals.
  • Nice abstract with clear goals and outlines.
  • Abstract OK. Learning goals a bit roughly defined – could be more precise in this narrow topic.
  • Good abstract. Could be an interesting session to attendees.

This means that we don’t have reviewers that are on similar pages. I am unclear how people could view this so vastly differently. The group of people that wrote these comments need to have a root cause analysis of what happened here. Or this needs to be a case study for next year’s reviewers.

Note, I’ll say that I think the abstract did get off track and a little unfocused in explaining what will be covered. A few grammar mistakes I forgive, but it also needs some tightening with fewer words and more focus on what’s covered.

“Only issue is the second sentence reads a little oddly.” – I agree with this one, but if you’re going to make this comment, expand. Why? Help someone better understand this tangled mess of a language we call English.

“Abstract targeted at Mgr/Team Lead. Unfortunately this is not the typical audience at PASS Summit.”What’s the typical audience? Do we not look to have some atypical or niche sessions. This is a place that I wish reviewers would remember that not everyone goes to every session. There will be 15 or so sessions at any given time. Niche topics are OK.

“level too low” – For a 100 jumpstart session with no pre-reqs. Again, why?

“If there is a prerequisite, the session level should be 200 (instead of 100).” – OK, if this is a guideline for speakers, fine, but reviewers don’t get to make things up. I couldn’t find anything about this on the PASS site. The rest of the comment did ask for more details, which I do like.

“Abstract is ok, with a few wording and grammar choices that could be improved/changed to make reading easier” –  I’m torn here. I think writing and grammar are important, and if the speaker makes mistakes here, they may do so in their Powerpoint and distract from the presentation. However, this process is also competitive and time sensitive, so minor issues I’d overlook and perhaps even have the speaker (or committee) correct for the event.

Old topic for BIML. Need to add some new features.” – Stop this. What you learned isn’t what others know. We have people at all levels, so don’t program for your level. At worst, there should be guidelines about how many topics at a level for each area. Without that, don’t make these comments. This person, in my opinion, is a poor reviewer and critic.

“Demo percentage seems low for such a topic but, overall, looks like a good session” – Do we have guidelines for how much demo per topic or type of session? If not, then DO NOT use this as your evaluation criteria. If we do, where is it?

“topic is interesting but not really “hot” or “latest”.” – The rest of the comment is good, but here I’m confused. What does hot or latest matter? Are we trying to be hot or latest with all sessions? If not, and I’d say not, then why the comment?

“goals not compelling” – Again, why not. What makes something compelling? Or at least describe what isn’t compelling to you, or for the audience.

“can this really be covered in 10 minutes?” – First, who are you that you question the speaker’s ability here. Especially if you don’t know if this is Joe Developer or Stacia Misner. Don’t make these comments if you haven’t seen the session. If you have, disclose that. If you think there is too much content for the time, note that, but I think this is too subjective a comment. Your (the reviewer’s) learning or teaching ability is not being evaluated here.

“Abstract: not compelling attendees. Topic: goals are very low, I don’t think this session is interesting” – I realize that reviewers might be rushed, but this is poor. Not compelling attendees? What does that mean? It’s a comment that doesn’t help, and by the way, I disagree. The second part of this also isn’t detailed. What is too low? I’d disagree. The goals list things that are specific, and certainly not what I’d call too basic. The last item shows the reviewer’s thoughts, which should be low weighted. This is about what a percentage of the attendees will want. Not all of them, some of them.

“This is more related to dba track rather than prodev. Also is survival really career development? Many would say that working 15 years as a lone dba could equate to failure in some peoples eye’s and I would struggle to want to see this session based upon info provided.” – I’m surprised this got through the review process from Lance and Mindy. I know they read a lot of comments, but this one is unprofessional and rude. The first part and the last partial sentence are fair. Those are opinions, though I think this is easily a pro-dev topic. The second sentence shows how many reviewers see the world very narrowly. I think the abstract was on balancing workload and life, which is professional development. My opinion here. However noting the someone is a failure for working the same job 15 years is ridiculous. The lone DBA is a company decision, not an individual one, but apart from the grammar errors, this one shows a person that is quite myopic on what a successful career looks like. I wouldn’t have this person back reviewing abstracts.

“Dinking the abstract rating for identifying information.” – I. Hate. This.

Honestly, I think this is a jealousy or pet peeve for a few people over, well, I’ll say it, Brent Ozar’s marketing. He does it well, better than most people and this comment/attitude is crap. Go look at all the marketing for all conferences. They mention the speaker’s names. Speakers attract attendees. If we don’t care about the speakers, why do we have this page? If you want a speaker independent review, then blank out the names, but don’t ding abstracts for this.



Final Thoughts

This is a subjective process. I am glad at least three people review the abstracts because any one person might not like the topic/speaker, have a bad day, not be interested in the topic, etc.

For the record, I’m no better and have to constantly remind myself as I choose content and edit articles that it’s not my opinion. I have to think more broadly and put myself in the position of the beginner, the intermediate person, the expert DBA that’s getting into R, the person that wants to delve deeply into a topic.

Really I want this to be discussed and debated on a few levels. What guidelines should reviewers use, and what feedback helps speakers. I would like to see a better program that helps build a better event overall. Again, we all could use skills in evaluating sessions better, so make this more public and let’s ensure those that volunteer have better guidelines on how to examine abstracts.

Lastly, I think judging based on levels is silly. We can’t agree on this, so if you think the level is too high or low, that can be adjusted. Make a comment this should be XX and let’s have speakers edit this before the schedule is released.The same thing for minor spelling or grammar, or phrasing. Let’s remember we have a lot of non-English speakers, and correct some of these items rather than just saying a native English speaker would deliver a better session.

Let’s remember, writing != speaking.

About way0utwest

Editor, SQLServerCentral
This entry was posted in Blog and tagged , , . Bookmark the permalink.

10 Responses to Choosing Content for the Summit

  1. Andy Leonard says:

    Excellent post, Steve.

    I didn’t post my feedback comments because I felt they reflect poorly on the volunteers who provided the feedback. That’s not a criticism of those who shared their feedback. It’s just my reason for choosing to not join them. Had I shared, my post would have been simply more of the same. Plus, I don’t want to discourage the volunteers who sacrificed time to both review the abstracts and provide the feedback.

    I agree with you about Session Levels. It’s a silly distraction and an engine of loss for the PASS Summit session selection process. It may be boasting to say this but I’m going to write it anyway: I got some negative feedback on my session levels. I’ve trained thousands of people in SSIS. Hundreds have paid for my training services over the past decade. I led a team of dozens of ETL developers who delivered two multi-year enterprise projects simultaneously. I believe I know as much about the maturity of a data integration developer and the proper level of my material as those reviewing my abstracts.

    Thanks for writing this. I’m a big believer in the role of feedback – to presenters and potential presenters as well as to the PASS Summit Selection folks. Let’s hope they respond to your feedback as they want us to respond to theirs.


    • way0utwest says:

      Thanks, Andy. I don’t wish to discourage volunteers either, but willingness to help != skilled or capable of helping. I want this to be used, not to bash people this year, but as a way to train and guide volunteers in the future, so that they can better evaluate sessions in a way that all of us, the entire community, find helpful.

  2. Andy Leonard says:

    I hope you didn’t interpret my comment to mean I thought you were “being mean.” I didn’t. I posted a while back that volunteering is not an acceptable excuse for doing a poor job. If you aren’t good at it, stop volunteering.

    In defense of the volunteers selecting sessions for the PASS Summit, one of the rules disallows someone from helping select sessions in a track if they’ve submitted a topic to the Summit in that track. I understand the purpose of this rule, but think about it: This basically says “we want you helping pick sessions in an area where you either don’t have expertise or have chosen to not share your expertise this year.” I don’t have a good solution, but it’s a recipe for what we’re seeing in the feedback.

    • way0utwest says:

      I didn’t take it that way, but rather that I might be doing so.

      I agree with you, that we aren’t letting volunteers help. Specifically they also don’t allow coaches to choose sessions. Seems like a missed opportunity, but I do understand some issues here.

  3. Pinal Dave says:

    Thank you Steve!

    You always voice of people. I truly feel this all the time. Thank you!

    ” Let’s remember we have a lot of non-English speakers, and correct some of these items rather than just saying a native English speaker would deliver a better session. “

  4. Pinal Dave says:

    Now if I can go back and edit my previous comment I would correct myself

    You are always “voice of the people”. (I just realized I made mistake in grammar).

  5. Thank you Steve for taking your time to write this. It includes all of my thoughts about this as well.
    And the last phrase is very important: “Let’s remember, writing != speaking.”
    Just as the committee isn’t necessarily a trained and experienced “Editorial Review Board”, like magazines have. Imho one should keep both in mind and maybe adjust the process according to that. In the end we don’t want a skill contest in writing 500-character-agendas, or do we? While it actually may increase speaking quality on a rhetorical and entertainment level, the focus should remain on technical content. I think…

  6. Pingback: Summit Session Feedback | FishHeadTed

  7. Thank you all for sharing as that is what the SQL Family is all about. I’m in and have gotten off my .. and created a blog post with my submissions and feedback. I have been debating doing this since the conversation started weeks ago. I have had some reservations similar to what Andy shared and in the end feel that we can all benefit from the shared knowledge.

  8. Pingback: The Missing Piece of SQL Conferences | SirSQL

Comments are closed.