Social Media Monitoring

• About this group
• Forums

• Home
• Groups
• Social Media Monitoring
• Forums
• FREE Resources - Checklists, Presentations, Newsletters, SM Policies
• Social Media Monitoring Industry Survey by Jason Falls

This thread has been closed so unfortunately you can't post here any more.

• Post visible to registered members
• 
  Prof. Dr. Urs E. Gattiker    Premium Member   Group moderator

  The company name is only visible to registered members.

  Re: Jason Falls - why not all things should be measured

  jan
  
  First, thanks so much for posting this. Second, after I started filling out the survey I began reflecting a bit.
  
  For starters, this is a great survey because it illustrates wonderfully what Einstein said a long time ago:
  
  "Everything that can be counted does not necessarily count. Everything that counts cannot necessarily be counted." #Einstein #SM
  
  SECTION OF SURVEY ENTITLED: How often do you ...</b>
  If you study the questions asked here, they tell me how often you might use things but once I have these data collected... what will it mean exactly?
  
  For instance, 50% use free tools: But why do they use free tools, is it because they are better, cost nothing or both reasons matter why people choose free tools?
  
  ==> Data collection is the easy part, unfortunately, the devil is in the details or answering what the findings mean and why these data are they way they are.
  
  SECTION OF SURVEY ENTITLED: For paid solutions, how comfortable are you with ...
  This section forces respondents to choose between a positive or a negative answer .... valid measuring requires that there is a mid-point (e.g., undecided, neutral, etc.). The rating:
  ===> Comfortable enough to figure it out VERSUS ===> May need help
  is a jump from positive to negative .... whereby we end with 3 anchors that are negative and 2 positive.
  
  ==> Measuring properly is a tricky business but respondents need to be given a neutral option to answer if one uses a Likert-type scale.
  
  SECTION OF SURVEY ENTITLED: For paid solutions, how satisfied are you with ...
  To reduce response bias, we are asked to avoid double-barreled questions like the plague.
  A double-barreled question combines two or more issues or attitudinal objects in a single question.
  This section has several bouble-barreled questions like this one:
  The ability to filter/manage the data
  Am I responding to filtering or managing of data? In turn, how can responses be interpreted if I, as the researcher, am not sure what the person responded to exactly?
  
  ===> Designing questions wrongly results in invalid answers making data useless
  
  SECTION OF SURVEY ENTITLED - What is most important to you in a paid monitoring solution?
  This section does again not offer the reader a mid-point on the scale (e.g., undecided)
  There are a few more issues.... but for brevity's sake I will not address them here:
  
  CONCLUSION
  This is an interesting survey but for various methodological reasons its data will suffer from
  
  - response bias,
  - non-response bias (e.g., are people who did not fill in the survey or like me who did not complete the survey different from those that filled it out completely and submitted)
  - possible halo effects as well as concerns regarding
  - validity (e.g., US respondents vs. others, etc.)
  
  Why I welcome the survey, for the reasons given above I have chosen not to fill it out. Nevertheless, I am certain that regardless of these weaknesses affecting the quality of the findings and data interpretations, the findings will get a lot of coverage on various blogs as well as Twitter.
  
  However, if we want our metrics and findings to be taken seriously by the executive suite, we have to design our surveys to account if not avoid some of the challenges outlined above.
  Failing to do so will undermine the usefulness of data collected and while these metrics may result in actions, the decisions taken may be based on the wrong information.
  
  Respectfully
  
  Urs
  http://My.ComMetrics.com - benchmark your blog, improve performance
  
  .
  
  • 01 Mar 2010, 5:08 pm
  Prof. Dr. Urs E. Gattiker wrote: > jan > > First, thanks so much for posting this. Second, after I started > filling out the survey I began reflecting a bit. > > For starters, this is a great survey because it illustrates > wonderfully what Einstein said a long time ago: > > "Everything that can be counted does not necessarily count. > Everything that counts cannot necessarily be counted." #Einstein #SM > > SECTION OF SURVEY ENTITLED: How often do you ...</b> > If you study the questions asked here, they tell me how often you > might use things but once I have these data collected... what will it > mean exactly? > > For instance, 50% use free tools: But why do they use free tools, is > it because they are better, cost nothing or both reasons matter why > people choose free tools? > > ==> Data collection is the easy part, unfortunately, the devil is in > the details or answering what the findings mean and why these data > are they way they are. > > SECTION OF SURVEY ENTITLED: For paid solutions, how comfortable are > you with ... > This section forces respondents to choose between a positive or a > negative answer .... valid measuring requires that there is a > mid-point (e.g., undecided, neutral, etc.). The rating: > ===> Comfortable enough to figure it out VERSUS ===> May need help > is a jump from positive to negative .... whereby we end with 3 > anchors that are negative and 2 positive. > > ==> Measuring properly is a tricky business but respondents need to > be given a neutral option to answer if one uses a Likert-type scale. > > SECTION OF SURVEY ENTITLED: For paid solutions, how satisfied are you > with ... > To reduce response bias, we are asked to avoid double-barreled > questions like the plague. > A double-barreled question combines two or more issues or attitudinal > objects in a single question. > This section has several bouble-barreled questions like this one: > The ability to filter/manage the data > Am I responding to filtering or managing of data? In turn, how can > responses be interpreted if I, as the researcher, am not sure what > the person responded to exactly? > > ===> Designing questions wrongly results in invalid answers making > data useless > > SECTION OF SURVEY ENTITLED - What is most important to you in a paid > monitoring solution? > This section does again not offer the reader a mid-point on the scale > (e.g., undecided) > There are a few more issues.... but for brevity's sake I will not > address them here: > > CONCLUSION > This is an interesting survey but for various methodological reasons > its data will suffer from > > - response bias, > - non-response bias (e.g., are people who did not fill in the survey > or like me who did not complete the survey different from those that > filled it out completely and submitted) > - possible halo effects as well as concerns regarding > - validity (e.g., US respondents vs. others, etc.) > > Why I welcome the survey, for the reasons given above I have chosen > not to fill it out. Nevertheless, I am certain that regardless of > these weaknesses affecting the quality of the findings and data > interpretations, the findings will get a lot of coverage on various > blogs as well as Twitter. > > However, if we want our metrics and findings to be taken seriously by > the executive suite, we have to design our surveys to account if not > avoid some of the challenges outlined above. > Failing to do so will undermine the usefulness of data collected and > while these metrics may result in actions, the decisions taken may be > based on the wrong information. > > Respectfully > > Urs > http://My.ComMetrics.com - benchmark your blog, improve performance > > .
• Post visible to registered members