A blog by Rob J Hyndman 

Twitter Gplus RSS

Internet surveys

Published on 19 January 2012

I received the fol­low­ing email today:

I am prepar­ing a the­sis … I need to con­duct the widest pos­si­ble poll, and it occurred to me that per­haps you could guide me toward an internet-​​based way in which this can be done eas­ily. I have a ten-​​question ques­tion­naire pre­pared, that I wish to have an ran­dom sam­ple of the pop­u­la­tion respond to. I have no bud­get for this, so I hope you can sug­gest a way in which a good num­ber of responses can be har­vested using blogs or sites you may be aware of.

Here is my response.

There are two issues here. The first is to find a con­ve­nient web-​​based data-​​collection tool. One pop­u­lar approach is to use a sur­vey form on Google Docs. The results are auto­mat­i­cally saved to a Google spread­sheet. There are many online expla­na­tions of how to set up your sur­vey form includ­ing this from Google help or this from dig­i­tal inspi­ra­tions. A more sophis­ti­cated tool for more com­plex sur­veys is Sur­vey­Mon­key. This allows skip­ping ques­tions based on pre­vi­ous responses, response val­i­da­tion, and other use­ful fea­tures. For researchers col­lect­ing data, I gen­er­ally rec­om­mend that they use Sur­vey­Mon­key. But for a quick poll of a small group, Google Docs is ade­quate. Using either tool, the responses can be down­loaded and imported into R or some other sta­tis­ti­cal analy­sis pack­age. Web-​​based data col­lec­tion avoids all the prob­lems asso­ci­ated with enter­ing and encod­ing data, although one draw­back is the tech bar­rier for some audi­ences. You won’t be able to use web-​​based data col­lec­tion for a sur­vey of the elderly, or of remote Ama­zon­ian tribes, or of many other pop­u­la­tions where not every­one uses the inter­net. But if it is rea­son­able to assume that all mem­bers of the pop­u­la­tion use the inter­net, then web-​​based col­lec­tion is much bet­ter than paper-​​based forms.

The sec­ond issue is more dif­fi­cult. That is, how to get a ran­dom sam­ple of the pop­u­la­tion. Here, there are no magic tech solu­tions. Adver­tis­ing on blogs or other sites will sim­ply give you a biased sam­ple favour­ing those who read the blogs and have the time and inter­est to respond. Then you have to make the coura­geous assump­tion that the respon­ders are rep­re­sen­ta­tive of the pop­u­la­tion of inter­est. It is bet­ter to iden­tify the pop­u­la­tion of inter­est first, and find some way of ran­domly sam­pling it where each mem­ber of the pop­u­la­tion has equal prob­a­bil­ity of being selected in the sam­ple. How this can be done depends on the par­tic­u­lar pop­u­la­tion being stud­ied. I sug­gest you dis­cuss a sam­pling strat­egy with the sta­tis­ti­cians at your uni­ver­sity. There are also some good online ref­er­ences includ­ing “Best prac­tices” from the AAPOR, and “What is a sur­vey?” by Fritz Scheuren. A use­ful text­book is Sam­pling: Design and Analy­sis by Sharon Lohr (Duxbury Press, 2009, 2nd ed.).

Related Posts:

2 Comments  comments 
  • Anony­mous

    It’s not my inten­tion to serve as an adver­tise­ment, but in terms of design­ing an online sur­vey for my PhD,  I have had tremen­dous suc­cess with Spread­sheet Con­verter… http://​www​.spread​sheet​con​verter​.com/

    It’s an excel plug-​​in that allows you to design your sur­vey in Excel and then con­vert it to html.  I found it gave me far more flex­i­bil­ity in design­ing my sur­vey than any of the sur­vey web­sites, and it allows you to make some pretty com­plex on-​​the-​​fly cal­cu­la­tions as well as present live graphs that change with respon­dent choices.  The down­sides are that it’s not free (~$100), and that you have to host the sur­vey some­where your­self.  Responses can be sent via email or, more use­fully, writ­ten to an SQL data­base, although obvi­ously that requires a lit­tle bit of programming.

  • Pingback: Internet surveys | Research tips « Tips On Paid Surveys