Introduction to Qualtrics

Introduction to Qualtrics

Introduction to Qualtrics Sean Freeder 04/05/17 D-Lab, University of California, Berkeley What Well Cover Today Basics of Qualtrics user interface Creating simple survey questions Managing survey flow and randomization More advanced techniques

Miscellaneous Qualtrics Tips Stupid human tricks Prepping data for external analysis Basics of Qualtrics User Interface: Look & Feel: Options for what the survey will generally look like to your respondents. Survey Flow: Extremely useful for managing survey order/randomization/content. More on this later. Survey Options: Mostly can be ignored. Some useful options for very specific problems. Tools: A few very useful options (search-and-replace, test survey, quotas) that well cover later. Preview Survey: Allows you to see what respondents will eventually see. Should be used regularly.

DISTRIBUTIONS: Used to create your survey link DATA & ANALYSIS: Used to export survey data and to check individual results REPORTS: Used to quickly check aggregate results of survey without having to download dataset Basics of Qualtrics User Interface: Levels of Control in Qualtrics QUESTION: At minimum, a string of text. More often, text followed by one or more response options. BLOCK: A collection of any number of questions. Used for grouping related questions together for organization, randomization, or looping. SURVEY FLOW: A collection of any number of blocks, instructions or

metadata. Used to determine what will be presented to your respondents, and in what order it will show up. Basics of Qualtrics User Interface: Types of Building Blocks (Elements) in Qualtrics BLOCK: A set of survey questions BRANCH: If/then instructions for displaying other elements EMBEDDED DATA: Metadata that can be set by the user RANDOMIZER: Used for randomizing order/display of elements WEB SERVICE: Accesses APIs used rarely

AUTHENTICATOR: Allows select respondents survey access used rarely END OF SURVEY: Concludes survey upon reaching this element CONJOINT: Displays conjoint experiment (not covered today) Basics of Qualtrics User Interface: Particularly Useful Block Options QUESTION RANDOMIZATION: Allows randomization at the block level LOOP & MERGE: Allows slightly different versions of the same block to be completely repeatedly

COPY BLOCK: Useful for quick duplication COPY BLOCK TO: Useful for saving frequently used content across surveys Creating Simple Survey Questions Types of Questions Static Content: Useful for instructions and image display Standard Questions: Almost always use these questions Specialty Questions: Usually not useful,

Constant Sum and Drill Down good Advanced: Collecting metadata, timing is especially useful Creating Simple Survey Questions Basic Question Tips Multiple Choice: Use single/multiple, vertical/horizontal options typically Matrix Table: For rating multiple things simultaneously. Use likert. Bipolar sometimes even useful for single answers.

Text Entry: Multiline, increase box size Slider/Rank Order: Sometimes useful, but more confusing to respondents. Rank Order good for forcing rankings Creating Simple Survey Questions Question Options (located on the right) Validation Options: Used to make sure respondents actually answer your questions FORCE: Question must be answered. Use on questions that survey flow relies on for logic.

REQUEST: Reminds subjects to answer questions. Use on all other questions. Add Page Break: Used to separate questions from one another on the actual survey Use liberally! Randomization can mess up page breaks Creating Simple Survey Questions Question Options (located in the gear) Add Display/Skip Logic: Extremely useful for managing survey flow

Carry Forward: Takes previously used options and displays them again Add Javascript: Advanced, but very useful Randomization: Change the order of options Recode Values: Change output value/variable labels Managing Survey Flow and Randomization If/Then Logic Display Logic: Controls whether current question is displayed based on previous question answers. Skip Logic: Controls whether next question(s) are

displayed based on current question answers. Branch (in Survey Flow): Controls whether a block is displayed based on previous question answers. Managing Survey Flow and Randomization Randomization Options Randomization: Randomizes the order/appearance of response/ item options (under gearbox) Answer Randomization: Randomizes the order/appearance of scale options (under gearbox, sometimes) Question Randomization: Randomizes the order/appearance of

questions (under block options) Randomizer: Randomizes the order/appearance of survey elements (under survey flow) Managing Survey Flow and Randomization Randomization Techniques Question/Answer Randomization Randomize Order, All: Presents all options in a random order Present Only: Chooses N options from Y in a random order Advanced: Allows you to apply the above randomizations to only a select subset of the options

WARNING: Advanced randomization at the block level will disturb existing page breaks. If you use this AND need page breaks, you can set the number of questions allowed (usually 1) at the bottom of the Advanced Randomization panel. Managing Survey Flow and Randomization Randomization Techniques, Continued Block Randomizer: Will present N of the Y blocks listed below in the signal flow Evenly Present Elements: Determines randomization technique for whether a given block shows up

Selected: Each new respondent will be shown the block after the one displayed to the previous respondent Not Selected: Any given block has probability P of showing Selecting will guarantee equal numbers of people see each condition, not selecting will create expectation of equality Managing Survey Flow and Randomization Elements of Signal Flow: Embedded Data Embedded Data will attach a variable with a given value to any respondent who encounters it in the survey flow Embedding Set Values: Name the variable anything, then set

the value to a desired number or text Embedding URL Values: Name the variable after the URL field you want to collect Uses of embedded data: Creating treatment/control groups, capturing IDs for repeat use of respondents in panel studies, setting up complex if/then logic chains Managing Survey Flow and Randomization Elements of Signal Flow: End of Survey When a respondent encounters an End of Survey element, they will be shown a customizable message, and the survey will end.

Surveys can have multiple End of Survey elements in them, triggered by different conditions, but any given respondent can only hit a single one. Uses of End of Survey: Providing a redemption code for workers, preventing access to the survey for an unqualified respondent, redirecting respondents to a different survey More Advanced Techniques Quotas Quotas are found under Tools, and used to ensure you only get a certain number of respondents of a certain type

To use a quota, simply select a question that has response options you want to select on, then set the number of people allowed to select that option. When the actual number exceeds the quota, clicking that option will automatically end the survey. Generally, you want to put quota questions at the beginning of the survey. Otherwise, youll have very angry respondents! More Advanced Techniques Using Timers Timers allow you to measure and control how much time is being spent on a given page

Access: Question Options > Advanced > Timing Placing it before a question will tell you how long is spent on a page. Timers record timing for pages, not questions or blocks. Under the options on the right, you can force people to stay on a given page either up to or no more than a specified length of time. More Advanced Techniques Testing Your Survey You can preview the survey an infinite number of times using the Preview Survey button at the top. You can also preview an individual question using Preview Question on the right

question options panel, and preview a block by select View Block from Block Options. To make sure randomization is working, you can use Test Survey to pipe through fake respondents (up to 1000). NOTE: fake respondents will make all answer selections randomly. Both previews and tests are automatically recorded, and will need to be deleted later before you download the data. More Advanced Techniques Piped Text You can use the interface to pipe in text of all kinds into the text of

a question, response options, or scale labels. Access piped text by clicking either on the question text box and selecting Piped Text, or by selecting a response/scale option and selecting Piped Text from the dragdown menu. All piped text appears in the editor as ${x:///}, where x is equal to q: text from previous questions and responses e: text from embedded data lm: text from a loop-and-merge More Advanced Techniques

Loop and Merge If you want people to answer a large number of very similar questions in a row, Loop And Merge provides an easy way Access this function by selecting Loop and Merge under Block Options Loop and Merge will repeat the entire block while displaying new Field Value(s) for each loop. Type in values as desired, then on the question text, enter ${lm://Field/x}, where x corresponds to the field column from the Loop and Merge page. More Advanced Techniques

Using Javascript Qualtrics is Javascript-compliant Most things you want to do in Qualtrics can be done without Javascript, but JS can be used to set up very complicated randomization schemes To implement, click add Javascript from the Gear menu on the first question in your survey. Note that JS will not function until the page after the one it gets implemented on. Miscellaneous Qualtrics Tips When setting Display Logic, be very careful of AND versus OR.

Data output for multiple answer questions is annoying. Use the Survey Flow feature to move blocks around. Try not to copy lots of versions of the same question until you KNOW that is going to be the final version of that question. Empty the trash regularly to prevent slowdown Label all questions in advance HIDE your affiliation with Berkeley! (Look & Feel; shorten survey link w/ bitly.com) Stupid Human Tricks Use page breaks as often as makes sense Think carefully about the order of your questions demographics often are best last

Always put instructions on a separate page from activities Use bold, italics, and COLOR thoughtfully for emphasis. Do as much as you can to hide the purpose of your survey! Strongly consider using an attention check (or threat of one) Include a comments section at the very end of the survey Keep tasks and texts that respondents have to engage in as brief as possible Prepping Data for External Analysis Make sure that you have cleared all Preview and Test data before downloading Select from the Tools tab under the Data & Analysis header

To download your data: Export and Import > Export Data Choose Download Data Table If you later find that you cannot tell which blocks certain respondents were shown, go back and download the Legacy Format, which keeps this info. Under Download all fields, choose Use numeric values This is a personal preference, but it makes working with your data in STATA or R much easier in the long run Good luck and

thank you!

Recently Viewed Presentations

  • HRC 2015 Marketing Research Study

    HRC 2015 Marketing Research Study

    Calls attention to "deficits" often environmental: What does success mean? Can students defy the trend and expectations? BOTTOM LINE. Success means different things to different people; K-12 see it as test scores, Afterschool tend to see it more in the...
  • Remains by Simon Armitage - WordPress.com

    Remains by Simon Armitage - WordPress.com

    Remains by Simon Armitage . Bronze. Can you find examples of casual conversational language? What's the effect of writing about this event in this style? Select the three most powerful images of violence. Identify and language devices and explain their...
  • The Numeracy Professional Development Project in Secondary ...

    The Numeracy Professional Development Project in Secondary ...

    A suggested progression Start with rows of counters in columns Use a straw to generate different number properties Repeat for different numbers Generalise number properties with words Extend from counters to grids or arrays Generalise properties using symbols Investigate Two...
  • Secondary One Option Choosing your Secondary Schools for

    Secondary One Option Choosing your Secondary Schools for

    Secondary One Posting Exercise(P9)Timeline for S1 Posting Release of PSLE Result. S1 Option Start. 24-30 Nov 2017. SI-IS (7 days) School (5 days) End of S1 Option. 30 Nov . 2017
  • 投影片 1 - Tcss

    投影片 1 - Tcss

    動植物英文名稱的由來 Word Origins of Animals & Plants 伍德基 T.K.NG
  • Multinational Finance - Michigan State University

    Multinational Finance - Michigan State University

    Arbitrage Profit = no net investment and no risk. This example is designed to illustrate the mechanism of arbitrage and its effect on market prices. With daily volume of more than $5 trillion in the most recent (April 2016) survey...
  • Access the Defense Equal Opportunity Management Institute at

    Access the Defense Equal Opportunity Management Institute at

    Requesting a DEOCS. Go to . www.DEOMI.org. Click on "Organizational Assessment" Click on "DEOMI Organizational Climate Survey (DEOCS)" S:\Joint Groups\EO\Army Program\Training\References
  • Term 1  Sept  Dec Year 9 Introduction to

    Term 1 Sept Dec Year 9 Introduction to

    Physical Theatre (Movement, Physicality & Theatre Style) Year 10. Component 1 (40%) Devising (Introduction to Brecht) Component 1 - Mock (40%) Devising (Portfolios and Evaluation) Year 11. Component 1 (40%) Devising - Creating Performance (end of performance) Component 1 (40%)...