|
|
| The Importance of Validity - How to avoid garbage in-garbage out |
|
|
|
|
Validity is one of the most important concepts in survey research. Without validity, you have meaningless results and have wasted a great deal of time, energy, and money. Much of the confusion surrounding validity is probably due to the cavalier manner in which the term frequently is used. This Research Note presents general issues surrounding validity, as well as different types of validity.
Validity Defined
For many consumers their only concern with validity is that their survey has it. After all, isn't validity just another of 'those statistical things'? According to the American Psychological Association, validity "...refers to the appropriateness, meaningfulness, and usefulness of the specific inferences made from test scores." (Standards for Psychological and Educational Testing, 1985, p. 9). In other words, if your findings need to be appropriate, meaningful and useful, they need to be valid.
For the purposes of survey research, this definition raises two main points. First, a survey item or scale is not "valid" in and of itself. Rather, you need to consider the purpose and context of the survey items to determine whether or not the inferences you have made are appropriate.
Second, using a particular survey item may be valid for one interpretive purpose but not for another. Don't assume that just because an item was valid in one case, that it will be valid any time you want to use it. Take, as example, a survey designed to measure employee perceptions of the amount of communication in the organization. If you validated the items by demonstrating that resulting data is related to objective measures, then the survey would be "valid" for measuring the amount of communication. However, these same items may not be appropriate for measuring other communication issues such as quality of communication or satisfaction with communication.
It is important to note that it is common to refer to a 'valid' item on a survey. This indicates that inferences based on the item will be valid in that particular interpretive context.
Types of Validity
Everyone agrees that validity is important, but how do you assess validity? Researchers talk about three main types of validity: content, criterion-related, and construct validity. All the different approaches provide evidence about the appropriateness of the inferences made about items.
Content Validity
Content validity determines if the survey items are representative of the topic being measured. As a first step, you must clearly state what you are interested in measuring, then you must judge whether the items are representative of the topic.
In A Nutshell:
Define what you are interested in measuring, for example "Quality."
Choose the specific aspects which require feedback, for example, "Error Rate."
Judge whether your items relate to the definitions you developed and adequately cover all aspects.
Criterion-Related Validity
Criterion-related validation relies on statistical analyses rather than judgments as in content validation. Criterion-related validation involves calculating a "validity coefficient" by correlating the survey items with another measure (criteria) already known to be related to other aspects of the attribute. For example, if satisfaction with the service department relates to the number of friends one refers to the service department, then we could correlate scores on a measure of satisfaction to an index of referrals.
Criterion measures are often complex and it is difficult to ensure they cover the scope of the survey; therefore, they are often the most difficult part of the process. If the criterion is not well developed, then there is a limit to the confidence you can have in the validity of your items. As a result, take great care to ensure that both the criterion and the items are developed carefully.
In A Nutshell:
Develop criterion.
Measure correlation between survey and criterion.
Construct Validity
Construct validation attempts to understand what is being measured by examining the relationship between constructs (an abstract idea used as an explanatory concept--such as motivation or satisfaction).
In essence, construct validation concerns theories and the relationship between constructs proposed by a theory. Construct validation involves understanding why items are related by examining the underlying concepts. The process is hypothesizing a relationship then collecting data to test the hypotheses.
In order to use construct validation, you must understand the theoretical relationship of the construct to other constructswhat it should and should not be related to. Develop relationship patterns using the survey and other measures (either other surveys, observations, ratings, or other data), then systematically examine that pattern.
In A Nutshell:
Determine the construct to be measured, for example, "Quality."
Determine relationship between the construct and other constructs, for example, "Satisfaction.'
Examine pattern of relationships.
|
About The Author:
Adapted from original publication: August, 1995
Pearson NCS
In 1962, National Computer Systems, Inc. was incorporated and became known as NCS. With the acquisition by Pearson Education in 2000, the name was officially and legally changed to NCS Pearson, Inc. Our business name in the markets we serve continues to be Pearson NCS.
http://search.pearson.com/wedo/index.htm
|
|
|
|
|
| No Feedback Yet. |
| Login using the form on the top of the page to post feedback if you have registered with TxDirector. If you have not yet registered, click here. Registration is free. |
|
Related Links
|
|
|
|
| About These Links: The links above are dynamically generated. Some links may be more relevant than others. |
|
 |
|
|
 |
© 2005 TxDirector.com - Maintained by Lanstat Incorporated - Design by Cumuli Design
Page Load Time: 0.124 secs
|
|
Disclaimer: TxDirector.com cannot accept any responsibility for any injury or damage that you may cause to yourself, others, or property when following any advice given on this site. TxDirector.com has no control of how you may use information you get from this site and does not attest to the validity of any information found within. Much of this information comes from third parties (newsletter readers and other contributors). Use advice found in our newsletters and on our site with common sense and at your own risk. If you see something in our newsletters or on our site that you disagree with, please let us know. Our goal is print only valuable information and advice. If you find any information on TxDirector.com or in our newsletters that is either erroneous and/or potentially harmful to others, please Contact Us, immediately.
|
|
|
