More Menu

Concept Of Objectivity, Validity and Reliability in Quantitative Research

byPosted at

 

Content Outline

  1. Introduction
  2. The Concept of Objectivity in the Context of Research
  3. Reliability
  4. Validity
  5. Summary

Introduction

The concepts of objectivity, validity and reliability are inter related and extremely important in a quantitative research study as these studies make use of empirical methods to investigate different phenomena or aspects of a particular phenomenon by collecting and analysing numeric data or other data that can be provided numerical values. In order to be able to generalize their findings, quantitative researchers take great care to maintain objectivity and ensure that these findings are uncontaminated by any external influences including their own presence, their values, beliefs, biases and behaviours. This is done by making sure that their findings are valid and reliable. One can make a case that validity and reliability play an important role in every step of a quantitative research right from the creation and definition of variable to the analysis and presentation of the findings. This module focuses on helping you, the student, understand how to make sure that you maintain objectivity and at the same time ensure that your research is valid and reliable.

The Concept of Objectivity in the Context of Research

When one talks about being objective, we mean that we should be unbiased and remain un-influenced by our personal feelings, opinions or prejudices. Rather, our understanding or interpretation should be based on facts. In other words, you as the researcher should maintain a distance from the research so that the findings of the research are based on the data collected and not on the values, beliefs or opinions of the researcher
Reiss and Sprenger (2014) provide strategies for maintaining objectivity in quantitative research which are explained below:
  • Focus on the facts and data collected- your interpretation should be based on the factual data that you have collected. You should be able to prove your hypothesis or your claims through the factual evidence that you have collected. 
  • Ensure that the research is value free- as far as possible, your research should not reflect your own values alone. Our values influence the choice of the research design, the selection of the tools of measurement, the development of a hypothesis/research statement as well as the interpretation of the results. However, you can ensure that your values do not impinge on the design of the research and the interpretation of data by developing a stated value premise (Myrdal, 1969: p.56, 65- 66). Myrdal provides guidelines on how this could be developed.
    These include:
    > Relevance - the value should be held by people or groups of people in society and should not be something that the researcher alone perceives as important
    > Significance – a substantial number of people should view the value as being important; alternatively, a small but socially powerful group should consider the value as being one of great consequence.
    > Feasible- values that aim for perfection or unattainable goals should not be considered as a base for research 
  • Ensure that the research is free from personal biases 
  • Ensure that the instrument we use for our research is reliable and valid.

Reliability

The concept of reliability is closely linked to consistency. A research study is considered to be reliable if the findings or results of the research are the same or are similar when the entire process has been repeated or if the same tools are used on different occasions. The term reliability is most often used with the reference to the measurement instruments as in how reliable is your tool for data collection. The more number of times a measurement instrument or tool for data collection has been used, the greater the chances of it being reliable. This is because repeated use of the instrument or tool will either provide similar results or will throw up inconsistencies and inaccuracies. When any instrument or tool has been used multiple times with similar results across all the uses, we can conclude that the said instrument or tool is reliable.

Let us suppose that we are examining psychological factors such as social isolation, loneliness, feelings of uselessness, abuse and neglect amongst senior citizens in India. Since we are interested in social isolation, we could use Lubben Social Network Scale as one of our tools. There are three versions of the Lubben Scale in addition to the original scale (LSNS) which has 10 items. The Revised Lubben Scale (LSNS- R) has 12 items while an abridged version has six (LSNS- 6) and an extended version has 18 (LSNS-18). LSNS, LSNS- R and LSNS- 6 measure social isolation amongst senior citizens (aged 65 and above) by examining the size, closeness, and frequency of contacts of the senior citizen with family (related by birth or marriage) and friends. The extended scale adds another dimension by differentiating between friends and neighbours. All items are given equal weight and scores for each item range from zero to five, the total scores range from 0- 50, 0 – 60, 0 to 30, 0 – 90for LSNS, LSNS- R, LSNS – 6 and LSNS – 18 respectively. Low scores indicate risk of social isolation and indicate poor support systems. 

However, there may be times when the tools or instruments needed for our research are not readily available. Moreover, unreliability in data may be caused by one of the following:
  •  Participant error: when data is collected across time, errors may occur due to changes within the participant such as illness, tiredness and hunger all of which may lead to a lack of concentration and focus. 
  • Researcher error: the possibility of this error occurring are higher when the number of data collectors or researchers increases and/or the number of measurements increase
What should be done to ensure reliability of our tools or instruments in such situations? Three common ways of establishing reliability of our tools or instruments are discussed in the subsections below.
  1. Test- retest method: In the test –retest method, we use the same tool or instrument two times at two different times on the same respondent and then calculate the correlation between the two sets of data. This correlation which is also known as the reliability correlation tells us how consistent or dependable the tool/ instrument is. Continuing with the above example, let us suppose that we are developing a scale to assess social support amongst senior citizens rather than use the Lubben Social Network Scale–6. If we decide to use the test-retest reliability test, we will administer the scale developed by us to each senior citizen participating in our study two times over a pre-determined time period and correlate the results of each of the tests to assess the reliability of our scale.
  2. Internal consistency method: This method focuses on assessing the extent to which a set of questions accurately measure the specific sub topic within a tool. If we continue with the above example, to measure internal consistency, we could write three or four statements delating with feelings of uselessness and place them at different points in the scale. Once we get the data, we group all these together and calculate the correlation between all four statements. Coefficient alpha or Cronbach's Alpha (so named because Cronbach was one of the first to acknowledge its value in the 1950s thereby bringing in into common use) is widely used for this purpose.
  3. Interrater method: Sometimes we gather data through observation. To ensure that there is no bias from the ‘data collector’, the interrater method is used where two ‘data collectors’ conduct the observation at the same time using the same observation guide. Both will also observe the same respondent or subject. The data is then compared either using Cohen’s Kappa (Robson, 2002, pp. 341- 42) or Spearman-Brown formula (Rosenthal & Rosnow, 1991, pp. 51- 55). This method can also be used when data collection is done by a team of data collectors to ensure consistency amongst the data collectors. 

Validity

Validity focuses on the extent to which a tool or an instrument measures that which it was supposed to measure. For example, we would check that the rating scale used for assessing psychological factors like loneliness and isolation amongst senior citizens actually measures loneliness and isolation and not physical infirmity or illnesses of senior citizens. Robson (2002, p. 553) points out that validity at its most simple, refers to the truth status of research reports. In other words, validity needs to be established across the research at every step from deciding the methodology to presentation of the findings.

Let us now look at a few common types of validity in research. 
  • Face validity or content validity: this determines whether others (peers or experts) also agree that the tool you are using actually measures what you want it to. If you have developed a tool or a scale to assess isolation and loneliness amongst senior citizens, you would ask experts or peers to examine the tool/scale and get their opinion on its validity while also ensuring that you have covered all factors related to isolation and loneliness. This would ensure that you do not discover at a later stage that your tool had focused on issues other than isolation and loneliness such as pain and neglect, for example.  
  • Construct validity: this is used in studies that seek to study the causal relationship between two variables. Here, we try to see how closely related our results are to the theory on which our hypothesis was based. For example, theory suggests that creating awareness and providing knowledge would result in improved practice. As part of our research on improving services for children in government homes, we could create a short program to improve the knowledge and awareness of caregivers in the government homes and assess the effect of this program on the practice of the caretakers. The greater the correlation between the awareness program and the improvement in practice, the higher the construct validity of the tool being used. The correlation coefficient is most commonly used to measure the construct validity of a tool. 
  • Predictive criterion validity: Like construct and face validity, this also focuses on establishing the validity of the tool or instrument of measurement. As the name suggests, this looks at how a current tool can be used to make predictions as to future performances. For example, could the scale you have developed for understanding feelings of uselessness and neglect in senior citizens be used to predict isolation and loneliness amongst senior citizens? If a senior citizen feels useless and neglected, is there a greater chance that s/he may become lonely and isolated in the future? 
  • Internal validity: this also focuses on the cause –effect relationship amongst the variables being studied. Here we examine whether we have sufficient evidence to show whether the changes that we have noted in the dependent variable are indeed the result of changes in the independent variable. Let us consider the example of a research study that seeks to improve the nutritional intake of adolescent girls by creating awareness in them regarding health foods. To this end, the researcher decides to provide information regarding nutritious foods and also recipes for nutritious and healthy snacks to adolescent girls in order to improve their awareness regarding health foods. A control and experiment group design is used and data on food intake is collected prior to the intervention, a week after the intervention and again after a gap of six months to assess retention of information. When we analyse the data, we must make sure that we have sufficient evidence to show whether the intake of nutritious food has improved due to the intervention. Also we need to make sure that the improvement in intake of nutritious food has been the result of the intervention and not due to other reasons. The data from the control group will help us do this. The stronger the evidence we have to prove that the intervention has definitely impacted positively on the intake of nutritious food among the adolescent girls, the higher the internal validity of our research
  • External validity: this is related to the extent to which the results and findings of a research study are generalizable. The greater the generalizability of the results of a research, the greater is its external validity. Continuing with the above example, if our sample has covered adolescent girls from a variety of urban settings, we could generalize that the findings of our study are true for adolescent girls in all urban settings. This means that our study has a high degree of external validity. To enhance the external validity, the study could be extended to cover adolescent girls in rural settings as well. 

Summary

  • Objectivity requires that we as researchers be unbiased and remain un-influenced by our personal feelings, opinions or prejudices when conducting the research.
  • The findings of the research must be based on the data collected and not on the values, beliefs or opinions of the researcher to ensure objectivity. 
  • Objectivity in research can be maintained by focusing on the data, ensuring that the research is value free, making sure that the researcher’s biases do not affect the research as well as ensuring that the instruments are valid and reliable. 
  • Reliability refers to the consistency of the tool of measurement used in the research
  • The three commonly used methods of ensuring reliability include test –retest method, internal consistency method and interrater method. 
  • Validity refers to the extent to which tool or an instrument measures that which it was supposed to measure.
  • Face validity, construct validity and prediction criterion validity are three types of validity that focus on the tool of measurement. 
  • Internal validity seeks to establish whether we have sufficient evidence to show whether the changes that we have noted in the dependent variable are indeed the result of changes in the independent variable. 
  • External validity is related to the extent to which the results and findings of a research study are generalizable 

References

  • Myrdal, Gunnar (1969), Objectivity in Social Research, London: Pantheon Books, New York
  • Reiss, Julian and Sprenger, Jan, "Scientific Objectivity", The Stanford Encyclopedia of Philosophy (Fall 2014 Edition), Edward N. Zalta (ed.), URL = .http://plato.stanford.edu/archives/fall2014/entries/scientific-objectivity/
  • Robson, C. (2002). Real World Research. A resource for social scientists and practitionerresearchers (2nd Ed.) Oxford: Blackwell. 
  • Rosenthal, R. and Rosnow, R. L. (1991). Essentials of Behavioral Research: Methods and Data Analysis. Second Edition. McGraw-Hill Publishing Company, pp. 46-65.
Post a Comment
Tag

Comments

Thank You

Find your topic