MBA610 CU- MBA-Sem 2- MBA610 -Business Research Methods-converted-converted

Figure 6.9 All the above figures (the study I data and the study II data) represent the respective means. Graphically, these can be represented as shown in below. Figure 6.9 The graph relating to Study I indicates that there is an interaction between the treatment and the level which, in other words, means that the treatment and the level are not independent of each other. The graph relating to Study II shows that there is no interaction effect which means that treatment and level in this study are relatively independent of each other. 100 CU IDOL SELF LEARNING MATERIAL (SLM)

The 2 × 2 design need not be restricted in the manner as explained above i.e., having one experimental variable and one control variable, but it may also be of the type having two experimental variables or two control variables. For example, a college teacher compared the effect of the class size as well as the introduction of the new instruction technique on the learning of research methodology. For this purpose he conducted a study using a 2 × 2 simple factorial design. His design in the graphic form would be as follows: Figure 6.10 But if the teacher uses a design for comparing males and females and the senior and junior students in the college as they relate to the knowledge of research methodology, in that case we will have a 2 × 2 simple factorial design wherein both the variables are control variables as no manipulation is involved in respect of both the variables. Illustration: (4 × 3 simple factorial design). The 4 × 3 simple factorial design will usually include four treatments of the experimental variable and three levels of the control variable. Graphically it may take the following form: Figure 6.11 This model of a simple factorial design includes four treatments viz., A, B, C, and D of the experimental variable and three levels viz., I, II, and III of the control variable and has 12 101 CU IDOL SELF LEARNING MATERIAL (SLM)

different cells as shown above. This shows that a 2 × 2 simple factorial design can be generalized to any number of treatments and levels. Accordingly we can name it as such and such (–×–) design. In such a design the means for the columns provide the researcher with an estimate of the main effects for treatments and the means for rows provide an estimate of the main effects for the levels. Such a design also enables the researcher to determine the interaction between treatments and levels. Complex factorial designs: Experiments with more than two factors at a time involve the use of complex factorial designs. A design which considers three or more independent variables simultaneously is called a complex factorial design. In case of three factors with one experimental variable having two treatments and two control variables, each one of which having two levels, the design used will be termed 2 × 2 × 2 complex factorial design which will contain a total of eight cells as shown below in below. Figure 6.12 A pictorial presentation is given of the design shown below 102 CU IDOL SELF LEARNING MATERIAL (SLM)

Figure 6.13 9. The dotted line cell in the diagram corresponds to Cell 1 of the above stated 2 × 2 × 2 design and is for Treatment A, level I of the control variable 1, and level I of the control variable 2. From this design it is possible to determine the main effects for three variables i.e., one experimental and two control variables. The researcher can also determine the interactions between each possible pair of variables (such interactions are called ‘First Order interactions’) and interaction between variable taken in triplets (such interactions are called Second Order interactions). In case of a 2 × 2 × 2 design, the further given first order interactions are possible: Experimental variable with control variable 1 (or EV × CV 1); Experimental variable with control variable 2 (or EV × CV 2); Control variable 1 with control variable 2 (or CV1 × CV2); Three will be one second order interaction as well in the given design (it is between all the three variables i.e., EV × CV1 × CV2). To determine the main effects for the experimental variable, the researcher must necessarily compare the combined mean of data in cells 1, 2, 3 and 4 for Treatment A with the combined mean of data in cells 5, 6, 7 and 8 for Treatment B. In this way the main effect for experimental variable, independent of control variable 1 and variable 2, is obtained. Similarly, the main effect for control variable 1, independent of experimental variable and control variable 2, is obtained if we compare the combined mean of data in cells 1, 3, 5 and 7 with the combined mean of data in cells 2, 4, 6 and 8 of our 2 × 2 × 2 factorial design. On similar lines, one can determine the main effect for the control 103 CU IDOL SELF LEARNING MATERIAL (SLM)

variable 2 independent of experimental variable and control variable 1, if the combined mean of data in cells 1, 2, 5 and 6 are compared with the combined mean of data in cells 3, 4, 7 and 8. To obtain the first order interaction, say, for EV × CV1 in the above stated design, the researcher must necessarily ignore control variable 2 for which purpose he may develop 2 × 2 design from the 2 × 2 × 2 design by combining the data of the relevant cells of the latter design as shown in Fig. below Figure 6.14 Similarly, the researcher can determine other first order interactions. The analysis of the first order interaction, in the manner described above, is essentially a sample factorial analysis as only two variables are considered at a time and the remaining one is ignored. But the analysis of the second order interaction would not ignore one of the three independent variables in case of a 2 × 2 × 2 design. The analysis would be termed as a complex factorial analysis. It may, however, be remembered that the complex factorial design need not necessarily be of 2 × 2 × 2 type design, but can be generalized to any number and combination of experimental and control independent variables. Of course, the greater the number of independent variables included in a complex factorial design, the higher the order of the interaction analysis possible. But the overall task goes on becoming more and more complicated with the inclusion of more and more independent variables in our design. Factorial designs are used mainly because of the two advantages. i. They provide equivalent accuracy (as happens in the case of experiments with only one factor) with less labor and as such are a source of economy. Using factorial designs, we can determine the main effects of two (in simple factorial design) or more (in case of complex factorial design) factors (or variables) in one single experiment. ii.They permit various other comparisons of interest. For example, they give information about such effects which cannot be obtained by treating one single factor at a time. The determination of interaction effects is possible in case of factorial designs. 104 CU IDOL SELF LEARNING MATERIAL (SLM)

SUMMARY We have noticed that research design is a plan, structure and strategies of the collection measurement and analysis of data. Research design purports to obtain answers to research questions and controlling variance. Moreover, research design answers the question as objectively, validly and economically as possible. Main functions of the research design are to maximize the effect of systematic variance, control extraneous variance through randomization, elimination, matching and statistical control and minimize the error variance. A good research design is characterized by feasibility, flexibility, generalizability, theory base, cost and time. The overall blueprint of the experiment is called experimental design. Basic elements of experimental design are choice or existence of control group and allocation of the subjects randomly to both the groups. Factors, levels, condition, treatment, main effect and interaction effect are the basic terms which are used in experimental design We discussed the concept of research design. Research design is the blueprint of your research work. After a discussion of the need and purpose, its functions are discussed. There are different kinds of research designs based on a number of factors. You have to choose one depending upon the purpose of your research work KEY WORDS/ABBREVIATIONS • Descriptive Studies: Research studies that are carried out to describe an object, phenomena, process, or organization in the present. • Experimental Studies: Research studies that are undertaken to study cause and effect relations between variables are called experimental studies. • Research Process Reliability: The measure of being able to measure the variables with the same accuracy at different times under similar conditions refer to reliability. • Research Design: The strategy that a researcher adopts to undertake his/ her research. It concerns the operationalization of hypothesis, data collection, and data analysis. • Content Analysis: It is a method of data collection for studying events that have taken place in the past on the basis of literature. • Validity: Validity is a measure of the extent of what y. 105 CU IDOL SELF LEARNING MATERIAL (SLM)

LEARNING ACTIVITY 1. Define survey approach to data collection. Differentiate between survey and case study method 2. Explain the meaning of following in context of experimental design; a) factor b) levels c) main effects d) treatment UNIT END QUESTIONS (MCQ AND DESCRIPTIVE) A. Descriptive Types Questions 1. Explain experimental design method in research? 2. Discuss the basic elements and types of experimental design. 3. Explain, what do you mean by research design? 4. Discuss the basic purposes of research design. How can you minimize the extraneous variance? Discuss the various ways to control the extraneous variable. 5. Discuss criteria of a research design with appropriate example. 6. Make distinction between random assignment and random selection in terms of their uses in research. 7. Discuss the qualities of research design B. Multiple Choice Questions 1. Who said ‘Research design is the plan, structure, and strategy of investigation conceived so as to obtain answers to research questions and to control variance’. a. Myers b. Mcguigan c. Matheson d. Kerlinger 106 CU IDOL SELF LEARNING MATERIAL (SLM)

2. Which technique is not included to control the effect the extraneous variable? a. Matching b. Elimination c. Extinction d. Randomization 3. Which one of the following is not considered as the purpose of research design. a. Error variance b. Extraneous variable c. Statistical variance d. Systematic variance 4. Which one of the following is considered as most appropriate technique of control of the extraneous variables? a. Elimination b. Randomization c. Matching d. Statistical Control 5. A Good design possess following qualities except one: a. Feasible b. Simple c. Efficient d. Theory base 107 CU IDOL SELF LEARNING MATERIAL (SLM)

Answer 1. d 2. c 3. c 4. b 5. b REFERENCES • Campbell D.T & Stanley, J.C (1963). Experimental and Quasi-Experimental Designs for Research. New York: Russell Sage Foundation. Kirk, R.E. (1968). • Experimental Design: Procedures for the Behavioral Sciences. Belmont: Calif Brooks/Cole. Solomon, B K (1949). • An Extension of Control Group Design. Psychological Bulletin, 46,137-50. Broota, K.D. (1997). Experimental Design. New Age International, New Delhi • Kerlinger, F N (1986). Foundations of Behavioral Research. New York: Holt Rinehart and Winston. Matheson et.al. (1970) Experimental Psychology (Holt), Myers, A. (1980). • Experimental Psychology. New York: Van Nostrand. Thyer, B.A. (1993) ‘ Single- systems Research Design’ in R.M. Grinnell (ed), Social Work, Research and Evaluation (4th ed) • Itasca Illinois: Peacock. Winer, B.J. (1971). Statistical Principles in Experimental Design. New York: McGraw Hill. 108 CU IDOL SELF LEARNING MATERIAL (SLM)

UNIT 7: MEASUREMENT Structure Learning Objectives Introduction Concept of measurement Problems in measurement in research – Validity and Reliability Measurement error Types of reliability. Types of validity Levels of measurement – Nominal, Ordinal, Interval, Ratio Summary Key Words/Abbreviations Learning Activity Unit End Questions (MCQ and Descriptive) References LEARNING OBJECTIVES After studying this unit, you will be able to: • Explain the concept of Measurement • Describe various levels of Measurement • Discuss concepts of Validity and Reliability INTRODUCTION The process of measurement is central to quantitative research. Quantitative research requires that the variables under consideration can be measured. It provides the fundamental connection between empirical observation and mathematical expression of quantitative 109 CU IDOL SELF LEARNING MATERIAL (SLM)

relationships. In simple terms, in measurement, the researcher assigns numerals to objects, events or properties according to certain rules. For example, unemployment increased by 5%. The female literacy rate was 54% as estimated in 2001 Census. It is relatively easy to measure properties like height, weight, or time than attributes like beauty, friendship or choosing a religion which are abstract CONCEPT OF MEASUREMENT Measurement is the process of systematically assigning numbers to objects and their properties, to facilitate the use of mathematics in studying and describing objects and their relationships. Some types of measurement are fairly concrete: for instance, measuring a person’s weight in pounds or kilograms, or their height in feet and inches or in meters. Note that the particular system of measurement used is not as important as a consistent set of rules: we can easily convert measurement in kilograms to pounds, for instance. Although any system of units may seem arbitrary (try defending feet and inches to someone who grew up with the metric system!), as long as the system has a consistent relationship with the property being measured, we can use the results in calculations. Measurement is not limited to physical qualities like height and weight. Tests to measure abstractions like intelligence and scholastic aptitude are commonly used in education and psychology, for instance: the field of psychometrics is largely concerned with the development and refinement of methods to test just such abstract qualities. Establishing that a particular measurement is meaningful is more difficult when it can’t be observed directly: while you can test the accuracy of a scale by comparing the results with those obtained from another scale known to be accurate, there is no simple way to know if a test of intelligence is accurate because there is no commonly agreed-upon way to measure the abstraction “intelligence.” To put it another way, we don’t know what someone’s actual intelligence is because there is no certain way to measure it, and in fact we may not even be sure what “intelligence” really is, a situation quite different from that of measuring a person’s height or weight. These issues are particularly relevant to the social sciences and education, where a great deal of research focuses on just such abstract concepts. PROBLEMS IN MEASUREMENT IN RESEARCH – VALIDITY AND RELIABILITY Reliability refers to the consistency or repeatability of an operationalized measure. A reliable measure will yield the same results over and over again when applied to the same thing. It is 110 CU IDOL SELF LEARNING MATERIAL (SLM)

the degree to which a test consistently measures whatever it measures. If you have a survey question that can be interpreted several different ways, it is going to be unreliable. One person may interpret it one way and another may interpret it another way. You do not know which interpretation people are taking. Even answers to questions that are clear may be unreliable, depending on how they are interpreted. Reliability refers to the consistency of scores obtained by the same persons when they are reexamined with the same tests on different occasions, or with different sets of equivalent items, or under other variable examining conditions. Research requires dependable measurement. Measurements are reliable to the extent that they are repeatable and that any random influence which tends to make measurements different from occasion to occasion or circumstance to circumstance is a source of measurement error. Errors of measurement that affect reliability are random errors and errors of measurement that affect validity are systematic or constant errors. Reliability of any research is the degree to which it gives an accurate score across a range of measurement. It can thus be viewed as being ‘repeatability’ or ‘consistency’. In summary – Inter-rater: Different people, same test. Test-retest: Same people, different times. Parallel-forms: Different people, same time, different test. Internal consistency: Different questions, same construct. Test-retest, equivalent forms and split-half reliability are all determined through correlation. There are a number of ways of determining the reliability of an instrument. The procedure can be classified into two groups – External Consistency Procedures: It compare findings from two independent processes of data collection with each other as a means of verifying the reliability of the measure. For example, test-retest reliability, parallel forms of the same test, etc. Internal Consistency Procedures: The idea behind this procedure is that items measuring them same phenomenon should produce similar results. For example, split-half technique Reliability is important because it enables researchers to have some confidence that the measure they taken are close to the true measure. Validity is important because it tell researchers that the measure they taken is actually measures what they hope it does. So, if researchers want to know how good the measurement is, they should depend on the reliability and validity of a measurement. Reliability is synonym of repeatability and consistency. Reliability defined as the degree to which test scores are free from errors of measurement (AERA et al., 1999, p.180 in Neukrug 111 CU IDOL SELF LEARNING MATERIAL (SLM)

& Fawcett, 2006). The degree of reliability can decide whether the scores or data that researchers obtained can be relied to measure a variable or construct. Measurement error An unreliable measurement is caused by error source of variability. There are two types of error which are Systematic Measurement Error and Unsystematic Measurement Error. Systematic measurement error is the factors that affect measurement systematically across the time. It is predictable and can be eliminated if it gets identified. It is also related to validity of a measurement. Systematic measurement error arises when researchers unknown to the test developer and a test measure something others than the trait that researchers tend to measure. These may seriously influence the validity of a test. Unsystematic measurement error is the effects or errors that unpredictable and inconsistent. It is related to reliability of a measurement. Item selection, test administration, and test scoring are examples of unsystematic measurement error. Item selection means that error happened in the instrument itself. The example of this error such as instrument which includes not valid questions or items, contents can’t fair to all respondents even though it is already considered as good, and there are too many items inside the test. Test administration error includes uncomfortable room, dim lighting, noise in room, fatigue, nervous, and others which may influence respondents’ performances. For the test scoring error, it happened when the format of test not using machine-score multiple-choice items. Subjective judgment in scoring occurred especially for the projective test and essay questions. Rorschah Inkblot Test, Sentence Completion Test, and Thematic Apperception Test are related to subjective judgment. Types of reliability. There are two major types of reliability which are Reliability as Temporal Stability and Reliability as Internal Consistency. Reliability as Temporal Stability is related to the times to collect data. Reliability as Temporal Stability includes Test-retest and Alternate-forms Reliability. Internal Consistency includes Split-half, Coefficient Alpha, and Interscorer Reliability. Test-retest reliability defined as the relationship between scores from one test given at two different administrations (Neukrug & Fawcett, 2006). Alternate-forms Reliability is the relationship between the scores from two version of same test. In this type of reliability, everything in the different version test such as the difficulty level, number of items, and 112 CU IDOL SELF LEARNING MATERIAL (SLM)

content should be same. Split-half reliability defined as correlating one-half of the test to the other half. Researchers can divide the test into two parts which are first half and second half. They also can divide the items by odd numbers and even numbers of the items. Spearman- Brown used when the numbers of items in test is short. Spearman-Brown is more accurate when the numbers of items is few. Coefficient Alpha and Kuder Richardson determined by correlating the scores of each item with total scores on the test. Kuder Richardson used when the items need to be answered by “yes” and “no”. Interscorer Reliability defined as correlating the scores from two or more observers’ rating to the same phenomenon. Observers should be trained to rating on the events or behaviors of respondents. Test-retest is appropriate be used when researchers aim to measure the behaviors of respondents across times. Coefficient Alpha is appropriate to be used in both uni-dimensionality tests. Split the test by odd and even numbers is appropriate to be used when the difficulties of items have carefully ordered. If the difficulties level of items is not carefully orders, the method of split the test to first half and second half is appropriate. Interscorer reliability used when the test involves subjectivity of scoring. FACTORS EFFECTING THE RELIABILITY OF A RESEARCH It is quite impossible to have a research tool which is 100% accurate. Not only the instrument cannot be so but also because to control the factors effecting the reliability. Some of these factors are – Wording of Questions – a slight ambiguity in the wording of questions or statements can affect the reliability of a research instrument as respondents may interpret the questions differently at different times, resulting in different responses. Physical Setting - in the case of an instrument being used in an interview, any change in the physical setting at the time of the repeat interview may affect the responses given by a respondent, which may affect reliability. Respondent’s Mood - a change in a respondent’s mood when respondent to questions or writing answers in a questionnaire can change and may affect the reliability of that instrument. Nature of Interaction - in an interview, the interaction between the interviewer and the interviewee can affect responses significantly. During the repeat interview the responses given may be different due to a change in interaction, which could affect reliability. Regression Effect of An Instrument – when a research instrument is used to measure attitudes towards an issue, some respondents, after having expressed their opinion, may feel that they have been either too negative or too positive towards the issue. The second time they may express their opinion differently, thereby affecting reliability. Validity refers to whether the measure actually measures what it is supposed to measure. If 113 CU IDOL SELF LEARNING MATERIAL (SLM)

a measure is unreliable, it is also invalid. That is, if you do not know what it is measuring, it certainly cannot be said to be measuring what it is supposed to be measuring. On the other hand, you can have a consistently unreliable measure. For example, if we measure income level by asking someone how many years of formal education they have completed, we will get consistent results, but education is not income (although they are positively related). If the ‘trade dress’ of a product refers to the total image of a product, then measuring how people perceive the product’s color and shape by themselves falls far short of measuring the product’s ‘trade dress’. It is an invalid measure. In general, validity is an indication of how sound your research is. More specifically, validity applies to both the design and the methods of your research. Validity in data collection means that your findings truly represent the phenomenon you are claiming to measure. Valid claims are solid claims Validity is described as the degree to which a research study measures what it intends to measure. There are two main types of validity, internal and external. Internal validity refers to the validity of the measurement and test itself, whereas external validity refers to the ability to generalize the findings to the target population. Both are very important in analyzing the appropriateness, meaningfulness and usefulness of a research study. Some factors which affect internal validity are - Subject variability; Size of subject population; Time given for the data collection or experimental treatment; History; Attrition; Maturation; Instrument/task sensitivity etc. The important factors affect external validity are - Population characteristics (subjects); Interaction of subject selection and research; Descriptive explicitness of the independent variable; The effect of the research environment; Researcher or experimenter effects; Data collection methodology; The effect of time etc. Validity refers to an accuracy of a measure. A measurement is valid when it measures what the researchers supposed to measure (Gregory, 2007). For example, IQ tests are supposed to measure intelligence and depression tests are supposed to measure depression level or symptoms of respondents. Normally, the inferences drawn from a valid test are appropriate, meaningful, and useful. Types of validity There are three types of validity which are Content Validity, Criterion Validity, and Construct Validity. For the Criterion Validity, it includes Predictive Validity and Concurrent Validity. For the Construct Validity, it includes Convergent and Discriminant Validity. 114 CU IDOL SELF LEARNING MATERIAL (SLM)

Content validity determined by the degree to which the questions, tasks, or items on a test are representative of the universe of behavior the test was designed to sample (Gregory, 2007). The appropriateness of content of a measurement is determined by experts. Researchers make a judgment on whether the items in a measurement have covered all domains that they want to measure. For example, teacher would like to develop a test which tends to measure the learning of students toward a subject from chapter 1 to 5. The type and number of questions are designed. Sixty multiple-choices questions and 60 minutes are given to the students to do the test. Ten questions will cover each chapter and the rest questions will cover chapter five which considered as the most important chapter in the test. Validity of content also can be made by the experts’ rating towards each item to decide whether the items can indicate the content or not. Two experts evaluate each item on the four-point scale. The rating of each expert on each item can be dichotomized into weak relevance of content (rating of 1 and 2) and strong relevance of content (rating of 3 and 4).If both experts agree that the item is strongly relevance, then the item will be put in cell D; if both experts agree that the item has weak relevance, the item will be put in cell A. Cell B and C involved the items that agreed by one expert and disagreed by another expert. For the Criterion validity, both Predictive and Concurrent validity will made by comparing them with others criterion. Concurrent validity correlates test scores with criterion scores and these two types of scores are obtained in the same time. For example, researchers would like to measure the reading ability of students by using the Reading Achievement Test. Researchers compare the Reading Achievement Test scores of students with the teachers’ rating scores on students’ reading abilities. High correlation between the two scores indicates that there is high concurrent validity in the test. For the Predictive validity, it correlates test scores with criterion scores which obtained in the future. It means that the scores or data are obtained in different time. For example, Employment Test used to measure the performances of employee in a company or organization. At first, researchers give the test to employee and after six months, the supervisors asked to give evaluation to the performances of employee. Then, researchers compare the test scores and supervisors’ rating scores to see the level of validity. The difference between Concurrent and Predictive validity is the time frame used to obtain the data and scores. For Construct validity, construct is a theoretical, intangible quality or trait in which individuals differ. It is abstract and hard to be measured. Thus, it needs some indicators or signs to represent it. A construct is a collection of related behaviors that can represent the 115 CU IDOL SELF LEARNING MATERIAL (SLM)

things that researcher want to measure. Construct validity is evidence that an idea or concept is being measured by a test (Neukrug & Fawcett, 2006). For example, depression is a construct and it manifested by some behaviors such as lethargy, difficulty concentrate and loss of appetite. Homogeneity refers to a test measure a single construct. Homogeneous refers to the single component or subtest in a Homogeneity test. The purpose of homogeneity is selecting items which potential to form a homogeneous scale. Convergent validity defined as a test highly correlates with other variables which have same or overlap constructs. For example, researchers would like to take the Beck Depression Inventory-II (BDI-II) to compare with others tests which have same variables as well. The result shows that, BDI-II has high correlation with Scale for Suicide Ideation (r =.37); Beck Hopelessness Scale (r = .68); Hamilton Psychiatric Rating Scale for Depression (r =.71); and Hamilton Rating Scale for Anxiety (r =.47). Lastly, for the Discriminant validity, it means that a test does not correlate with the variables or test which are not measure the different variables or constructs. FACTORS AFFECTING THE VALIDITY OF A RESEARCH Factors that can affect validity can come in many forms, and it is important that these are controlled for as much as possible during research to reduce their impact on validity. The term history refers to effects that are not related to the treatment that may result in a change of performance over time. This could refer to events in the participant’s life that have led to a change in their mood etc. Instrumental bias refers to a change in the measuring instrument over time which may change the results. This is often evident in behavioral observations where the practice and experience of the experimenter influences their ability to notice certain things and changes their standards. A main threat to internal validity is testing effects. Often participants can become tired or bored during an experiment, and previous tests may influence their performance. This is often counterbalanced in experimental studies so that participants receive the tasks in a different order to reduce their impact on validity. If the results of a study are not deemed to be valid then they are meaningless to our study. If it does not measure what we want it to measure then the results cannot be used to answer the research question, which is the main aim of the study. These results cannot then be used to generalize any findings and become a waste of time and effort. It is important to remember that just because a study is valid in one instance it does not mean that it is valid for measuring something else. So, validity is very important in a research study to ensure that our results can be used effectively, and variables that may threaten validity should be 116 CU IDOL SELF LEARNING MATERIAL (SLM)

controlled as much as possible. Relationship between Reliability and Validity A good validity need to have good reliability established first. However, a good reliability does not lead to a good validity. A good reliability only reflex that the scores in a measurement is appeared consistently. A good validity may leads to reliability. When the measurement or test tends to measure what researchers tend to measure, the validity occurred and thus the reliability occurred also. In a test, reliability is necessary but not sufficient for validity. In other words, measure can be reliable but not valid; valid measures must be reliable, however. Measure is important in research. Measure aims to ascertain the dimension, quantity, or capacity of the behaviors or events that researchers want to explore. According to Maxim (1999), measurement is a process of mapping empirical phenomena with using system of numbers. Basically, the events or phenomena that researchers interested can be existed as domain. Measurement links the events in domain to events in another space which called range. In another words, researchers can measure certain events in certain range. The range is consisting of scale. Thus, researchers can interpret the data with quantitative conclusion which leads to more accurate and standardized outcomes. Without measure, researchers can’t interpret the data accurately and systematically. LEVELS OF MEASUREMENT – NOMINAL, ORDINAL, INTERVAL, RATIO Level of measurement refers to the relationship among the values that are assigned to the attributes for a variable. It is important because - First, knowing the level of measurement helps you decide how to interpret the data from that variable. When you know that a measure is nominal, then you know that the numerical values are just short codes for the longer names. Second, knowing the level of measurement helps you decide what statistical analysis is appropriate on the values that were assigned. If a measure is nominal, then you know that you would never average the data values or do a t-test on the data. S. S. Stevens (1946) clearly delineated the four distinguish levels of measurement. The levels are - nominal, ordinal, interval, or ratio. Stevens’s levels of measurement are 117 CU IDOL SELF LEARNING MATERIAL (SLM)

important for at least two reasons. First, they emphasize the generality of the concept of measurement. Although people do not normally think of categorizing or ranking individuals as measurement, in fact they are as long as they are done so that they represent some characteristic of the individuals. Second, the levels of measurement can serve as a rough guide to the statistical procedures that can be used with the data and the conclusions that can be drawn from them. With nominal-level measurement, for example, the only available measure of central tendency is the mode. Also, ratio-level measurement is the only level that allows meaningful statements about ratios of scores. One cannot say that someone with an IQ of 140 is twice as intelligent as someone with an IQ of 70 because IQ is measured at the interval level, but one can say that someone with six siblings has twice as many as someone with three because number of siblings is measured at the ratio level. NOMINAL: The nominal scale (also called dummy coding) simply places people, events, perceptions, etc. into categories based on some common trait. Some data are naturally suited to the nominal scale such as males vs. females, white vs. black vs. blue, and American vs. Asian. The nominal scale forms the basis for such analyses as Analysis of Variance (ANOVA) because those analyses require that some category is compared to at least one other category. The nominal scale is the lowest form of measurement because it doesn’t capture information about the focal object other than whether the object belongs or doesn’t belong to a category; either you are a smoker or not a smoker, you attended university or you didn’t, a subject has some experience with computers, an average amount of experience with computers, or extensive experience with computers. No data is captured that can place the measured object on any kind of scale say, for example, on a continuum from one to ten. Coding of nominal scale data can be accomplished using numbers, letters, labels, or any symbol that represents a category into which an object can either belong or not belong. In research activities a Yes/No scale is nominal. It has no order and there is no distance between Yes and No. The statistics which can be used with nominal scales are in the non- parametric group. The most likely ones would be - mode; cross tabulation - with chi-square. There are also highly sophisticated modelling techniques available for nominal data. Ordinal: An ordinal level of measurement uses symbols to classify observations into categories that are not only mutually exclusive and exhaustive; in addition, the categories have some explicit relationship among them. For example, observations may be classified into categories such as taller and shorter, greater and lesser, faster and slower, harder and easier, and so forth. However, each observation must still fall into one of the categories (the categories are exhaustive) but no more than one (the categories are mutually exclusive). Most of the commonly used questions which ask about job satisfaction use the ordinal level of measurement. 118 CU IDOL SELF LEARNING MATERIAL (SLM)

For example, asking whether one is very satisfied, satisfied, neutral, dissatisfied, or very dissatisfied with one’s job is using an ordinal scale of measurement. The simplest ordinal scale is a ranking. When a market researcher asks you to rank 5 types of tea from most flavorful to least flavorful, s/he is asking you to create an ordinal scale of preference. There is no objective distance between any two points on your subjective scale. For you the top tea may be far superior to the second preferred tea but, to another respondent with the same top and second tea, the distance may be subjectively small. Ordinal data would use non- parametric statistics. These would include - median and mode; rank order correlation; non- parametric analysis of variance. Modelling techniques can also be used with ordinal data. INTERVAL: An interval level of measurement classifies observations into categories that are not only mutually exclusive and exhaustive, and have some explicit relationship among them, but the relationship between the categories is known and exact. This is the first quantitative application of numbers. In the interval level, a common and constant unit of measurement has been established between the categories. For example, the commonly used measures of temperature are interval level scales. We know that a temperature of 75 degrees is one degree warmer than a temperature of 74 degrees, just as a temperature of 42 degrees is one degree warmer than a temperature of 41 degrees. Numbers may be assigned to the observations because the relationship between the categories is assumed to be the same as the relationship between numbers in the number system. For example, 74+1= 75 and 41+1= 42. The intervals between categories are equal, but they originate from some arbitrary origin, that is, there is no meaningful zero point on an interval scale. The standard survey rating scale is an interval scale. When you are asked to rate your satisfaction with a piece of software on a 7 point scale, from Dissatisfied to Satisfied, you are using an interval scale. Interval scale data would use parametric statistical techniques - Mean and standard deviation; Correlation; Regression; Analysis of variance; Factor analysis; Plus a whole range of advanced multivariate and modelling techniques. Remember that you can use non-parametric techniques with interval and ratio data. But non- parametric techniques are less powerful than the parametric ones. RATIO: The ratio level of measurement is the same as the interval level, with the addition of a meaningful zero point. There is a meaningful and non-arbitrary zero point from which the equal intervals between categories originate. For example, weight, area, speed, and velocity are measured on a ratio level scale. In public policy and administration, budgets and the number of program participants are measured on ratio scales. In many cases, interval and ratio scales are treated alike in terms of the statistical tests that are applied. A ratio scale is the top level of measurement and is not often available in social research. The factor which 119 CU IDOL SELF LEARNING MATERIAL (SLM)

clearly defines a ratio scale is that it has a true zero point. The simplest example of a ratio scale is the measurement of length (disregarding any philosophical points about defining how we can identify zero length). Ratio scale data would use the same as for Interval data. According to Virginia L. Senders the simple way to learn the levels of measurement or to select a measurement scale is as follows – If one object is different from another, then we use a nominal scale. If one object is bigger or better or more of anything than another, then we use an ordinal scale. If one object is so many units (degrees, inches, etc.) more than another, then we use an interval scale. If one object is certain times as big or bright or tall or heavy as another, then we use a ratio scale. The following criteria should be considered in the selection of the measurement scale for variables in a study. Researcher should consider the scale that will be most suitable for each variable under study. Important points in the selection of measurement scale for a variable are – Scale selected should be appropriate for the variables one wishes to categorize. • It should be of practical use. • It should be clearly defined. • The number of categories created (when necessary) should cover all possible values. • The number of categories created (when necessary) should not overlap, i.e., it should be mutually • exclusive. The scale should be sufficiently powerful. • Variables measured at a higher level can always be converted to a lower level, but not vice versa. For example, observations of actual age (ratio scale) can be converted to categories of older and younger (ordinal scale), but age measured as simply older or younger cannot be converted to measures of actual age. The four levels of measurement discussed above have an important impact on how you collect data and how you analyze them later. Collect at the wrong level, and you will end of having to adjust your research, your design, and you analyze. Make sure you consider carefully the level at which you collect your data, especially in light of what statistical procedures you intend to use once you have the data in hand. SUMMARY 120 CU IDOL SELF LEARNING MATERIAL (SLM)

This unit describes concepts and constructs, different types of variables, the various levels of measurement, scales and indices, reliability and validity of measurement, quantitative research methods and the limitations of quantitative research. There are different types of variables such as quantitative, qualitative, discrete, continuous, and dependent; and independent variables. There are four levels of measurements: nominal, ordinal, interval and ratio. To be useful in research, measurements must be valid and reliable. A learning of all of the above is helpful in learning quantitative research. Learning Quantitative Research Quantitative research relates to empirical methods which are based on facts and figures. Concepts can be used by researchers to make meaningful summaries of observations. A construct is a combination of concepts that are difficult to observe directly, but can be inferred from related behavior patterns. A variable is a phenomenon or event that can be measured or manipulated and is used in quantitative research. The values of the dependent variable depend on the effects or changes in the independent variables. The values of the independent variable are not manipulated and are only observed or measured. The dependent variable is what the researcher wishes to explain. Scales and indices represent composite measures of variables or measurements that are based on more than one item. All scales are tested to ensure reliability and validity of measurement before being used in research. A measurement is reliable if it consistently gives the same answer at different points in time. If a measuring device measures what it is supposed to measure the device is a valid one. Quantitative and qualitative research have differences and similarities in many ways that govern the methodology and outcome of research. KEY WORDS/ABBREVIATIONS • Quantitative: A description or analysis of a phenomenon that involves Research specific measurement of variables. • Variable: A phenomenon or event that is measured or manipulated in research. • Reliability: A measurement that consistently gives the same answer at different points in time. • Validity: A measurement that measures what it is supposed to measure. • Ratio: The ratio level of measurement is the same as the interval level, with the addition of a meaningful zero point. LEARNING ACTIVITY 121 CU IDOL SELF LEARNING MATERIAL (SLM)

1. Discuss various factors effecting the reliability of a research 2. Define various types of Validity. UNIT END QUESTIONS (MCQ AND DESCRIPTIVE) A. Descriptive Types Questions 1. What do you mean by measurement? Explain 2. Discuss the problems in measurement in research. 3. Define Validity and Reliability. 4. Differentiate between Nominal and Ordinal levels of measurement. 5. Differentiate between Interval and Ratio. B. Multiple Choice Questions 1. Which of the following is NOT one of the main purposes of a theory? a. Suggest several alternative explanations of phenomena. b. Explain why phenomena are related and what this means. c. Describe the relationships between observed phenomena d. Predict how the results of research studies will turn out. 2. Asking participants to recruit further participants by word-of-mouth is what type of sampling? a. Self-Selecting b. Snowball c. Quota d. Stratified random. 122 CU IDOL SELF LEARNING MATERIAL (SLM)

3. If groups of participants are selected to represent sub-groups in the population (e.g. such as selecting an entire class of psychology students to be compared to a group of history students), this is known as… a. Simple random sampling. b. Opportunity sampling. c. Haphazard sampling. d. Cluster sampling. 4. A good theory is… a. General, parsimonious and testable. b. Precise, complex and testable c. Precise, parsimonious and testable d. Precise, parsimonious and untestable. 5. Which of the following is NOT a characteristic of an untestable hypothesis? a. Inadequate definition of concepts. b. Circularity c. Non-directionality. d. Appeal to unscientific notions. Answer 1. a 2. b 3. d 4. c 5. c REFERENCES • Kabir, S.M.S. (2016). Basic Guidelines for Research: An Introductory Approach for All Disciplines. Book Zone Publication, ISBN: 978-984-33-9565-8, Chittagong- 4203, Bangladesh. • Kabir, S.M.S. (2017). Essentials of Counseling. Abosar Prokashana Sangstha, ISBN: 978-984- 8798-22-5, Bangla bazar, Dhaka-1100. Kabir, S.M.S., Mostafa, M.R., Chowdhury, A.H., & Salim, M.A.A. (2016). Bangladesher Samajtattwa (Sociology of Bangladesh). Protik Publisher, ISBN: 978-984-8794-69-2, Dhaka-1100. Kabir, S.M.S. (2018). 123 CU IDOL SELF LEARNING MATERIAL (SLM)

• Psychological health challenges of the hill-tracts region for climate change in Bangladesh. Asian Journal of Psychiatry, Elsevier,34, 74–77. • Kabir, S.M.S., Aziz, M.A., & Jahan, A.K.M.S. (2018). Women Empowerment and Governance in Bangladesh. ANTYAJAA: Indian journal of Women and Social Change, SAGE Publications India Pvt. Ltd, 3(1), 1-12. • Allam, S.S. & Kabir, S.M.S. (2015). Classroom Management in Secondary Level: Bangladesh Context. International Journal of Scientific and Research Publications, 5(8), 1-4, ISSN 2250-3153, www.ijsrp.org. • Allam, S.S., Kabir, S.M.S., & Akhtar, R. (2015). General Observation, Cognition, Emotion, Social, Communication, Sensory Deficiency of Autistic Children. Indian Journal of Health and Wellbeing, 6(7), 663-666, ISSN-p-2229-5356, e-2321-3698. Kabir, S.M.S. (2013). • Positive Attitude Can Change Life. Journal of Chittagong University Teachers’ Association, 7, 55-63. Kabir, S.M.S. & Mahtab, N. (2013). Gender, Poverty and Governance Nexus: Challenges and Strategies in Bangladesh. Empowerment a Journal of Women for Women, Vol. 20, 1-12 124 CU IDOL SELF LEARNING MATERIAL (SLM)

UNIT 8: SAMPLING 125 Structure Learning Objectives Introduction Concepts of Statistical Population The Basics of Population Sub population Sample Sampling Frame Obtaining and organizing a sampling frame Sampling frames problems Sampling Error Random sampling Sample Size Non-Response Characteristics of a good sample. Summary Key Words/Abbreviations Learning Activity Unit End Questions (MCQ and Descriptive) References LEARNING OBJECTIVES CU IDOL SELF LEARNING MATERIAL (SLM)

After studying this unit, you will be able to: • State the concepts of Statistical • Explain concept of Sample, Sampling Frame and Sampling Error • Describe the characteristics of Good Sample • Discuss Sample Size and Non-Response INTRODUCTION Sampling has been an age old practice in everyday life. Whenever we want to buy a huge quantity of a commodity, we decide about the total lot by simply examining a small fraction of it. It has been established that the sample survey if planned properly, can give very precise information. Since in surveys a part of the population is only surveyed and inference is drawn about the whole population, the results likely to be different from the population values. But the advantage with the sample survey is that this type of error can be measured and controlled and it can be eliminated to great extent by employing properly trained persons in surveys. The other advantage of sample surveys are that it is less time consuming and involves less cost. Usually, the population is too large for the researcher to attempt to survey all of its members. A small, but carefully chosen sample can be used to represent the population. The sample reflects the characteristics of the population from which it is drawn CONCEPTS OF STATISTICAL POPULATION In statistics, a population is a set of similar items or events which is of interest for some question or experiment. A statistical population can be a group of existing objects (e.g. the set of all stars within the Milky Way galaxy) or a hypothetical and potentially infinite group of objects conceived as a generalization from experience (e.g. the set of all possible hands in a game of poker).A common aim of statistical analysis is to produce information about some chosen population. In statistical inference, a subset of the population (a statistical sample) is chosen to represent the population in a statistical analysis. The ratio of the size of this statistical sample to the size of the population is called a sampling fraction. It is then possible to estimate the population parameters using the appropriate sample statistics. In statistics the term \"population\" has a slightly different meaning from the one given to it in ordinary speech. It need not refer only to people or to animate creatures - the population of Britain, for instance or the dog population of London. Statisticians also speak of a population of objects, or events, or procedures, or observations, including such things as the 126 CU IDOL SELF LEARNING MATERIAL (SLM)

quantity of lead in urine, visits to the doctor, or surgical operations. A population is thus an aggregate of creatures, things, cases and so on. Although a statistician should clearly define the population he or she is dealing with, they may not be able to enumerate it exactly. For instance, in ordinary usage the population of England denotes the number of people within England's boundaries, perhaps as enumerated at a census. But a physician might embark on a study to try to answer the question \"What is the average systolic blood pressure of Englishmen aged 40-59?\" But who are the \"Englishmen\" referred to here? Not all Englishmen live in England, and the social and genetic background of those that do may vary. A surgeon may study the effects of two alternative operations for gastric ulcer. But how old are the patients? What sex are they? How severe is their disease? Where do they live? And so on. The reader needs precise information on such matters to draw valid inferences from the sample that was studied to the population being considered. Statistics such as averages and standard deviations, when taken from populations are referred to as population parameters. They are often denoted by Greek letters: the population mean is denoted by μ(mu) and the standard deviation denoted by ς (low case sigma) The Basics of Population A population can be defined by any number of characteristics within a group that statisticians use to draw conclusions about the subjects in a study. A population can be vague or specific. Examples of population (defined vaguely) include the number of new- born babies in North America, total number of tech start-ups in Asia, average height of all CFA exam candidates in the world, mean weight of U.S. taxpayers and so on. Population can also be defined more specifically, such as the number of new-born babies in North America with brown eyes, the number of start-ups in Asia that failed in less than three years, the average height of all female CFA exam candidates, mean weight of all U.S. taxpayers over 30 years of age, among others. Most times, statisticians and researchers want to know the characteristics of every entity in a population, so as to draw the most precise conclusion possible. This is impossible or impractical most times, however, since population sets tend to be quite large. For example, if a company wanted to know whether each of its 50,000 customers serviced during the year was satisfied, it might be challenging, costly and impractical to call each of the clients on the phone to conduct a survey. Since the characteristics of every individual in 127 CU IDOL SELF LEARNING MATERIAL (SLM)

a population cannot be measured due to constraints of time, resources, and accessibility, a sample of the population is taken. Population Samples A sample is a random selection of members of a population. It is a smaller group drawn from the population that has the characteristics of the entire population. The observations and conclusions made against the sample data are attributed to the population. The information obtained from the statistical sample allows statisticians to develop hypotheses about the larger population. In statistical equations, population is usually denoted with an uppercase N while the sample is usually denoted with a lowercase n. Population Parameters A parameter is data based on an entire population. Statistics such as averages and standard deviations, when taken from populations, are referred to as population parameters. The population mean and population standard deviation are represented by the Greek letters µ and σ, respectively. The standard deviation is the variation in the population inferred from the variation in the sample. When the standard deviation is divided by the square root of the number of observations in the sample, the result is referred to as the standard error of the mean. While a parameter is a characteristic of a population, a statistic is a characteristic of a sample. Inferential statistics enables you to make an educated guess about a population parameter based on a statistic computed from a sample randomly drawn from that population. Real World Example of Population For example, let's say a denim apparel manufacturer wants to check the quality of the stitching on its blue jeans before shipping them off to retail stores. It is not cost effective to examine every single pair of blue jeans the manufacturer produces (the population). Instead, the manufacturer looks at just 50 pairs (a sample) to draw a conclusion about whether the entire population is likely to have been stitched correctly. Sub population A subset of a population that shares one or more additional properties is called a sub 128 CU IDOL SELF LEARNING MATERIAL (SLM)

population. For example, if the population is all Egyptian people, a sub population is all Egyptian males; if the population is all pharmacies in the world, a sub population is all pharmacies in Egypt. By contrast, a sample is a subset of a population that is not chosen to share any additional property. Descriptive statistics may yield different results for different sub populations. For instance, a particular medicine may have different effects on different sub populations, and these effects may be obscured or dismissed if such special sub populations are not identified and examined in isolation. Similarly, one can often estimate parameters more accurately if one separates out sub populations: the distribution of heights among people is better modeled by considering men and women as separate sub populations, for instance. Populations consisting of sub populations can be modeled by mixture models, which combine the distributions within sub populations into an overall population distribution. Even if sub populations are well-modeled by given simple models, the overall population may be poorly fit by a given simple model – poor fit may be evidence for the existence of sub populations. For example, given two equal sub populations, both normally distributed, if they have the same standard deviation but different means, the overall distribution will exhibit low kurtosis relative to a single normal distribution – the means of the sub populations fall on the shoulders of the overall distribution. If sufficiently separated, these form a bimodal distribution; otherwise, it simply has a wide peak. Further, it will exhibit [overdispersion] relative to a single normal distribution with the given variation. Alternatively, given two sub populations with the same mean but different standard deviations, the overall population will exhibit high kurtosis, with a sharper peak and heavier tails (and correspondingly shallower shoulders) than a single distribution. SAMPLE A \"sample\" is a miniature representation of and selected from a larger group or aggregate. In other words, the sample provides a specimen picture of a larger whole. This larger whole is termed as the \"population\" or \"universe\". In research, this term is used in a broader sense; it is a well-defined group that may consist of individuals, objects, characteristics of human beings, or even the behavior of inanimate objects, such as, the throw of a dice or the tossing of a coin. Basically there are two requirements of a sample: it has to be 'representative' and 'adequate'. If the nature of the population has to be interpreted from a sample, it is necessary for the sample to be truly representative of the population. Moreover, it calls for drawing a representative 'proportion' of the population. The population may contain a finite number of 129 CU IDOL SELF LEARNING MATERIAL (SLM)

members or units. Sometimes, the population may be 'infinite' as in the case of air pressure at various points in the atmosphere. Therefore, a population has to be defined clearly so that there is no ambiguity as to whether a given unit belongs to the population or not. Otherwise, a researcher will not know what units to consider for selecting a sample. For example, we want to learn the infant feeding habits of mothers. Here, the population is not well defined: we are not told about the background of the mothers that have to be included in this survey. After all, may be mothers from rural, urban, slum background belonging to different socio- economic groups adopting different practices. Hence, to define it accurately, we have to specify the group as, say, urban slum mothers etc. This would be the sampling unit. The unit of inquiv is the subject on which the information is obtained. In other words, the sampling unit is one which is used for selection. For example, in a community survey on stunting among children, the sampling unit could be a family but the unit of enquiry could be a child less than 5 years of age. Note, one sampling unit can have one or two units of enquiry. Meaning of Sampling According to Levin and Rubin, statisticians use the word, population, to refer not only to people, but, to all items that have been chosen for study. They use the word, sample, to describe a portion chosen from the population. According to Croach and Housden, a sample is a limited number taken from a large group for testing and analysis, on the assumption that the sample can be taken as representative for the whole group. Measurement and Sampling According to Boyce, sampling makes an estimate about some of the characteristics of a population. To sample is to make a judgment or a decision about something after experiencing just part of it. Concepts in Sampling For clarity and brevity, some concepts and preliminaries of sampling theory. • Sampling Units and Population: a unit may be taken as a well-defined and identifiable element or a group of elements on which observations can be made. The aggregate of these units is termed as population and the population is said to be finite, if the units are countable. The population is sub-divided into suitable small units known as sampling units for the purpose of sampling. Sampling units may consist of one or more elementary units and each elementary unit belongs to one and one sampling unit. • Sampling Frame: a sampling frame is a list of sampling units with identification particulars indicating the location of the sampling units. A sampling frame represents the 130 CU IDOL SELF LEARNING MATERIAL (SLM)

population under investigation, and it is the base of drawing a sample. As far as possible, it should be up-to–date, i.e., free from omissions and duplications. • Sample: a fraction of the population is said to constitute a sample. The number of units included in the sample is known as the size of the sample. • Sampling Fraction: the ratio of the sample size, n, to the population size, N, is known as sampling fraction and it is denoted by (n / N). •Sampling Procedure/Method: this is the method of selecting a sample from a population. • Census: this denotes all the elements or units of a population which are used to explain the features of population. It usually refers to complete enumeration of all persons in the population. • Population Parameter and Sample Estimator: any function of the values of units in the population, such as population mean or population variance, is termed a population parameter. There can only be one set of values for a population and the population values are treated as constant. However, the function of the values of the units in the sample, such as sample mean and sample variance, is known as a statistic. The value of the mean and variance differ from sample to sample and, therefore, it is a random variable. Advantages of Sampling Some of the key advantages of sampling are: i) it costs less ii) takes less time iii) data are acquired quickly iv) fewer mistakes are likely v) a more detailed study can be done. SAMPLING FRAME In statistics, a sampling frame is the source material or device from which a sample is drawn. It is a list of all those within a population who can be sampled, and may include individuals, households or institutions. Importance of the sampling frame is stressed by Jessen and Salant and Dillman. In many practical situations the frame is a matter of choice to the survey planner, and sometimes a critical one. [...] Some very worthwhile investigations are not undertaken at all because of the lack of an apparent frame; others, because of faulty frames, have ended in a disaster or in cloud of doubt.— Raymond James Jessen 131 CU IDOL SELF LEARNING MATERIAL (SLM)

Obtaining and organizing a sampling frame In the most straightforward cases, such as when dealing with a batch of material from a production run, or using a census, it is possible to identify and measure every single item in the population and to include any one of them in our sample; this is known as direct element sampling. However, in many other cases this is not possible; either because it is cost- prohibitive (reaching every citizen of a country) or impossible (reaching all humans alive). Having established the frame, there are a number of ways for organizing it to improve efficiency and effectiveness. It's at this stage that the researcher should decide whether the sample is in fact to be the whole population and would therefore be a census. This list should also facilitate access to the selected sampling units. A frame may also provide additional 'auxiliary information' about its elements; when this information is related to variables or groups of interest, it may be used to improve survey design. While not necessary for simple sampling, a sampling frame used for more advanced sample techniques, such as stratified sampling, may contain additional information (such as demographic information). For instance, an electoral register might include name and sex; this information can be used to ensure that a sample taken from that frame covers all demographic categories of interest. (Sometimes the auxiliary information is less explicit; for instance, a telephone number may provide some information about location. Sampling frame qualities An ideal sampling frame will have the following qualities: • all units have a logical, numerical identifier • all units can be found – their contact information, map location or other relevant information is present • the frame is organized in a logical, systematic fashion • the frame has additional information about the units that allow the use of more advanced sampling frames every element of the population of interest is present in the frame every element of the population is present only once in the frame no elements from outside the population of interest are present in the frame the data is 'up-to-date' Types of sampling frames The most straightforward type of frame is a list of elements of the population (preferably the entire population) with appropriate contact information. For example, in an opinion poll, 132 CU IDOL SELF LEARNING MATERIAL (SLM)

possible sampling frames include an electoral register or a telephone directory. Other sampling frames can include employment records, school class lists, patient files in a hospital, organizations listed in a thematic database, and so on. On a more practical levels, and sampling frames have the form of computer files. Not all frames explicitly list population elements; some list only 'clusters'. For example, a street map can be used as a frame for a door-to-door survey; although it doesn't show individual houses, we can select streets from the map and then select houses on those streets. This offers some advantages: such a frame would include people who have recently moved and are not yet on the list frames discussed above, and it may be easier to use because it doesn't require storing data for every unit in the population, only for a smaller number of clusters. Sampling frames problems The sampling frame must be representative of the population and this is a question outside the scope of statistical theory demanding the judgment of experts in the particular subject matter being studied. All the above frames omit some people who will vote at the next election and contain some people who will not; some frames will contain multiple records for the same person. People not in the frame have no prospect of being sampled. Because a cluster-based frame contains less information about the population, it may place constraints on the sample design, possibly requiring the use of less efficient sampling methods and/or making it harder to interpret the resulting data. Statistical theory tells us about the uncertainties in extrapolating from a sample to the frame. It should be expected that sample frames, will always contain some mistakes. In some cases, this may lead to sampling bias. Such bias should be minimized, and identified, although avoiding it completely in a real world is nearly impossible. One should also not assume that sources which claim to be unbiased and representative are such. In defining the frame, practical, economic, ethical, and technical issues need to be addressed. The need to obtain timely results may prevent extending the frame far into the future. The difficulties can be extreme when the population and frame are disjoint. This is a particular problem in forecasting where inferences about the future are made from historical data. In fact, in 1703, when Jacob Bernoulli proposed to Gottfried Leibniz the possibility of using historical mortality data to predict the probability of early death of a living man, Gottfried Leibniz recognized the problem in replying: Nature has established patterns originating in the return of events but only for the most part. 133 CU IDOL SELF LEARNING MATERIAL (SLM)

New illnesses flood the human race, so that no matter how many experiments you have done on corpses, you have not thereby imposed a limit on the nature of events so that in the future they could not vary.— Gottfried Leibniz Leslie Kish posited four basic problems of sampling frames: • Missing elements: Some members of the population are not included in the frame. • Foreign elements: The non-members of the population are included in the frame. • Duplicate entries: A member of the population is surveyed more than once. • Groups or clusters: The frame lists clusters instead of individuals. • Problems like those listed can be identified by the use of pre-survey tests and pilot studies. SAMPLING ERROR In statistics, sampling errors are incurred when the statistical characteristics of a population are estimated from a subset, or sample, of that population. Since the sample does not include all members of the population, statistics on the sample, such as means and quartiles, generally differ from the characteristics of the entire population, which are known as parameters. For example, if one measures the height of a thousand individuals from a country of one million, the average height of the thousand is typically not the same as the average height of all one million people in the country. Since sampling is typically done to determine the characteristics of a whole population, the difference between the sample and population values is considered an error. Exact measurement of sampling error is generally not feasible since the true population values are unknown. 8.5.1 Random sampling In statistics, sampling error is the error caused by observing a sample instead of the whole population. The sampling error is the difference between a sample statistic used to estimate a population parameter and the actual but unknown value of the parameter. An estimate of a quantity of interest, such as an average or percentage, will generally be subject to sample-to- sample variation. These variations in the possible sample values of a statistic can theoretically be expressed as sampling errors, although in practice the exact sampling error is typically unknown. Sampling error also refers more broadly to this phenomenon of random sampling variation. Random sampling, and its derived terms such as sampling error, simply specific procedures for gathering and analyzing data that are rigorously applied as a method for arriving at 134 CU IDOL SELF LEARNING MATERIAL (SLM)

results considered representative of a given population as a whole. Despite a common mislearning, \"random\" does not mean the same thing as \"chance\" as this idea is often used in describing situations of uncertainty, nor is it the same as projections based on an assessed probability or frequency. Sampling always refers to a procedure of gathering data from a small aggregation of individuals that is purportedly representative of a larger grouping which must in principle be capable of being measured as a totality. Random sampling is used precisely to ensure a truly representative sample from which to draw conclusions, in which the same results would be arrived at if one had included the entirety of the population instead. Random sampling (and sampling error) can only be used to gather information about a single defined point in time. If additional data is gathered (other things remaining constant) then comparison across time periods may be possible. However, this comparison is distinct from any sampling itself. As a method for gathering data within the field of statistics, random sampling is recognized as clearly distinct from the causal process that one is trying to measure. The conducting of research itself may lead to certain outcomes affecting the researched group, but this effect is not what is called sampling error. Sampling error always refers to the recognized limitations of any supposedly representative sample population in reflecting the larger totality, and the error refers only to the discrepancy that may result from judging the whole on the basis of a much smaller number. This is only an \"error\" in the sense that it would automatically be corrected if the totality were itself assessed. The term has no real meaning outside of statistics. According to a differing view, a potential example of a sampling error in evolution is genetic drift; a change in a population’s allele frequencies due to chance. For example, the bottleneck effect; when natural disasters dramatically reduce the size of a population resulting in a small population that may or may not fairly represent the original population. What may make the bottleneck effect a sampling error is that certain alleles, due to natural disaster, are more common while others may disappear completely, making it a potential sampling error. Another example of genetic drift that is a potential sampling error is the founder effect. The founder effect is when a few individuals from a larger population settle a new isolated area. In this instance, there are only a few individuals with little gene variety, making it a potential sampling error. The likely size of the sampling error can generally be controlled by taking a large enough random sample from the population, although the cost of doing this may be prohibitive; see sample size determination and statistical power for more detail. If the observations are collected from a random sample, statistical theory provides probabilistic estimates of the likely size of the sampling error for a particular statistic or estimator. These are often expressed in terms of its standard error. 135 CU IDOL SELF LEARNING MATERIAL (SLM)

Bias problems Sampling bias is a possible source of sampling errors, wherein the sample is chosen in a way that makes some individuals less likely to be included in the sample than others. It leads to sampling errors which either have a prevalence to be positive or negative. Such errors can be considered to be systematic errors. Non-sampling error Sampling error can be contrasted with non-sampling error. Non-sampling error is a catch-all term for the deviations from the true value that are not a function of the sample chosen, including various systematic errors and any random errors that are not due to sampling. Non- sampling errors are much harder to quantify than sampling error. SAMPLE SIZE A sample size is a part of the population chosen for a survey or experiment. For example, you might take a survey of dog owner’s brand preferences. You won’t want to survey all the millions of dog owners in the country (either because it’s too expensive or time consuming), so you take a sample size. That may be several thousand owners. The sample size is a representation of all dog owner’s brand preferences. If you choose your sample wisely, it will be a good representation. When Error can Creep in When you only survey a small sample of the population, uncertainty creeps in to your statistics. If you can only survey a certain percentage of the true population, you can never be 100% sure that your statistics are a complete and accurate representation of the population. This uncertainty is called sampling error and is usually measured by a confidence interval. For example, you might state that your results are at a 90% confidence level. That means if you were to repeat your survey over and over, 90% of the time you would get the same results. A sample is a percentage of the total population in statistics. You can use the data from a sample to make inferences about a population as a whole. For example, the standard deviation of a sample can be used to approximate the standard deviation of a population. Finding a sample size can be one of the most challenging tasks in statistics and depends upon many factors including the size of your original population. General Tips 136 CU IDOL SELF LEARNING MATERIAL (SLM)

Step 1: Conduct a census if you have a small population. A “small” population will depend on your budget and time constraints. For example, it may take a day to take a census of a student body at a small private university of 1,000 students but you may not have the time to survey 10,000 students at a large state university. Step 2: Use a sample size from a similar study. Chances are, your type of study has already been undertaken by someone else. You’ll need access to academic databases to search for a study (usually your school or college will have access). A pitfall: you’ll be relying on someone else correctly calculating the sample size. Any errors they have made in their calculations will transfer over to your study. Step 3: Use a table to find your sample size. If you have a fairly generic study, then there is probably a table for it. For example, if you have a clinical study, you may be able to use a table published in Machin et. al’s Sample Size Tables for Clinical Studies, Third Edition. Step 4: Use a sample size calculator, like this one. Step 5: Use a formula. There are many different formulas you can use, depending on what you know (or don’t know) about your population. If you know some parameters about your population (like a known standard deviation), you can use the techniques below. Cochran’s Sample Size Formula The Cochran formula allows you to calculate an ideal sample size given a desired level of precision, desired confidence level, and the estimated proportion of the attribute present in the population. Cochran’s formula is considered especially appropriate in situations with large populations. A sample of any given size provides more information about a smaller population than a larger one, so there’s a ‘correction’ through which the number given by Cochran’s formula can be reduced if the whole population is relatively small. The Cochran formula is: Where: • e is the desired level of precision (i.e. the margin of error), • p is the (estimated) proportion of the population which has the attribute in question, • q is 1 – p. 137 CU IDOL SELF LEARNING MATERIAL (SLM)

The z-value is found in a Z table. Cochran’s Formula Example Suppose we are doing a study on the inhabitants of a large town, and want to find out how many households serve breakfast in the mornings. We don’t have much information on the subject to begin with, so we’re going to assume that half of the families serve breakfast: this gives us maximum variability. So p = 0.5. Now let’s say we want 95% confidence, and at least 5 percent—plus or minus—precision. A 95 % confidence level gives us Z values of 1.96, per the normal tables, so we get ((1.96)2 (0.5) (0.5)) / (0.05)2 = 385. So a random sample of 385 households in our target population should be enough to give us the confidence levels we need. Modification for the Cochran Formula for Sample Size Calculation in Smaller Populations If the population we’re studying is small, we can modify the sample size we calculated in the above formula by using this equation: Here n0 is Cochran’s sample size recommendation, N is the population size, and n is the new, adjusted sample size. In our earlier example, if there were just 1000 households in the target population, we would calculate 385 / (1+( 384 / 1000)) = 278 So for this smaller population, all we need are 278 households in our sample; a substantially smaller sample size. NON RESPONSE What is non-response? BA lot of things can go wrong in a survey. One of the most important problems is non- response. It is the phenomenon that the required information is not obtained from the 138 CU IDOL SELF LEARNING MATERIAL (SLM)

persons selected in the sample. The consequences of non-response One effect of non-response is that is reduces the sample size. This does not lead to wrong conclusions. Due to the smaller sample size, the precision of estimators will be smaller. The margins of error will be larger. A more serious effect of non-response is that it can be selective. This occurs if, due to non- response, specific groups are under- or over-represented in the survey. If these groups behave differently with respect to the survey variables, this causes estimators to be biased. To say it in other word: estimates are significantly too high or too low. Causes of non-response Non-response can have different causes. It is a good idea to distinguish these various types of non-response. Research has shown that different types of non-response may have different effects on estimators. The first step in getting the participation of a sample person in a survey is to make contact. If this is not possible, you have non-response due to no-contact. No-contact Refusal Not-able If it is possible to make contact with a person, you can establish whether he or she belongs to the target population of the survey. If not, you can discard this case. You can ignore this person, because it is case of over-coverage. If a person belongs to the target population, you have to persuade him to co-operate. If this is not successful, you have a case of non-response due to refusal. Even if there is contact, and the person wants co-operate, there can still be circumstances 139 CU IDOL SELF LEARNING MATERIAL (SLM)

preventing obtaining answers to the questions. Examples are illness or language problems. This is non-response due to not-able. If selected persons belong to the target population, can be contacted, are prepared to participate, and are able to participate, then you have response. Demonstrate: het effect van non-response There will be general elections in the country of Samplonia. The National Elderly Party (NEP) seems to do well in the campaign. An opinion poll is carried out to estimate the percentage of voters this party will attract. To determine how precise the estimator is, sample selection is repeated a large number of times. The percentage of voters is computed for each sample. The distribution of all these estimates is shown in a histogram. The average of all estimates is computed. The estimators is unbiased if this average is (approximately) equal to the true population percentage (25.4%). p>To carry out a simulation, you first set the sample size. You do that by clicking on the green square adjacent to Sample size. There are three possible sample sizes: 200, 400 or 800. You can choose to generate non-response in the survey. You do that by clicking on the green square below Non-response. The probability of non-response increases with age in this demonstration. For young people, this probability is equal to 80%, for middle-aged people it is 50%, and for elderly, the probability of non-response is 20%. You start the simulation by clicking on Start. If there is no non-response, the estimates will be neatly concentrated around the true percentage of voters in the population (25.4%). If there is non-response, the estimates will be significantly too low. Why are to estimates too low? The reason is the elderly are under-represented in the samples, because non-response is highest among them. It is the elderly who vote for NEP. So, there will be too few NEP-voters in the samples. Note that non-response causes the variation of the estimates to increase. This is also a typical non-response effect. Non-response reduces the sample size, and therefore increases the variance of estimators, leading to larger margins of error. CHARACTERISTICS OF GOOD SAMPLE 140 CU IDOL SELF LEARNING MATERIAL (SLM)

A good sample should have the characteristics of (i) Representativeness and (ii) Adequacy, as already described earlier in the unit. It is essential that the sample should be 'representative' of the population if the information from the sample is to be generalized for that population. The term representative sample means an ideal 'miniature' or 'replica' of the population from which it has been drawn. A good sample should also be 'adequate' or of sufficient size to allow confidence in the stability of its characteristics. An adequate sample is considered to be one that contains enough cases to ensure reliable results. Hence, planning the size of the sample in advance is very important. It varies with the nature of the characteristics under study and its distribution. It may be mentioned that representativeness and adequacy do not automatically ensure accuracy of results. The sampling and data collection techniques need to be selected and employed carefully to obtain higher degrees of precision in results and generalizations about the population. SUMMARY You probably think of research as something very abstract and complicated. It can be, but you’ll see (I hope) that if you learn the different parts or phases of a research project and how these fit together, it’s not nearly as complicated as it may seem at first glance. A research project has a well-known structure – a beginning, middle and end. We introduce the basic phases of a research project in The Structure of Research. In that section, we also introduce some important distinctions in research: the different types of questions you can ask in a research project; and, the major components or parts of a research project. Before the modern idea of research emerged, we had a term for what philosophers used to call research – logical reasoning. So, it should come as no surprise that some of the basic distinctions in logic have carried over into contemporary research. In Systems of Logic we discuss how two major logical systems, the inductive and deductive methods of reasoning, are related to modern research. KEY WORDS/ABBREVIATIONS • Population: a population is any group of individuals or units that have one or more characteristics in common and are of interest to the researcher. It may consist of all the units or individuals of a particular type or a more restricted part of that group. • Probability is the ratio of the number of ways in which a favored way can occur to the total number of ways the event can occur. It may range from zero, when there is no chance whatever, of the favored event, to 1.0, where there is absolute certainty 141 CU IDOL SELF LEARNING MATERIAL (SLM)

that nothing else could happen. • Probability Sampling: in probability sampling, the units of a population are not selected at the discretion of the researcher but by means of certain procedures which ensure that every unit of the population has one fixed probability of being included in the sample. • Non-probability Sampling: in non-probability sampling, the units are selected at the discretion of the researcher. The researcher uses higher judgment or experience while selecting the sample. • Sampling Frame: a complete, accurate, and up-to-date list of all the units in a population is called a sampling frame. LEARNING ACTIVITY 1. Derive the uses and utilities of Cochran’s Sample Size Formula 2. Draw a report on various causes of non-response UNIT END QUESTIONS (MCQ AND DESCRIPTIVE) A. Descriptive Types Questions 1. Explain Statistical Population in detail. 2. Define Sampling Frame. 3. What is Sampling Error? State its types. 4. Discuss advantages and disadvantages of Non-Response 5. Discuss the Characteristics of a good sample? B. Multiple Choice Questions 1. Which of the following is NOT a characteristic of an untestable hypothesis? a. Circularity 142 CU IDOL SELF LEARNING MATERIAL (SLM)

b. Appeal to unscientific notions. c. Inadequate definition of concepts. d. Non-directionality. 2. Event A can be inferred to cause event B if… a. If A doesn’t happen, B doesn’t happen either. b. Whenever A happens, B happens. c. A happens before B. d. All of these 3. The definition of a psychological construct such as ‘love’ in such a way as to allow measurement of it is known as… a. Conceptualization. b. An operational definition. c. Scale of measurement. d. Hypothesizing 4. The order in which participants complete a task is an example of what level of measurement? a. Interval b. Nominal c. Ratio d. Ordinal 143 CU IDOL SELF LEARNING MATERIAL (SLM)

5. What level of measurement would be used if participants were asked to choose their favorite picture from a set of six? a. Nominal b. Ordinal c. Ratio d. Interval Answer 1. d 2. d 3. b 4. d 5. a REFERENCES • Dalen, Van. Deobold, & Meyer, William. J. (1962): Learning Educational ~Research: An Introduction. New York: McGraw Hill Book Company Inc. Kerlinger, Fred N. (1993): • Foundation of Behavioral Research: Educational and Psychological Enquiry.New York: Holt, Rinehart and Winston. Koul, Lokesh. (1997): • Methodology of Educational Research. New Delhi: Vikas Publishing House Pvt.Ltd. Upasani, K.N. (1987): • Conducting Educational Research. Examination Reforms Unit, S.N.D.T. Women's University, Pune: Kalpana Mudranalaya, Vockell, Edward L. (1983): • Educational Research. New York: MacMillan Publishing Co. Inc 144 CU IDOL SELF LEARNING MATERIAL (SLM)

UNIT 9: PROBABILITY SAMPLE Structure Learning Objectives Introduction Simple Random Sample Systematic Sample Stratified Random Sample, Cluster Sample & Multi-stage sampling Determining size of the sample – Practical considerations in sampling and sample size. Summary Key Words/Abbreviations Learning Activity Unit End Questions (MCQ and Descriptive) References LEARNING OBJECTIVES After studying this unit, you will be able to: • State definition of Simple random sample • Explain Systematic sample • Describe Stratified random sample • Discuss Cluster sample and Multi stage sampling INTRODUCTION Probability sampling is defined as a sampling technique in which the researcher chooses samples from a larger population using a method based on the theory of probability. For a 145 CU IDOL SELF LEARNING MATERIAL (SLM)

participant to be considered as a probability sample, he/she must be selected using a random selection. What are the steps involved in probability sampling? Follow these steps to conduct probability sampling: 1. Choose your population of interest carefully: Carefully think and choose from the population, people you believe whose opinions should be collected and then include them in the sample. 2. Determine a suitable sample frame: Your frame should consist of a sample from your population of interest and no one from outside to collect accurate data. 3. Select your sample and start your survey: It can sometimes be challenging to find the right sample and determine a suitable sample frame. Even if all factors are in your favor, there still might be unforeseen issues like cost factor, quality of respondents, and quickness to respond. Getting a sample to respond to a probability survey accurately might be difficult but not impossible. But, in most cases, drawing a probability sample will save you time, money, and a lot of frustration. You probably can’t send surveys to everyone, but you can always give everyone a chance to participate, this is what probability sample is all about. When to use probability sampling? Use probability sampling in these instances: 1. When you want to reduce the sampling bias: This sampling method is used when the bias has to be minimum. The selection of the sample largely determines the quality of the research’s inference. How researchers select their sample largely determines the quality of a researcher’s findings. Probability sampling leads to higher quality findings because it provides an unbiased representation of the population. 2. When the population is usually diverse: Researchers use this method extensively as it helps them create samples that fully represent the population. Say we want to find out how many people prefer medical tourism over getting treated in their own country. This sampling method will help pick samples from various socio-economic strata, background, etc. to 146 CU IDOL SELF LEARNING MATERIAL (SLM)

represent the broader population. 3. To create an accurate sample: Probability sampling help researchers create accurate samples of their population. Researchers use proven statistical methods to draw a precise sample size to obtained well-defined data. Advantages of probability sampling Here are the advantages of probability sampling: 1. It’s Cost-effective: This process is both cost and time effective, and a larger sample can also be chosen based on numbers assigned to the samples and then choosing random numbers from the more significant sample. 2. It’s simple and straightforward: Probability sampling is an easy way of sampling as it does not involve a complicated process. It’s quick and saves time. The time saved can thus be used to analyze the data and draw conclusions. 3. It is non-technical: This method of sampling doesn’t require any technical knowledge because of its simplicity. It doesn’t require intricate expertise and is not at all lengthy. SIMPLE RANDOM SAMPLE What is a Simple Random Sample? A simple random sample is a subset of a statistical population in which each member of the subset has an equal probability of being chosen. A simple random sample is meant to be an unbiased representation of a group. An example of a simple random sample would be the names of 25 employees being chosen out of a hat from a company of 250 employees. In this case, the population is all 250 employees, and the sample is random because each employee has an equal chance of being chosen. Random sampling is used in science to conduct randomized control tests or for blinded experiments. Researchers can create a simple random sample using a couple of methods. With a lottery method, each member of the population is assigned a number, after which numbers are 147 CU IDOL SELF LEARNING MATERIAL (SLM)

selected at random. The example in which the names of 25 employees out of 250 are chosen out of a hat is an example of the lottery method at work. Each of the 250 employees would be assigned a number between 1 and 250, after which 25 of those numbers would be chosen at random. Because individuals who make up the subset of the larger group are chosen at random, each individual in the large population set has the same probability of being selected. This creates, in most cases, a balanced subset that carries the greatest potential for representing the larger group as a whole, free from any bias. For larger populations, a manual lottery method can be quite onerous. Selecting a random sample from a large population usually requires a computer-generated process, by which the same methodology as the lottery method is used, only the number assignments and subsequent selections are performed by computers, not humans. Room for Error With a simple random sample, there has to be room for error represented by a plus and minus variance (sampling error). For example, if in a high school of 1,000 students a survey were to be taken to determine how many students are left-handed, a random sampling can determine that eight out of the 100 sampled are left-handed. The conclusion would be that 8% of the student population of the high school are left-handed, when in fact the global average would be closer to 10%. The same is true regardless of subject matter. A survey on the percentage of the student population that has green eyes or is physically incapacitated would result in a mathematical probability based on a simple random survey, but always with a plus or minus variance. The only way to have a 100% accuracy rate would be to survey all 1,000 students which, while possible, would be impractical. Advantages of Simple Random Samples 1. Lack of Bias Because individuals who make up the subset of the larger group are chosen at random, each individual in the large population set has the same probability of being selected. This creates, in most cases, a balanced subset that carries the greatest potential for representing 148 CU IDOL SELF LEARNING MATERIAL (SLM)

the larger group as a whole. 2. Simplicity As its name implies, producing a simple random sample is much less complicated than other methods, such as stratified random sampling. As mentioned, individuals in the subset are selected randomly and there are no additional steps. To ensure bias does not occur, researchers must acquire responses from an adequate number of respondents, which may not be possible due to time or budget constraints. Disadvantages of Simple Random Samples 1. Difficulty Accessing Lists of the Full Population In simple random sampling, an accurate statistical measure of a large population can only be obtained when a full list of the entire population to be studied is available. In some instances, details on a population of students at a university or a group of employees at a specific company are accessible through the organization that connects each population. 2. Time Consuming When a full list of a larger population is not available, individuals attempting to conduct simple random sampling must gather information from other sources. If publicly available, smaller subset lists can be used to recreate a full list of a larger population, but this strategy takes time to complete. Organizations that keep data on students, employees, and individual consumers often impose lengthy retrieval processes that can stall a researcher's ability to obtain the most accurate information on the entire population set. 3. Costs In addition to the time it takes to gather information from various sources, the process may cost a company or individual a substantial amount of capital. Retrieving a full list of a population or smaller subset lists from a third-party data provider may require payment each time data is provided. If the sample is not large enough to represent the views of the entire population during the first round of simple random sampling, purchasing additional lists or databases to avoid a sampling error can be prohibitive. Sample Selection Bias 149 CU IDOL SELF LEARNING MATERIAL (SLM)