University of Toronto
Faculty of Management
Fall 2000
3051 & 3062 Research Methodology
Instructors
Martin G. Evans
Office #532
978-1949
|
Andrew A. Mitchell
Office #501
978-4915
|
Scott Hawkins
Office #506
978-4196
|
Class Time: Thursday 1.00 - 4.00
Place: Room 500
This Methodology seminar is designed for Organizational
Behaviour and Marketing Ph.D. students and will meet for two
semesters as MGT 3051 in the Fall and MGT 3062 in the spring [probably 2001].
During the fall semester, the course will focus primarily on
measurement and experimental methods, and in the spring semester
the focus will be on the analysis of correlational data.
The course focuses on the identification and testing of
relationships between theoretical variables. Consequently, it
covers measurement of theoretical variables as well as the use of
experimentation and correlational methods to identify and test
relationships between theoretical variables.
There are three main objectives in this course. The first
objective is to provide students with an understanding of the
methodological concepts required to conduct empirical research.
The second is to develop the ability of students to use these
concepts in critically evaluating research. The third is to
develop students' skills in using these concepts to design their
own research projects.
Important web sites:
- Here
Is Bill Trochim's
[http://trochim.human.cornell.edu/kb/index.htm] on-line
Textbook. You will probably want to refer to this
throughout the course.
- Academy of Management Research Methods Division:
http://www.aom.pace.edu/rmd Click here from this web
page .
- Strutural Equations Modeling Network (SEMNET):
http://www.gsu.edu/~mkteer/semnet.html. Click here from
this web page.
- American Marketing Association: http://www.ama.org.
Click here to reach this
page. Links there will take you to their publications,
etc.
- The Association for Consumer Research:
http://acr.webpage.com. Click here to reach this
site.
- Society for Consumer Psychology (Division 23 of the
American Psychological Association:
http://www.cob.ohio-state.edu/scp. Click here to
reach this site.
- Society for Industrial and Organizational Psychology
(Division 14 of the American Psychological Association):
http://www.siop.org. Click here
to reach this site.
Web Sites to help you write better papers:
Since the course is designed primarily to provide students
with an understanding of important methodological concepts and
the ability to apply them, it will not cover the statistical
theory used in experimental or correlational analysis. Students
are urged to take additional courses in psychology, econometrics
and statistics that will provide such a background.
The following course requirements will be used to achieve the
objectives outlined previously:
- Extensive readings from psychology, marketing and
organizational behaviour on methodological concepts. Make
sure that you understand the concepts covered in the
readings and think about how they would be applied in
different research situations. All of
the assigned readings are to be read and
thought about prior to class.
- A combination of lecture and rigorous class discussions.
You are expected to contribute to every class through
your questions and discussions.
- Evaluation for the Fall semester course will be based
upon three written assignments:
- Article Critique You will be
provided with an article in Week 9. The
assignment is to write a critical analysis of the
article in terms of its methodological strengths
and weaknesses. The written critique is due prior
to class in Week 11. 30%
- Research Design Project You will
choose a research topic of interest and develop a
hypothesis or hypotheses. The project then
involves designing an experimental or
quasi-experimental study to test the hypothesis
or hypotheses. A brief description of the topic
and hypotheses to be tested is due prior
to class in Week 12. The final written project is
due at the end of the first week of January,
2000. 50%
- Final Examination. Held the last
day of classes. 20%
The topics to be covered in each of the class sessions are as
follows:
TOPIC
|
DATE
|
| 1. Introductory Session |
September 14 |
| 2. Philosophy of Science |
September 21 |
| 3. General Research Strategies |
September 28 |
| 4. Measurement I: Scaling &
Reliability |
October 5 |
| 5. Measurement II: Validity of Measures
and Scale Construction |
October 12 |
| 6. Hypotheses and Statistical Tests |
October 19. |
| 7. Experimental Design and the Concepts
of Experimental Validity |
October 26 |
| 8. Quasi-Experimental Design I:
Non-Equivalent Control Group Designs |
November 2 |
| 9. Quasi-Experimental Design II: Time
Series Designs |
November 9 |
| 10. Experimental Research I: Research
Strategies |
November 16 |
| 11. Experimental Research II: Enhancing
Internal Validity |
November 23 |
| 12. Experimental Research III: Analysis
of Variance and Covariance |
November 30 |
| 13 Experimental Research IV: Analysis of
Variance and Covariance (cont) |
December 7 |
14
a. Experimental Versus Non-Experimental Research
b. Generalizability Theory |
December 14 |
Supplementary Session
Basics of Matrix Algebra |
If Required |
| Final Examination |
Dec 21 |
Week 2
Philosophy of Science
- Thomas, L. (1981). Medicine without science. The
Atlantic Monthly, April, 40-42.
- Hempel, Carl G. (1966). Philosophy of natural science.
Englewood Cliffs, NJ: Prentice Hall. Chapters 1 & 2.
- Kuhn, Thomas (1962). The Structure of Scientific
Revolutions. Chicago, Il: University of Chicago
Press. Chapters 2, 3, 6, 7, 20.
- Popper, Karl (1962). Conjecture and Refutations.
New York: Harper. Chapter 2.
- Cook, D.T. and Campbell, D.T. (1979). Quasi-experimentation:
Design & analysis issues for field settings.
Chicago: Rand McNally Publishing Co. Chapter 1.
- Schools' Brief: The philosophy of Science. The
Economist, April 25, 1987, 70-71.
Suggested (but not required) as an accessible introduction to
philosophical and ethical isses :
Woodward, J., & Goodstein, G. (1996). Conduct,
misconduct and the structure of science. American
Scientist, 84, 479-490.
There are some useful ideas on Causality at
Judea Pearl's site.
Week 3
General Research Strategies
- Davis, M.S. (1971). That's interesting! Towards a
phenomenology of sociology and a sociology of
phenomenology. Philosophy of Social Sciences, 1,
309-344.
- Mackenzie, K.D. and House, R.J. (1978). Paradigm
development in the social sciences: a proposed research
strategy. Academy of Management Review, 3,
7-24.
- Platt, J.R. (1964). Strong inference. Science,
146, 347-352.
- Greenwald, A.G. (1975). On the inconclusiveness of
'critical' cognitive tests of dissonance versus
self-perception theories. Journal of Experimental
Social Psychology, 11, 490-99.
- Greenwald, A.G., Pratkanis, A.T., Leippe, M.R., &
Baumgartner, M.H., Under what conditions does theory
obstruct research progress. Psychological Review,
1986, 93, 216-289
Comments on the above:
- McKay, D.G. (1988). Under what conditions can
theoretical psychology survive and prosper?
Integrating the rational and empirical
epistemologies. Psychological Review,
95, 559-565.
- Greenberg, J., Solomon, S., Pyszczynski, T,
& Steinberg, L. (1988). A reaction to
Greenwald, Pratkanis, Leippe, &
Baumgartner (1986): Under what conditions
does research obstruct theory progress. Psychological
Review, 95, 566-571.
- Moser, K., Gadenne, V., & Schroder, J.
(1988). Under what conditions does
confirmation seeking obstruct scientific
progress? Psychological Review, 95,
572-574.
- Greenwald, A., & Pratkanis, A.R. (1988).
On the use of 'theory' and the usefulness of
theory. Psychological Review, 95,
575-579.
- I found a summary of this debate
on the web
- Whetten, D. A., & Cameron, K.S. (1991). Organizational
behavior, an experiential approach (2nd ed., pp.
165-208). New York: Harper & Row. Skill Learning:
Becoming a more creative problem solver
Week 4
Measurement I: Scaling and Reliability
- Kerlinger, F.N. (1973). Foundations of behavioral
research, 2nd Edition. New York: Holt, Rinehart,
& Winston Chapters 25 & 26.
- Ghiselli, E.E., Campbell, J.P., & Zedeck, S. (1981). Measurement
theory for the behavioral sciences. San Francisco:
W.H. Freeman, Chapters 1, 8, & 9.
- Stine, W.W. (1989). Meaningful information: The role of
measurement in statistics. Psychological Bulletin,
105, 147-155.
- Michell, J. (1986). Measurement scales and statistics: A
clash of paradigms. Psychological Bulletin, 100,
398-407.
- Cortina, J.M. (1993). What is coefficient alpha? An
examination of theory and application. Journal of
Applied Psychology, 78, 98-104.
- Webb, E.J. and Weick, K.E. (1979). Unobtrusive measures
in organizational theory: A Reminder. Administrative
Science Quarterly, 24, 650-659.
Supplementary:
- Gaito, J. (1980). Measurement scales and statistics:
Resurgence of an old misconception. Psychological
Bulletin, 87, 564-567.
- Webb, E.J. et al. (1966). Unobtrusive measures:
Nonreactive research in the social sciences.
Chicago, Ill.: Rand Mcnally, 1966. Chapters l-6.
- Hedges, L. (1987). How hard is hard sciences, how soft is
soft science: The empirical cumulativeness of research.
American Psychologist, 42(5):443--455. Interesting
comparison of meta-analysis in physics and psychology
arguing one may not be less cumulative than the other.
Here is a Precis
of his argument.
- Nunnally, J.C. (1976). Psychometric theory (2nd
ed.). New York: McGraw-Hill. Chapters 6-7.
- Peter, J.P. (1979). Reliability: A review of the
psychometric basis and recent marketing practices. Journal
of Marketing Research, 16, 6-17.
Week 5
Measurement II: Validity of Measures and Scale Construction
- Ghiselli, E.E., Campbell, J.P., & Zedeck, S. (1981). Measurement
theory for the behavioral sciences (pp 265-278). San
Francisco: W.H. Freeman. Chapter 10.
- Kerlinger, F.N. (1973). Foundations of behavioral
research (2nd ed.). New York: Holt, Rinehart, &
Winston. Chapter 27.
- Campbell, D.T. and D.W. Fiske (1959). Convergent and
discriminant validation by the multi-trait and
multi-method matrix. Psychological Bulletin, 56,
81-105. (especially pp. 81-85 and 100-104).
- Evans, M.G., Donald T. Campbell's Methodological
contributions to Organizational Science. In J. A. C. Baum
& W. J. McKelvey (eds.) Variations in Organizational
Science: In honor of Donald T. Campbell. Thousand Oaks:
CA: Sage Publications, 311-338.
Only read, and lightly, pp 311-322; most of this material
will be of concern in MGT3062.
- Churchill, G.A. (1979). A paradigm for developing better
measures of marketing constructs. Journal of
Marketing Research, 16, 1979, 64-73.
- Feldman, J.M., & Lynch, J.G. (1988). Self-generated
validity and other effects of measurement on belief,
attitude, intention, and behavior. Journal of Applied
Psychology, 73, 421-435.
Supplementary:
- Peter, J.P. (1981). Construct validity: A review of basic
issues and marketing practices. Journal of Marketing,
18, 133-45.
- Tesser, A. & Krause, H. (1976). On validating a
relationship between constructs. Educational and
Psychological Measurement, 36, 111-121.
- Cronbach, L.J. and P.E. Meehl (1952). Construct validity
in psychological tests. Psychological Bulletin,
52, 281-302.
Week 6
Hypotheses and Statistical Tests
Week's
website:
[http://trochim.human.cornell.edu/OJtrial/ojhome.htm]. A Powerful
analysis of power -- fun too!
- Kirk, R.E. (1982). Experimental design (2nd
Edition, pp. 25-43). Belmont, CA: Brooks/Cole Publishing
Co.
- Bakan, D. (1966). The test of significance in
psychological research. Psychological Bulletin, 66,
423-37.
- Bolles, R.D. (1962). The difference between statistical
hypotheses and scientific hypotheses. Psychological
Reports, 11, 639-45.
- Fern, E. F., & Monroe, K. B. (1996). Effect size
estimates: Issues and problems in interpretation. Journal
of Consumer Research, 23, 89-105.
- Rosenthal, R. (1979). The file drawer problem and
tolerance for null results. Psychological Bulletin,
86, 638-41.
- Tversky, A., & Kahnemann, D. (1971). Belief in the
law of small numbers. Psychological Bulletin, 2,
105-110.
- Sedlemeir, P., & Gigerenzer, G. (1989). Do studies of
statistical power have an effect on the power of studies?
Psychological Bulletin, 105, 309-316.
- Cohen, J. (1992)., A power primer. Psychological
Bulletin, 112, 155-159.
- Rosnow, R.L., & Rosenthal, R. (1989). Statistical
procedures and the justification of knowledge in the
social sciences. American Psychologist, 44,
1276-1284.
Supplementary:
- Greenwald, A.G. (1975). Consequences of prejudice against
the null hypothesis. Psychological Bulletin, 82,
1-19.
Week 7
Experimental Design and the Concepts of Experimental Validity
- Campbell, D.T., & Stanley, J.C. (1963). Experimental
and quasi-experimental designs for research (pp.
1-34). Chicago: Rand McNally.
- Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation:
Design and analysis issues for field settings.
Chicago: Rand McNally, Chapter 2.
- Evans, M.G., Donald T. Campbell's Methodological
contributions to Organizational Science. In J. A. C. Baum
& W. J. McKelvey (eds.) Variations in Organizational
Science: In honor of Donald T. Campbell. Thousand Oaks:
CA: Sage Publications, 311-338.
Only read pp. 322-330.
- Mook, D.G. (1983). In defense of external invalidity. American
Psychologist, 38, 379-387.
- Calder, B.J., Phillips, L.W., and Tybout, A.M. (1981).
Designing research for applications. Journal of
Consumer Research, 1981, 8, 197-207.
- Lynch, J.B. Jr. (1982). On the external validity of
experiments in consumer research. Journal of Consumer
Research, 9, 225-39.
- Kardes, F. R. (1996. In defense of experimental consumer
psychology. Journal of Consumer Psychology, 5,
279-296.
Week 8
Quasi-Experimental Design I: Non-Equivalent Control Group
Designs
- Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation:
Design and analysis issues for field settings.
Chicago: Rand McNally, Chapters 3, 4, & 8.
- Evans, M.G. (1975). Opportunistic organizational
research: The role of patchup designs. Academy of
Management Journal, 18, 98-108.
- Check here for Regression
to the Mean
Supplementary:
- Campbell, D.T., & Stanley, J.C. (1963). Experimental
and quasi-experimental designs for research.
Chicago: Rand McNally, pp. 34-64.
Week 9
Quasi-Experimental Design II: Time Series Designs
- Cook, T.D., & Campbell, D.T. (1979). Quasi-experimentation:
Design and analysis issues for field settings.
Chicago: Rand McNally, Chapter 5.
- Ostrom, C.W. (1978). Time series analysis: Regression
techniques. Sage University Paper Series on
Quantitative Applications in the Social Sciences, 07-009.
Beverly Hills: Sage Publications.
- Cook, T.D., Dintzer, L., & Mark, M.M. (1980). The
causal analysis of concomitant time series. In L. Bickman
(Ed.), Applied social psychology annual: Volume I
(pp 93-135). Beverly Hills: Sage Publications, 1980.
Due: Article Critique.
Week 10
Experimental Research I: Research Strategies
- Carlsmith, J.M., Ellsworth, P.C., & Aronson, E.
(1976). Methods of research in social psychology
(pp 8-25), Reading, MA: Addison-Wesley Publishing Co.
- Aronson, E., Brewer, M., & Carlsmith, J.M. (1985).
Experimentation in social psychology. In G. Lindzey and
E. Aronson (Eds.), The handbook of social psychology
(3rd ed., pp. 441-486). New York: Random House Co.
- McGuire, W.J. (1983). A contextualist theory of
knowledge: Its implications for innovation and reform in
psychological research. In L. Berkowitz (Ed.) Advances
in experimental social psychology, (Volume 16, pp.
1-47). New York: Academic Press.
- Sternthal, B., Tybout, A.M., Calder, B.J. (1987).
Confirmatory versus comparative approaches to judging
theory tests. Journal of consumer research, 14,
114-125.
Due: Research Design Project - brief topic and hypotheses
Week 11
Experimental Research II: Enhancing Internal Validity
- Sawyer, A.G. (1974). Demand artifacts in laboratory
experiments in consumer research. Journal of Consumer
Research, 1, 20-30.
- Shimp, T.A., Hyatt, E.M., & Snyder, D.J. (1991). A
critical appraisal of demand artifacts in consumer
research. Journal of Consumer Research, 18,
273-283.
- Greenwald, A.G. (1976). Within subject designs: To use or
not to use? Psychological Bulletin, 83,
314-20.
- Himmelfarb, S. (1975). What to do when the control group
doesn't fit into the factorial design. Psychological
Bulletin, 1975, 82, 363-68.
- Lord, F.M. (1963). Elementary models for measuring
change. In C.W. Harris (Ed.), Problems in Measuring
Change (pp. 21-38). Madison, WI.: The University of
Wisconsin Press.
Week 12 & 13
Experimental Research III: Analysis of Variance and
Covariance
- Kirk, R.E. (1982). Experimental Design (2nd ed.,
pp. 8-19, 44-84, 101-106, 715-727). Belmont, CA:
Brooks/Cole Publishing Co.,
- O'Grady, K.E. (1982). Measures of explained variance:
Cautions and limitations. Psychological Bulletin,
92, 766-77.
- Rosenthal, R., & Rosnow, R.L. (1985) Contrast
analysis. New York: Cambridge University Press.
Chapters 1-3.
Recommended:
- Winer, B.J. (1971). Statistical Principles of
Experimental Design (2nd ed., pp. 351-359). New
York: McGraw-Hill.
Week 14
a) Experimental vs. Non-Experimental Research
- Cronbach, L.J. (1957). The two disciplines of scientific
psychology. American Psychologist, 12,
671-684.
- McGrath, J.E. (1982). Dilemmatics: The study of research
choices and dilemmas. In J.E. McGrath, J. Martin, and
R.A. Kulka (Eds.), Judgement calls in research.
Sage: Beverly Hills, CA.
- Cooper, W.H. & Richardson, A.J. (1986). Unfair
comparisons. Journal of Applied Psychology, 71,
179-184.
- Hirschman, E.C. (1986). Humanistic inquiry in marketing
research: Philosophy, method and criteria. Journal of
Marketing Research, 23, 237-249.
- Evans, M.G., Donald T. Campbell's Methodological
contributions to Organizational Science. In J. A. C. Baum
& W. J. McKelvey (eds.) Variations in Organizational
Science: In honor of Donald T. Campbell. Thousand Oaks:
CA: Sage Publications, 311-338.
Only read, and lightly, pp 330-334.
Recommended:
- Blalock, H.M. (1964). Causal inferences in
non-experimental research. Chapel Hill, NC:
University of North Carolina Press. Chapter l-2.
- Dipboye, R.L. and Flanagan, M.F. (1979). Research
settings in Industrial and Organizational Psychology: Are
findings really more generalizable in the field than in
the laboratory? American Psychologist, 34,
141-150.
- Sharma, S., Durand, R.M., & Gur-Arie, O. (1981).
Identification and analysis of moderator variables. Journal
of Marketing Research, 18, 291-300.
b) Generalizability Theory
- Shavelson, R.J., Webb, N.M., Rowley, G.L. (1989).
Generalizability theory. American Psychologist, 44,
922-932.
Supplementary Session
Matrix Algebra
- Johnston, J. (1972). Econometric Methods (2nd
ed., pp. 68-95.). New York: McGraw-Hill. Chapter 4.
Reminder: Research Design due January 8th, 2001. [The REAL
new millenium]