Handbook of Ethics in Quantitative Methodology

Ethics in Quantitative Research

A.T. Panter and Sonya K. Sterba

Overview

In the Handbook of Ethics in Quantitative Methodology, the authors define an ethical framework as “a structured approach to evaluating ethical issues and making ethical decisions” (Panter & Sterba, 2005, p. 13). They emphasize that ethical frameworks are not meant to provide definitive answers to ethical dilemmas but rather serve as tools to guide researchers’ thinking about ethical issues and help them make informed choices.

Key Characteristics of Ethical Frameworks: Provide a Structured Approach: Ethical frameworks offer a systematic approach to analyzing ethical issues, rather than relying on intuition or personal judgment alone.

  1. Identify Ethical Issues: Frameworks help researchers identify and clarify potential ethical concerns that may arise in their research.

  2. Weigh Competing Values: Ethical frameworks provide a means to weigh different ethical principles and values, such as beneficence, nonmaleficence, justice, and respect for persons.

  3. Make Informed Decisions: Frameworks guide researchers in making informed and defensible ethical decisions that are consistent with ethical principles.

The example of Ethical Frameworks

Utilitarianism: Focuses on maximizing overall happiness or well-being, considering the benefits and harms to all involved.

The proximity of experimental outcomes to utilitarian consequences. In medicine, trials are usually designed with some particular utilitarian outcome in mind—that is, to test whether a certain specific intervention improves health care in some particular way. By contrast, many psychological trials are designed with the purpose of improving our understanding of how the mind works, rather than whether one particular intervention improves its function. There are utilitarian ethical arguments to be made here, too, of course, but the ethical consequences of “our theory is wrong” are considerably different than the ethical implications of “our treatment doesn’t work” or “our drug causes harm.” (There are exceptions in both disciplines, of course.)

Main Ethical Issues in Quantitative research

Quantitative sociological research, like any research involving human subjects, raises a variety of ethical concerns. These concerns can be broadly categorized into three main areas:

  1. Protecting the rights and well-being of research participants:

  • Informed consent: Researchers must obtain informed consent from research participants, ensuring they understand the purpose of the study, the potential risks and benefits, and their right to withdraw at any time.

  • Confidentiality and anonymity: Researchers must protect the confidentiality of participants’ data and maintain their anonymity unless they explicitly agree to be identified.

  • Vulnerable populations: Researchers must take extra precautions when working with vulnerable populations, such as children, the elderly, or individuals with disabilities, to minimize potential harm and maximize their protection.

  1. Ensuring the integrity and rigor of research:

  • Objectivity and bias: Researchers must strive for objectivity and minimize bias in their research design, data collection, analysis, and interpretation.

  • Transparency and replicability: Researchers should make their research methods and findings transparent and replicable to allow for scrutiny and verification.

  • Falsification and fabrication: Researchers must avoid falsification or fabrication of data, ensuring the accuracy and integrity of their research findings.

  1. Social and ethical implications of research findings:

  • Beneficence and nonmaleficence: Researchers should strive to maximize the benefits of their research and minimize potential harm to individuals and society.

  • Social justice and fairness: Researchers should consider the potential social and ethical implications of their research findings, promoting equity and fairness in society.

  • Dissemination and utilization of research: Researchers should disseminate their findings responsibly and consider how their research can be used to inform policy, practice, and public understanding of social issues.

Addressing these ethical concerns is crucial for conducting responsible and ethical quantitative sociological research that contributes to the advancement of knowledge while safeguarding the rights and well-being of research participants and society as a whole.

Ethical Issues in the Conduct and Reporting of Meta-Analysis

Meta-analysis is a type of research synthesis.

It involves the statistical integration of data from separate but similar studies typically using the summary statistics presented in research reports.

Meta-analysts

(a) systematically collect as many published and unpublished reports addressing a topic as possible,

(b) extract effect sizes from the reports,

(c) statistically combine the effect sizes to obtain an estimate of the average effect size and the associated confidence interval, and

(d) examine sample and study features that might influence study outcomes.

Unlike primary researchers, they face no issues regarding the treatment of the humans or animals who participate in their work.

8.10 Reporting Research Results

(a) Psychologists do not fabricate data. (See also Standard 5.01a, Avoidance of False or

Deceptive Statements.)

(b) If psychologists discover significant errors in their published data, they take

reasonable steps to correct such errors in a correction, retraction, erratum, or other appropriate publication means.

8.11 Plagiarism

Psychologists do not present portions of another’s work or data as their own, even if the other work or data source is cited occasionally

8.12 Publication Credit

(a) Psychologists take responsibility and credit, including authorship credit, only for work they have actually performed or to which they have substantially contributed (See also Standard 8.12b, Publication Credit.)

(b) Principal authorship and other publication credits accurately reflect the relative scientific or professional contributions of the individuals involved, regardless of their relative status. Mere possession of an institutional position, such as department chair, does not justify authorship credit. Minor contributions to the research or to the writing for publications are acknowledged appropriately, such as in footnotes or in an introductory statement.

(c) Except under exceptional circumstances, a student is listed as principal author on any multiple-authored article that is substantially based on the student’s doctoral dissertation. Faculty advisors discuss publication credit with students as early as feasible and throughout the research and publication process as appropriate. (See also Standard 8.12b, Publication Credit.)

8.13 Duplicate Publication of Data

Psychologists do not publish, as original data, data that have been previously published. This does not preclude republishing data when they are accompanied by proper acknowledgment

8.14 Sharing Research Data for Verification

(a) After research results are published, psychologists do not withhold the data on which their conclusions are based from other competent professionals who seek to verify the substantive claims through reanalysis and who intend to use such data only for that purpose, provided that the confidentiality of the participants can be protected and unless legal rights concerning proprietary data preclude their release. This does not preclude psychologists from requiring that such individuals or groups be responsible for costs associated with the provision of such information.

(b) Psychologists who request data from other psychologists to verify the substantive claims through reanalysis may use shared data only for the declared purpose.

Requesting psychologists obtain prior written agreement for all other uses of the data.

APA, called meta-analysis

reporting standards (MARS; APA Publication and Communication Board Working Group on Journal Article Reporting Standards,1 2008).

Introduction

1. Clear statement of the research question 34 0 9.74

2. Narrative account of the development of the research question 8 2 6.73

3. Theoretical, policy, and/or practical issues related to the research question 9 4 7.36

4. Rationale for the selection and coding of potential moderators and mediators of results 14 2 7.62

5. Types of study designs … their strengths and weaknesses 14 3 7.80

6. Independent (predictor) and dependent (outcome) variables of primary interest 25 1 9.07

7. Populations to which the question is relevant 23 2 8.45

8. Hypotheses, if any 14 4 7.46

Methods

1. Operational definitions of independent (predictor) and dependent (outcome) variable(s)

23 2 8.51

2. Eligible participant populations 24 1 8.88

3. Eligible research design features … 28 2 8.90

4. Time period in which studies needed to be conducted 20 3 8.17

5. Geographical and/or cultural restrictions 10 4 6.69

6. Whether unpublished studies were included or excluded 24 2 8.67

7. Reference and citation databases searched 30 2 9.19

8. Registries (including prospective registries) searched 17 7 7.30

9. Keywords used to enter databases and registries 17 4 8.05

10. Search software used to enter electronic databases (e.g., Ovid) 9 18 5.57

11. Conference proceedings searched

12. Listservs queried 7 10 5.89

13. Contacts made with researchers in the fi eld and how these researchers were chosen 5 9 6.12

14. Whether reference lists of reports were examined 12 4 7.62

15. Method of addressing reports in languages other than English 14 5 7.14

16. Aspects of reports used to determine relevance (i.e., title, abstract, and/or full text) 16 7 7.57

17. Number and qualifications of relevance judges 13 10 6.81

18. Indications of judge agreement if more than one judge examined each report 15 7 7.31

19. How judge disagreements were resolved 17 4 7.90

20. Number and qualifications of coders (e.g., level of expertise in the area, training) 10 10 6.59

21. Intercoder reliability or agreement 15 7 7.43

22. Whether each report was coded by more than one coder … how disagreements resolved

19 3 8.02

23. How missing data were handled 17 2 8.1224. Definitions of ALL coding categories … 13 3 7.60

25. Criteria of the quality scale and procedure for application 20 6 7.80

26. Study design features that were coded 23 1 8.74

27. Effect size metric(s) 34 0 9.68

28. Effect sizes calculating formulas … 8 11 6.24

29. Corrections made to effect sizes … 20 1 8.34

30. Effect size averaging and weighting method(s) 31 0 9.00

31. How effect size confidence intervals (or standard errors) were calculated 17 3 8.13

32. How effect size credibility intervals were calculated 10 4 6.89

33. How studies with more than one effect size were handled 27

34. Whether fi xed and/or random effects models were used 32 2 9.31

35. The justifi cation for the choice of the error model (fi xed, random) 22 7 8.00

36. How heterogeneity in effect sizes was assessed or estimated 23 1 8.93

37. Means and SDs for measurement artifacts 2 7 5.24

38. Tests and any adjustments for data censoring (e.g., publication bias, selective reporting 14 3 7.83

39. Tests for statistical outliers 9 8 6.14

40. Statistical power of the meta-analysis 3 23 4.07

41. Statistical programs or software packages used to conduct statistical analyses 16 9 7.36

MARS

Sure, here is a list of the most important points that the M.A.R.S. Meta-analysis Reporting Standards claim:

Title and Title Page

  • The title should clearly indicate that the report is a meta-analysis and should include the topic of the meta-analysis.
  • The author note should list all authors of the meta-analysis, their affiliations, and their contact information.
  • The funding source(s) for the meta-analysis should be disclosed in a footnote.

Abstract

  • The abstract should provide a concise overview of the meta-analysis, including the research question, the search strategy, the inclusion and exclusion criteria, the data analysis methods, and the main findings.

Introduction

  • The introduction should provide a clear statement of the research question or relation(s) under investigation.
  • The introduction should provide a brief overview of the relevant literature, including the theoretical, policy, and/or practical issues related to the question or relation(s) of interest.
  • The introduction should provide a rationale for the selection and coding of potential moderators and mediators of results.
  • The introduction should describe the types of study designs used in the primary research, their strengths, and their weaknesses.
  • The introduction should describe the types of predictor and outcome measures used, their psychometric characteristics.
  • The introduction should describe the populations to whom the question or relation is relevant.
  • The introduction should state the hypotheses, if any.

Methods

  • The methods section should provide a detailed description of the meta-analysis methods, including the search strategy, the inclusion and exclusion criteria, the data collection and coding procedures, the data analysis methods, and the assessment of heterogeneity.

Results

  • The results section should present the findings of the meta-analysis, including the effect sizes, the heterogeneity statistics, and the moderator and mediator analyses.
  • The results section should also present the sensitivity analyses and any other relevant findings.

Discussion

  • The discussion section should interpret the findings of the meta-analysis in light of the existing literature.
  • The discussion section should discuss the limitations of the meta-analysis, including any potential biases in the selection of studies or the data analysis methods.
  • The discussion section should discuss the implications of the findings for future research and practice.

References

  • The references section should list all of the sources cited in the meta-analysis report.

In addition to these points, the M.A.R.S. Meta-analysis Reporting Standards also include recommendations for reporting the following information:

  • The characteristics of the primary studies
  • The effect sizes for individual studies
  • The funnel plot
  • The forest plot

es, the Meta-analysis Reporting Standards (MARS) developed by the American Psychological Association (APA) outline crucial ethical considerations for researchers conducting meta-analyses. While the MARS guidelines primarily focus on reporting standards, they also address ethical principles that should be upheld throughout the meta-analysis process. Here are some key ethical points emphasized in the MARS:

  1. Respect for Participants: Meta-analyses rely on data collected from primary studies involving human participants. Researchers conducting meta-analyses must respect the rights and well-being of these individuals. This includes ensuring that the primary studies obtained informed consent from participants and adhered to ethical research practices.

  2. Transparency and Accountability: Meta-analysis researchers should maintain transparency in their methods and findings. This includes clearly reporting the search strategy, inclusion and exclusion criteria, data coding procedures, and analysis techniques. Openness allows for scrutiny and evaluation of the meta-analysis’s rigor and credibility.

  3. Accuracy and Integrity of Data: Researchers must exercise due diligence in ensuring the accuracy and integrity of the data extracted from primary studies. This involves careful verification of data sources, checking for errors or inconsistencies, and addressing potential biases or limitations in the data.

  4. Fair and Objective Interpretation: The interpretation of meta-analysis results should be fair, objective, and unbiased. Researchers should avoid selectively presenting or interpreting data to support a particular viewpoint. Instead, they should consider alternative explanations, potential limitations, and the broader context of the research findings.

  5. Responsible Dissemination and Utilization: Researchers have a responsibility to disseminate their meta-analysis findings in a responsible manner. This includes considering the potential impact of the findings on individuals, communities, and society at large. The dissemination should be accompanied by clear explanations of the study’s limitations and implications.

  6. Avoiding Misrepresentation and Misinterpretation: Meta-analysis researchers should take steps to avoid misrepresenting or misinterpreting their findings. This includes refraining from making exaggerated claims or drawing unwarranted conclusions that go beyond the scope of the data.

  7. Addressing Conflicts of Interest: Researchers should disclose any potential conflicts of interest that could influence the conduct or interpretation of the meta-analysis. This ensures transparency and helps maintain the integrity of the research.

  8. Respect for Intellectual Property and Authorship: Meta-analysis researchers must respect intellectual property rights and acknowledge the contributions of others. This includes proper citation of primary studies and giving due credit to the original authors of the research findings.

  9. Protecting Confidentiality and Privacy: When dealing with individual-level data, researchers must adhere to principles of confidentiality and privacy protection. This includes ensuring that data is appropriately secured and that participant anonymity is maintained unless explicitly agreed upon.

  10. Ethical Considerations in Data Sharing: Researchers should consider the ethical implications of sharing data used in meta-analyses. This includes ensuring that data sharing practices comply with relevant ethical guidelines and protect the privacy of participants.

Additional Issues Related to the Reporting of Meta Analysis

Space Limitation

Something must be said about practicality and the limitations imposed by editors. [For example] in a recent paper we had 26 pages of included references and the editors wanted us to condense a review of over 200 studies into 40 pages max with all tables. Adding all these potentially important details is impossible in most reports.

Journals have only limited printed pages, and the detail needed to report completely one study confl icts with the desire of the journal to publish as many (worthy) studies as possible. As noted above, in research synthesis this issue arises most frequently when considering whether to publish the table of characteristics and results of individual studies. And, as one of our respondents suggested, sometimes even just the references to these studies can go on for pages.

  
Questionable Behaviors by Researchers and Authors
  • Simultaneous Submission of the Same Manuscript to Different Journals
  • Piecemeal (or Fragmented) Publication

  • Duplicate Publication of the Same Work

  • Self-Plagiarism

  • Plagiarism