Rigor and Reproducibility

Seminar 1: Data Publication and Citation: How do I get credit for promotion?

• Presenters:  Lance Waller- Director, WHSC Data Science Initiative and Jennifer Doty, Research Data Librarian, Emory Libraries
  • Date of Presentation: 7/31/2020
• References From Presentation:
• DataCite:  https://datacite.org/cite-your-data.html
• FAIR guiding principles

Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., Axton, M., Baak, A., … Bourne, P. E. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18

• Waller LA and Miller GW. (2016) More than Manuscripts: Reproducibility, Rigor, and Research Productivity in the Big Data Era.  Toxicological Sciences. 149(2): 275-6.
• Waller LA(2018) Documenting and Evaluating Data Science Contributions In Academic Promotion In Departments of Statistics and Biostatistics. American Statistician
• Why should I publish data?  (And some tools for doing so)
  • http://www.lib.berkeley.edu/scholarly-communication/publishing/lifecycle/your-data
• Publishing data associated with increased citation rates.
  • https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0000308
  • http://best-practices.dataverse.org/data-citation/
  • Data Citation Synthesis Group: Joint Declaration of Data Citation Principles. Martone M. (ed.) San Diego CA: FORCE11; 2014 https://doi.org/10.25490/a97f-egyk

Seminar 2: Making Reproducibility Practical: Using R and R Markdown

• Presenter: Melinda Higgins, Research Professor, SON
  • Date of Presentation: Friday, September 4, 2020

Seminar 3: Rigor and Reproducibility at eLife

• Presenter: Ron Calabrese, PhD, Senior Editor at eLife
  • Date of Presentation: November 20th, 2020

Seminar 4: An Introduction to Open Science Principles and Tools

• Presenters:  Jeremy Kupsco, Research Informationist, WHSC Library, Kimberly Powell, Research Impact Librarian, WHSC Library, and Jennifer Doty, Research Data Librarian, Emory Libraries.
  • Date of Presentation: 02/26/2021
• Links from the Presentation
  • Report from the National Academies of Science, Engineering and Medicine: Open Science by Design:  Realizing a Vision for 21st Century Research https://www.nap.edu/catalog/25116/open-science-by-design-realizing-a-vision-for-21st-century
  • Preregistration
    • Open Science Framework(OSF): https://osf.io/registries
    • As Predicted: https://aspredicted.org/
    • PROSPERO(systematic reviews): https://www.crd.york.ac.uk/prospero/
    • Sample Preregistration: https://osf.io/jsznk
  • Registered Reports: https://www.cos.io/initiatives/registered-reports
  • Protocols.io: protocols.io
  • Open Science Framework: https://osf.io/
    • JHU OSF Laboratory Template: https://osf.io/ed4ps/
  • Emory Dataverse: sco.library.emory.edu/dataverse
  • NLM list of Data Sharing Repositories: https://www.nlm.nih.gov/NIHbmic/nih_data_sharing_repositories.html
  • ORCID Identifiers: https://orcid.org/ 

Seminar 5: Reproducing Reproducibility? Challenges in Reproducing Classifier and Predictive Models. 

Presenters: John Banja, PhD, Lance Waller, PhD

Presentation Date: 4/29/2021

Citations from Presentation:

Nagendran M, Chen Y, Lovejoy C, et al.  Artificial intelligence versus clinicians:  systematic review of design, reporting standards, and claims of deep learning studies.  BMJ 2020; 368:m689  doi: https://doi.org/10.1136/bmj.m689

You Look Like a Thing and I Love You: How Artificial Intelligence Works and Why It's Making the World a Weirder Place  November 5, 2019.  Janelle Shane 

Vollmer et al. (2020) Machine learning and artificial intelligence research for patient benefit:  20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ 368:l6927 doi: https://doi.org/10.1136/bmj.l6927 

Seminar 6:  An Introduction to the New NIH Data Management and Sharing Policy

• Presentation Data: September 17, 2021
• Presenters: Jeremy Kupsco, PhD, Research Informationist, Emory Libraries and Jennifer Doty, Data Science Librarian, Emory Libraries

Seminar 7: REDCap at Emory: An Introduction to Best Practices for Research

• Date of Presentation: November 18, 2020
  • Presenters: 
    • Sean Mann, MPH, M.Ed

      Product Manager, Research Solutions, Office of Information Technology (OIT)

    • Monica Crubezy, PhD

      Director of Research Solutions, Office of Information Technology (OIT)

Seminar 8: Why is it so hard to do good science?

• Presenter: Dr. Raymond Dingledine, PhD.  Professor in the Department of Pharmacology and Chemical Biology
  • Date of Presentation: February 3, 2022.
• Citations from Presentation
  • Dingledine R. Why Is It so Hard to Do Good Science? [published correction appears in eNeuro. 2018 Nov 8;5(5):]. eNeuro. 2018;5(5):ENEURO.0188-18.2018. Published 2018 Sep 6. doi:10.1523/ENEURO.0188-18.2018 Emory Access: Here
  • Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012 Mar 28;483(7391):531-3. doi: 10.1038/483531a. PMID: 22460880. Emory Access: Here
  • Tversky A, Kahneman D. Judgment under Uncertainty: Heuristics and Biases. Science. 1974 Sep 27;185(4157):1124-31. doi: 10.1126/science.185.4157.1124. PMID: 17835457. Emory Access: Here
  • Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981 Jan 30;211(4481):453-8. doi: 10.1126/science.7455683. PMID: 7455683. Emory Access: Here

  • Kahneman, Daniel, and Amos Tversky. “Prospect Theory: An Analysis of Decision under Risk.” Econometrica, vol. 47, no. 2, [Wiley, Econometric Society], 1979, pp. 263–91, https://doi.org/10.2307/1914185. Emory Access: Here

  • Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011. Emory Libraries Information

Seminar 9: Significance Testing: The Reason that Scientific Results Have Poor Reproducibility

• Presenter: Timothy Lash, DSc, MPH, Rollins Professor and Chair of Epidemiology Department.
• Webinar Date: 4/21/2022

• Jeremy Kupsco, PhD, Research Informationist, Emory Libraries
• Lance Waller, PhD, Director, WHSC Data Science Initiative 
• Melinda Higgins, PhD, Research Professor, SON
• Jennifer Doty, MS, Research Data Librarian, Emory Libraries
• Monica Crubezy, PhD, Director of Research Solutions, LITS