Rigor and Reproducibility

Seminar 1: Data Publication and Citation: How do I get credit for promotion?

• Presenters:  Lance Waller- Director, WHSC Data Science Initiative and Jennifer Doty, Research Data Librarian, Emory Libraries
  • Date of Presentation: 7/31/2020
• References From Presentation:
  • DataCite:  https://datacite.org/cite-your-data.html
  • FAIR guiding principles - Wilkinson, M. D., Dumontier, M., Aalbersberg, Ij. J., Appleton, G., Axton, M., Baak, A., … Bourne, P. E. (2016). The FAIR Guiding Principles for scientific data management and stewardship. Sci. Data, 3, 160018. https://doi.org/10.1038/sdata.2016.18
  • Waller LA and Miller GW. (2016) More than Manuscripts: Reproducibility, Rigor, and Research Productivity in the Big Data Era.  Toxicological Sciences. 149(2): 275-6.
  • Waller LA(2018) Documenting and Evaluating Data Science Contributions In Academic Promotion In Departments of Statistics and Biostatistics. American Statistician
  • Why should I publish data?  (And some tools for doing so)
    • http://www.lib.berkeley.edu/scholarly-communication/publishing/lifecycle/your-data
  • Publishing data associated with increased citation rates.
    • https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0000308
    • http://best-practices.dataverse.org/data-citation/
    • Data Citation Synthesis Group: Joint Declaration of Data Citation Principles. Martone M. (ed.) San Diego CA: FORCE11; 2014 https://doi.org/10.25490/a97f-egyk

Seminar 2: Making Reproducibility Practical: Using R and R Markdown

• Presenter: Melinda Higgins, Research Professor, SON
  • Date of Presentation: Friday, September 4, 2020

Seminar 3: Rigor and Reproducibility at eLife

• Presenter: Ron Calabrese, PhD, Senior Editor at eLife
  • Date of Presentation: November 20th, 2020

Seminar 4: An Introduction to Open Science Principles and Tools

• Presenters:  Jeremy Kupsco, Research Informationist, WHSC Library, Kimberly Powell, Research Impact Librarian, WHSC Library, and Jennifer Doty, Research Data Librarian, Emory Libraries.
  • Date of Presentation: 02/26/2021
• Links from the Presentation
  • Report from the National Academies of Science, Engineering and Medicine: Open Science by Design:  Realizing a Vision for 21st Century Research https://www.nap.edu/catalog/25116/open-science-by-design-realizing-a-vision-for-21st-century
  • Preregistration
    • Open Science Framework(OSF): https://osf.io/registries
    • As Predicted: https://aspredicted.org/
    • PROSPERO(systematic reviews): https://www.crd.york.ac.uk/prospero/
    • Sample Preregistration: https://osf.io/jsznk
  • Registered Reports: https://www.cos.io/initiatives/registered-reports
  • Protocols.io: protocols.io
  • Open Science Framework: https://osf.io/
    • JHU OSF Laboratory Template: https://osf.io/ed4ps/
  • Emory Dataverse: sco.library.emory.edu/dataverse
  • NLM list of Data Sharing Repositories: https://www.nlm.nih.gov/NIHbmic/nih_data_sharing_repositories.html
  • ORCID Identifiers: https://orcid.org/ 

Seminar 5: Reproducing Reproducibility? Challenges in Reproducing Classifier and Predictive Models. 

Presenters: John Banja, PhD, Lance Waller, PhD

Presentation Date: 4/29/2021

Citations from Presentation:

Nagendran M, Chen Y, Lovejoy C, et al.  Artificial intelligence versus clinicians:  systematic review of design, reporting standards, and claims of deep learning studies.  BMJ 2020; 368:m689  doi: https://doi.org/10.1136/bmj.m689

You Look Like a Thing and I Love You: How Artificial Intelligence Works and Why It's Making the World a Weirder Place  November 5, 2019.  Janelle Shane 

Vollmer et al. (2020) Machine learning and artificial intelligence research for patient benefit:  20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ 368:l6927 doi: https://doi.org/10.1136/bmj.l6927

Seminar 6:  An Introduction to the New NIH Data Management and Sharing Policy

• Presenters: Jeremy Kupsco, PhD, Research Informationist, Emory Libraries and Jennifer Doty, Data Science Librarian, Emory Libraries
• Presentation Date: September 17, 2021

Seminar 7: REDCap at Emory: An Introduction to Best Practices for Research

• Presenters: Sean Mann, MPH, M.Ed, Product Manager, Research Solutions, Office of Information Technology (OIT), Monica Crubezy, PhD, Director of Research Solutions, Office of Information Technology (OIT)

  • Date of Presentation: November 18, 2020

Seminar 8: Why is it so hard to do good science?

• Presenter: Dr. Raymond Dingledine, PhD.  Professor in the Department of Pharmacology and Chemical Biology
  • Date of Presentation: February 3, 2022.
• Citations from Presentation
  • Dingledine R. Why Is It so Hard to Do Good Science? [published correction appears in eNeuro. 2018 Nov 8;5(5):]. eNeuro. 2018;5(5):ENEURO.0188-18.2018. Published 2018 Sep 6. doi:10.1523/ENEURO.0188-18.2018 Emory Access: Here
  • Begley CG, Ellis LM. Drug development: Raise standards for preclinical cancer research. Nature. 2012 Mar 28;483(7391):531-3. doi: 10.1038/483531a. PMID: 22460880. Emory Access: Here
  • Tversky A, Kahneman D. Judgment under Uncertainty: Heuristics and Biases. Science. 1974 Sep 27;185(4157):1124-31. doi: 10.1126/science.185.4157.1124. PMID: 17835457. Emory Access: Here
  • Tversky A, Kahneman D. The framing of decisions and the psychology of choice. Science. 1981 Jan 30;211(4481):453-8. doi: 10.1126/science.7455683. PMID: 7455683. Emory Access: Here

  • Kahneman, Daniel, and Amos Tversky. “Prospect Theory: An Analysis of Decision under Risk.” Econometrica, vol. 47, no. 2, [Wiley, Econometric Society], 1979, pp. 263–91, https://doi.org/10.2307/1914185. Emory Access: Here

  • Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011. Emory Libraries Information

Seminar 9: Significance Testing: The Reason that Scientific Results Have Poor Reproducibility

• Presenter: Timothy Lash, DSc, MPH, Rollins Professor and Chair of Epidemiology Department.
• Webinar Date: 4/21/2022

Title: An Introduction to the NIH Data Management and Sharing Policy

• Speakers: Jeremy Kupsco, PhD, Research Informationist, Woodruff Health Sciences Center Library and Jennifer Doty, MSI, Research Data Librarian, Woodruff Library
• Date Recorded: September 29, 2022

Title: Data Sharing from a Data Sharer's Point of View.

Presenters: Lance Waller, PhD and Vicki Hertzberg, PhD

Date Presented: 11/19/2022

Title: The Only Constant is Change: Power in Healthcare Economics and Technology Over the Last 40 Years.

• Speaker:  John Banja PhD, Professor Emory Center for Ethics

Webinar Date: April 20th 2023.

Title: Building On A Legacy: Soaring to New Heights.

Keynote Speaker: Dr. Patti Brennan, Director of the National Library of Medicine.

KeyNote Presentation Title: Real Libraries for a future powered by Artificial Intelligence.

Panel:

Monica Crubezy, PhD, is the Director of Research Solutions in the Emory University Office of Information Technology. 

J. William Eley, MD, MPH, serves as executive associate dean, Medical Education and Student Affairs, Graduate Medical Education, and Continuing Medical Education, and professor, Department of Hematology and Medical Oncology, Emory University School of Medicine. 

Mary M. Gullatte, PhD, RN, ANP-BC, AOCN, LSSYB, FAAN, is the corporate director of Nursing Research and Evidence Based Practice for Emory Healthcare. She holds an adjunct faculty position at the Nell Hodgson Woodruff School of Nursing at Emory University.

M. Venkat Narayan, MD, MSC, MBA, is executive director of the Emory Global Diabetes Research Center. He is the Ruth and O.C. Hubert Professor of Global Health and Epidemiology at Rollins School of Public Health and a professor of medicine and endocrinology at Emory University School of Medicine. 

Lance A. Waller, PhD, is an Emory professor in the Department of Biostatistics and Bioinformatics, Rollins School of Public Health. He leads the Woodruff Health Science Center's Strategic Initiative in Data Science and serves as faculty co-leader for Emory’s new Center for AI Learning.

Speaker: Dr. Judy Gichoya, MD,MS.  Associate Professor Department of Radiology.

Date of Webinar: October 5, 2023

Title: How to ARRIVE Prepared: Guidelines for Successful and Reproducible Animal Research.

Speaker: Dr. Crystal Gergye, DVM, Clinical Veterinarian in Division of Animal Resources.

Date of Presentation: 3/21/2024

Title: Advancing Research Through the Lenses of Sex and Gender with Rigorous Methods and Reproducibility.

Speaker: Dr. Cecile Lahiri, MD Mac, Associate Professor, Emory Department of Medicine and Division of Infectious Disease

Presentation Date: 04/11/2024

• Jeremy Kupsco, PhD, Research Informationist, Emory Libraries
• Lance Waller, PhD, Director, WHSC Data Science Initiative 
• Melinda Higgins, PhD, Research Professor, SON
• Jennifer Doty, MS, Research Data Librarian, Emory Libraries
• Monica Crubezy, PhD, Director of Research Solutions, LITS