Resources for Text and Data Mining

Text and data mining (TDM) uses automated tools in order to identify, extract, and present relevant data to one's research from large or numerous sources. By processing the available data in this way, researchers hope to show trends or patterns in the available data. TDM is used in both the humanities and sciences, and can apply to a wide variety of types of data sets. TDM can use some AI and machine learning tools, but is not exclusive to such tools.

We are currently updating our policies and guidelines regarding use of acquiring, using and accessing e-resources for TDM projects and/or AI large language models. The AI environment is rapidly changing!  Current guidelines are now available. Where legally and financially possible, Emory Libraries will liaise with vendors to accommodate researchers wishing to use corpora derived from licensed e-resources for computational analysis and AI learning. Before using artificial intelligence tools (or access to large amounts of content) with any licensed library resource, researchers should contact the library for more information about governing terms.  

• (1) use of the content is restricted (research/academic only), non-commercial
• (2) security of the environment for modeling. This is particularly important for any allowed licensed use for machine learning. Most vendors will require that this data be in a secure environment for analysis.
• (3) restrictions on use of third party tools (for AI). Most vendors will not allow for use of their content in third-party tools due to training data concerns and copyright restrictions 
• (4) data management (where will this data live? Does it need to removed once analysis has been completed?).

As a general rule, check with the relevant subject librarian before beginning any project that involves use of Emory e-resources for TDM or AI purposes!

Databases often have their own rules and restrictions on what is and is not permissible when it comes to applying TDM methods to their data. In addition, access to these databases comes in a variety of forms, mediated by Emory Libraries.

Broadly, databases fall into four categories

1. Purchased Resources. These are resources Emory libraries has either purchased or with whom Emory has a Perpetual Access License (PAL). While there may be some restrictions on how data can be used--especially when it comes to publication--generally speaking these databases allow TDM in some capacity. One should not assume the these resources also include AI use.
2. Resources Accessible through Purchase. These are resources Emory may have access to in some capacity, but do not (presently) allow access to the kinds of data needed for TDM. However, if access to these data is needed, it may be purchased. To request that Emory purchase access to these data, please contact your relevant subject librarian.
3. Freely Available Resources. These databases are open access and either openly allow TDM or allow TDM broadly, but with specific (generally minor) restrictions.
4. Restricted Resources. These resources either forbid the use of their data for any and all TDM projects, or Emory does not have sufficient access to these databases to permit TDM usage.

As a general rule, check with the relevant subject librarian before beginning any project that involves use of Emory e-resources for TDM or AI purposes!