Text and data mining: 6. Tools and resources

A browser-based suite of tools for undertaking text analysis. No programming is required to use Voyant Tools.
Pre-processing and analysis methods include:
- Stopword removal
- Term frequencies
- Topic modelling
- Collocation
- Concordance (key words in context)

An online TDM portal (sign in using the Microsoft option and your University credentials). The Digital Scholar Lab allows you to clean and apply TDM methods to some Gale Primary sources content, or you can upload content from elsewhere to the Lab. No programming is required to use the Gale Digital Scholar Lab.
Pre-processing and analysis methods include:
- Basic cleaning methods
- Parts of speech tagging
- Named entity recognition
- Term frequencies (Ngrams)
- Topic modelling
- Sentiment analysis
- Clustering analysis

A collection of open source tools for data analysis, including several text analysis tools. No programming is required to use the tools at the Data Science Toolkit.
Pre-processing and analysis methods include:
- Optical character recognition
- Sentiment analysis
- Named entity recognition for people, times/dates and places

'Leaders or followers? Measuring political responsiveness in the US Congress using social media data'
The Data Science Toolkit's geocoder was used to add geolocation data to a set of Tweets by parsing information from a users profile. This allowed location to be investigated as a variable in the analysis of the political responsiveness of members of Congress on Twitter. Barberá et al. (2014)

A suite of stylometric tools provided as a package for use with the analysis software R. Stylometry uses computational analyses to quantify aspects of writing style; this can be useful for, among others, determining authorship or a work or identifying the likely time period a work was produced in.
No programming is required to use many of the tools in the stylo R package, as a graphical user interface is provided. However, advanced users can choose to use the package programmatically if they are familiar with using R.
Pre-processing and analysis methods include:
- Basic cleaning
- Term frequencies
- Clustering analysis

'Computer stylometry of C. S. Lewis’s The Dark Tower and related texts'
The Stylo R package was used to analyse the provenance of the unfinished novel The Dark Tower, generally attributed to C. S. Lewis. doi.org/10.1093/llc/fqx043

A freeware corpus analysis toolkit for concordancing and text analysis.
Pre-processing and analysis methods include:
- Concordance
- Term frequencies
- Collocation

A catalogue of tools that can be used to undertake text analysis

A set of peer reviewed tutorials aimed at novices getting started with digital tools and computational techniques in the humanities, including many lessons useful for getting started with TDM.

For information and links to more data and text mining tools and methods for visualising analysis outputs please see the Data Analysis and Visualisation Toolkit.

For advice or assistance with using text or data mining tools, contact the data scientists at the Sydney Informatics Hub.