Methodology

Basic Instructions

The Stages of Sustainability content analysis tool assesses documentation to determine the documentation’s alignment within the Stages of Sustainability model. The output provided will show the extent to which the documentation aligns with weak or strong sustainability. This analysis is possible and reliable because communications reflect worldviews, mentalities, and actions. The documentation uploaded to the tool must be converted to .txt format and encoded in UTF-8. Text editing software (e.g. Microsoft Notepad, Microsoft Word, Apple Pages, or OpenOffice Writer) can save documents in this format. If the document is in .pdf or any other format, there are tools to convert the document to a .txt file (https://stirlingpdf.io/?lang=en_US or https://tools.pdf24.org/en/ may be useful).

A single file or multiple files may be uploaded at once. Either drag and drop multiple files to the upload box or browse to the files and hold ctrl on the keyboard while selecting multiple files. Uploaded files are not saved on our servers and are immediately deleted after the analysis is completed.

Output from the Assessment

The output generated from the keyword content analysis is delivered in a .csv file. The data includes:

•  The name of the document(s) analyzed

•  The count of the individual keywords related to the Stages of Sustainability Model

•  A summation of the keyword counts by stage

•  A calculation of the keyword percentages by stage based on total word count of the document.

• If 4 or fewer files are uploaded, a graph of the results will be provided. If more than 4 files are uploaded, the graph will not be provided because the graph will be too  crowded.

The .csv file is compatible with modern spreadsheet software (e.g. Excel, OpenOffice, Google Sheets) and  statistical software (e.g, R, SAS, SPSS). Statistical analysis, charts, and graphs can be created with this data. Descriptive data such as standard deviation, standard error, or means can be calculated with the data. With the output data, various statistical analysis can be performed. The keyword percentages will be the most important value as these numbers represent the documentation’s alignment with each of the 5 stages of sustainability. For example, the keyword percentages can be regressed against other data to determine relationships between strong sustainability and another concept. The keyword percentages can also be assessed by ANOVA to determine trends within the data. 

Examples of assessment of documents to discover strong sustainability trends include:

• An assessment of the sustainability strength of business education texts (Landrum & Ohsowski, 2017)

• An assessment of the sustainability strength of GRI and non-GRI sustainability reports (Landrum & Ohsowski, 2018)

• An assessment of the sustainability strength of major sustainability principles, frameworks, guidelines and standards (Demastus & Landrum, 2024)

Examples of determining relationships between non-sustainability constructs to sustainability stage include:

• A study of organizational culture type and relation to sustainability strength (Demastus et al., 2024)

• A study of leadership and culture and relation to sustainability strength (Niehoff, 2022)

The data will most likely fail statistical assumption tests if there are differences in document type, large differences in document total word count, authorships, writing style, formatting, document structure, and time frame. This is a limitation to the methodology. It’s likely that if several documents are analyzed from different sources, a data transformation (e.g. log10, square root, Box-Cox, reciprocal, arcsine, normalization) will be necessary to pass statistical assumption tests (normality, homoscedasticity, independence, multicollinearity). If assistance is needed with this, please contact us, as we’ll be happy to help with research related to the Stages of Sustainability tool.

Methodological Limitations 

•  The method is sensitive to the data provided, thus the need for data transformation is likely.

•  The keywords used in the analysis are biased towards environmental sustainability.

•  Keyword content analysis does not consider context of the document. Nuance, emotion, language style, or emphasis are not considered in this analysis.

•  Keyword bias is present due to the authors’ subjective interpretation of strong sustainability.

Notes About the Computational Structure of the Tool

The keyword content analysis follows guidelines of Structural Topic Modelling (STM) (Roberts et al., 2014) by applying an unsupervised approach, meaning computer software counts the keywords in the uploaded files. The keyword content analysis applies Fully Automated Clustering (FAC), meaning no algorithms are used to induce data beyond word count (Grimmer and Stewart 2013; Lucas et al. 2015).

References