muideen-abubakar / esg_nlp

ESG_NLP

The importance of Environmental, Social, and Governance (ESG) issues has risen in prominence over the last decade. In the early 1990’s fewer than 20 publicly listed companies issued reports that included ESG data; that number grew to almost six thousand by 2014 (Serafeim and Yoon, 2021). Regulations for SEC filings, Acts to follow certain standards for responsibility about Climate Change and Human Governance, and Investor and Shareholder support has driven the motivation for these disclosures. There has been little research analyzing the non-financial information content (Kölbel et al., 2020, p. 8) in financial disclosures. The most common methods for analyzing the non-financial, narrative information content still remain a manual- or dictionary-based approach (Berkman et al., 2019; Matsumura et al., 2018; Reverte, 2016; Verbeeten et al., 2016; Clarkson et al., 2008; Cormier and Magnan, 2007, i.a.). Also, only a few studies concerning non-financial information focus on the actual narrative information content, and rather address the quantity of non-financial information published (Armbrust, Schäfer and Klinger, 2020 p. 2; Hummel and Schlick, 2016). Domain-specific BERT variations like FinBERT (Araci, 2019) and BioBERT (Lee and Yoon et al., 2019), that have been either fine-tuned or pre-trained on domain corpus instead of or in addition to the generic English language, have achieved great results in domain-specific classification tasks. The primary interest of this research is to harness that benefit for ESG specific text classification tasks. For the same, we study building an environment-specific variation of BERT by fine-tuning the pre-trained BERT weights using a Masked Language Model (MLM) task on ESG corpus and then further fine-tuning our model for the classification tasks to predict: 1. A change or no change, and 2. A positive or negative change (if any) in environmental scores of companies using ESG related text in their 10-Q filings. We use a Sequence Classification that we fine-tune with our classification task.