Text Mining in Insurance: From Unstructured Data to Meaning

Every day, insurance companies collect an enormous quantity of text data from multiple sources. We present a strategy to make beneficial use of the large amount of information available in documents by exploiting natural language processing. After a brief review of the basics of text mining, we describe a case study in which, by analyzing the accident narratives written by the researchers of the National Highway Traffic Safety Administration of the U.S. Department of Transportation, we aim to extract latent information that can be used to fine-tune policy premiums. The process involves two steps. First, we classify the reports according to the relevance of their content to determine the risk profiles of the people involved. Next, we use these profiles to create new latent risk covariates for a company’s ratemaking process.