Explainable Qualitative Analysis with Large Language Models

Abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities in reasoning, summarization, and natural language understanding across diverse domains. However, despite their growing adoption in decision-critical settings, the interpretability of their qualitative judgments remains limited. This paper explores explainable qualitative analysis using LLMs, focusing on how such models generate, structure, and justify non-numerical insights. We argue that qualitative reasoning—such as thematic interpretation, sentiment justification, and contextual inference—requires distinct explainability mechanisms beyond traditional feature attribution. A structured framework is proposed to extract, analyze, and validate explanations generated by LLMs during qualitative tasks. Through controlled experiments on text interpretation and expert-aligned reasoning benchmarks, we evaluate the faithfulness, consistency, and human-alignment of LLM-generated explanations. The findings highlight both the promise and current limitations of LLMs as explainable qualitative analysts and provide practical insights for deploying them responsibly in research and decision-support systems.