Abstract
Background
Generative AI tools (e.g., ChatGPT) are increasingly used by patients seeking health information, yet no validated instrument measures patients' capacity to use these tools effectively. This study aimed to identify the latent factor structure of a novel Generative AI Health Literacy questionnaire using exploratory and confirmatory factor analysis.
Methods
This cross-sectional study distributed a 57-item questionnaire to 653 adult patients (interim analysis toward a target n=880) at the department of medicine clinics. Skip logic produced two analytic blocks: Block 1 (all 653 respondents; 17 items on digital literacy and trust) and Block 2 (372 AI users; 37 items on AI-specific domains). Exploratory factor analysis (EFA), using polychoric correlations with promax rotation, was conducted separately per block, with parallel analysis guiding factor retention. EFA-informed confirmatory factor analysis (CFA) followed via structural equation modeling.
Results
Mean age was 53 ± 16.6 years; 53% were female, 50.4% White, 38.7% Black, and 46.7% college graduates. Overall, 57.1% used AI for health information, of whom 45% used ChatGPT. For Block 1, parallel analysis supported a two-factor solution: Digital Literacy (8 items; variance 0.47) and Trust in AI (4 items; variance 0.85); CFA showed reasonable fit (χ²=94.6, p0.001; CFI=0.990; RMSEA=0.042; SRMR=0.040). For Block 2, parallel analysis supported four factors: AI Use Frequency (5 items; variance 0.32), AI Competency (13 items; variance 0.6), Impact of AI on Health Understanding (3 items; variance 0.84), and Cognitive Evaluation of AI Output (3 items; variance 0.22). The four-factor CFA showed moderate fit (χ²=520, p0.001; CFI=0.930; RMSEA=0.062; SRMR=0.084). There is not currently a significant difference in AI use between gender groups (p=0.330) or pooled race groups (p=0.228).
CONCLUSION:s and Relevance: This analysis revealed a parsimonious four-factor structure, with AI Use Frequency demonstrating excellent psychometric properties. These preliminary findings will be cross-validated in the full sample. Once validated, this instrument could help identify patients—particularly older adults and those with lower digital literacy—who may benefit from targeted support navigating AI for health information.