Purpose: We examined factors associated with health literacy among elders with and without suspected COVID-19 symptoms (S-COVID-19-S). Methods: A cross-sectional study was conducted at outpatient departments of nine hospitals and health centers 14 February−2 March 2020. Self-administered questionnaires were used to assess patient characteristics, health literacy, clinical information, health-related behaviors, and depression. A sample of 928 participants aged 60–85 years were analyzed. Results: The proportion of people with S-COVID-19-S and depression were 48.3 and 13.4%, respectively. The determinants of health literacy in groups with and without S-COVID-19-S were age, gender, education, ability to pay for medication, and social status. In people with S-COVID-19-S, one-score increment of health literacy was associated with 8% higher healthy eating likelihood (odds ratio, OR, 1.08; 95% confidence interval, 95%CI, 1.04, 1.13; p < 0.001), 4% higher physical activity likelihood (OR, 1.04; 95%CI, 1.01, 1.08, p = 0.023), and 9% lower depression likelihood (OR, 0.90; 95%CI, 0.87, 0.94; p < 0.001). These associations were not found in people without S-COVID-19-S. Conclusions: The older people with higher health literacy were less likely to have depression and had healthier behaviors in the group with S-COVD-19-S. Potential health literacy interventions are suggested to promote healthy behaviors and improve mental health outcomes to lessen the pandemic's damage in this age group.