There are significant differences in costs and effectiveness among the second-line treatment options for type 2 diabetes (T2DM). We aimed to evaluate the cost-effectiveness of the second-line anti-diabetic therapy in T2DM patients inadequately controlled on metformin (MET) in Taiwan from the perspective of the National Health Insurance (NHI). The Cardiff T2DM model was used to predict the occurrence of mortality, diabetes-related complications, and drug adverse events. Five second-line treatments were selected for the analysis: sodium-glucose cotransporter 2 inhibitors (SGLT-2-i), glucagon-like peptide-1 receptor agonists (GLP-1-RA), dipeptidyl peptidase-4 inhibitor (DPP-4-i), sulfonylurea (SU), and insulin (INS). Treatment efficacy data were obtained from meta-analyses and randomized clinical trials, whereas cost data were derived from the NHI databases. The analysis found that SU + MET (DPP-4-i as triple therapy) had the lowest cost, and SU + MET (SGLT-2-i as triple therapy) was associated with a mean incremental benefit of 0.47 quality-adjusted life years (QALYs) at an incremental cost of NT$2769, resulting in an incremental cost-effectiveness ratio (ICER) of NT$5840/QALY. Compared to their next less costly strategies, SGLT-2-i + MET (SU as triple therapy) and SGLT-2-i + MET (DPP-4-i as triple therapy) had ICER values of NT$63,170/QALY and NT$64,090/QALY, respectively. These results were fairly robust to extensive sensitivity analyses, but were relatively sensitive to baseline HbA1c, HbA1c threshold, and utilities. The addition of either SU or SGLT-2-i to MET was found to be cost-effective, using the 2019 forecast for GDP per capita of Taiwan (NT$770,770) as the willingness to pay (WTP) threshold.