The workers exposed to metal fumes had an increased risk of metabolic syndrome, which was correlated with decreased serum adiponectin. Thus, we aimed to explore whether heavy metal exposure affects the adiponectin level. There were 96 male workers recruited from a shipyard at baseline. Apart from 82 participants completed the follow-up assessments, new participants were recruited in next year. Finally, there were 100 welding workers in the exposure group and 31 office workers in the control group. Inferential statistics on repeated measures were performed using generalized estimating equations. A weighted quantile sum (WQS) regression model was conducted to examine the joint effect of the multimetal exposure with serum adiponectin. Significantly negative associations of metals with adiponectin were detected in the welding workers, including Cr (β = −0.088; 95% CI: −0.148, −0.027), Mn (β = −0.174; 95% CI: −0.267, −0.081), Co (β = −0.094; 95% CI: −0.158, −0.029), Ni (β = −0.108; 95% CI: −0.208, −0.008), Cd (β = −0.067; 95% CI: −0.115, −0.018), and Pb (β = −0.089; 95% CI: −0.163, −0.015). The WQS regression suggested that Pb was the greatest contributor. In conclusion, our findings highlighted that welding workers exposed to heavy metals would reduce serum adiponectin.