negative regulation of vitamin metabolic process