negative regulation of cholesterol transporter activity