"The Foolish Thing To Do" is a single from Heaven 17 which featured Jimmy Ruffin. The track was written by all three members along with Nick Plytas. The track did not feature on any album. The B-side titled "My Sensitivity (Gets In The Way) was originally performed and written by Luther Vandross. Ruffin performed lead vocals on both tracks whilst Heaven 17 performed backing vocals and instruments. The single was released in mid-1988 and peaked at #80 in the UK, spending 2 weeks on the chart. The single would continue the band's decline in commercial success.