"Keep on My Side" was Ammonia's fourth single from their second album Eleventh Avenue. It was released in May 1998 on the Murmur Records, just after the album's release and was the last song the band released as a single before breaking up. The song, together with the other songs on the album was recorded in 1997 at Tarbox Studios in Cassadaga, New York, with producer Dave Fridmann (The Flaming Lips, Mercury Rev). Australian musicologist, Ian McFarlane, felt the single showed "melodic rock akin to UK bands like Radiohead or Teenage Fanclub."