The song was nearly excluded from the album when the co-producer Neil Dorfsman voted against its inclusion, but the rest of the band out-voted him.
A simple rock and roll rhythm is used, with chord changes limited to I, IV and V chords. The long introduction has become iconic in some circles, with an instantly recognisable melody played on a synthesiser organ patch, giving it a slight Cajun twist. The singer mentioned in the lyrics is said to perform "down in the tunnels, trying to make it pay", a reference to busking in the subway. The songs he plays are oldies, including "I Got a Woman", "Be-Bop-A-Lula", "What'd I Say", "My Sweet Lovin' Woman", and "Mack the Knife". He also plays talking blues.