He continued composing songs such as "Letter Full of Tears" for Gladys Knight & the Pips (1961) and "I'm Hanging Up My Heart for You" (1962) for Solomon Burke, both of which became Top 20 hits on the national charts. Covay also composed the song "I'm Gonna Cry (Cry Baby)" which Wilson Pickett recorded for Atlantic Records. Working for Atlantic, Covay had a major hit as a performer with his song "Mercy, Mercy" in 1964. The following year, The Rolling Stones covered the song on their 1965 album, "Out of Our Heads", to much acclaim. Covay co-wrote "See Saw" with Steve Cropper, which was recorded at Stax Records in Memphis, Tennessee. Covay's recording peaked at No. 5 on the R&B charts, and Aretha Franklin's version of the song also became a hit. In the following years, Covay penned a number of modest hits for artists including Etta James and Otis Redding. In 1967, Aretha Franklin recorded his song "Chain of Fools," which reached No. 1 on the R&B charts and won Franklin a 1968 Grammy for Best Female R&B Vocal Performance.
Over the next decades, Don Covay worked with Mercury Records, and had hits with songs like "I Was Checkin' Out, She Was Checkin In." His songs were recorded by a wide range of artists from Bobby Womack to rock acts like Steppenwolf and Small Faces. He briefly recorded for Philadelphia International Records and, after a hiatus, he did a stint as a backing singer for The Rolling Stones. After suffering a stroke in 1992, Covay received a Pioneer Award from the Rhythm and Blues Foundation in 1993. His final album, "Adlib" - which was released in 2000 - featured an impressive roster of artists including Wilson Pickett, Paul Schaffer, Otis Clay, Huey Lewis, and Ann Peebles. Covay passed away January 31, 2015 in Franklin Square, New York.