The band members, already accomplished musicians, first came together in early 2001. Originally comprised of guitarist/vocalist Todd Sheaffer, violinist/vocalist Tim Carbone, mandolinist/vocalist John Skehan, multi-instrumentalist/vocalist Andy Goessling, drummer/vocalist Carey Harmon, and bassist ]Dave Von Dollen, the group played both original compositions mostly written by Sheaffer, the former From Good Homes songwriter, as well as traditional songs. After just three weeks of rehearsals, they went into a local recording studio and tracked a five song demo. Recorded live with no overdubs except backing vocals, The Black Bear Sessions set Railroad Earth's career in motion. Within a week they had a manager on board who then sent their demo out to some of the country’s most prestigious festivals, quickly landing a slot at the Telluride Bluegrass Festival. That was their tenth gig. The Black Bear Sessions was released to the public in June 2001, and following their appearance at Telluride Sugar Hill Records approached the band and offered them a record deal. The Black Bear Sessions contains some of the most popular songs in Railroad Earth's repertoire, including "Seven Story Mountain" and "Head."
In June 2002, one year from the release of their debut, Railroad Earth’s second album, Bird in a House, came out on Sugar Hill Records, featuring songs such as "Bird In A House," "Like A Buddha," and "Mighty River.". The band continued its steady touring schedule, playing theatres, folk-music festivals, jam-band festivals, and other venues, while gathering much praise for its unique sound and enjoyable live concerts. The band eventually built up a fanbase who call themselves Hobos.
In 2003 Dave Von Dollen was replaced by bassist Johnny Grubb. In 2004 the band released its third studio effort, The Good Life, featuring "Storms," Mourning Flies," and "Goat." The band continued to tour throughout the country, and its first live album, Elko, was issued in late January 2006. They performed at the Austin City Limits Music Festival on September 15, 2007.
The band's fourth studio album, entitled Amen Corner, was released on June 10, 2008. The new album features songs such as "Been Down This Road," "Hard Livin'," "Waggin' The Dog," and "Lovin' You".
On November 11, 2009, Grubb announced his departure from the band after Railroad Earth's New Year's Eve run of shows in Portland, OR. In February 2010, Andrew Altman of the Codetalkers and Blueground Undergrass was named as his successor.
The band's self-titled fifth album was released on Oct 12, 2010. The album features Andy Goessling playing electric guitar for the first time, plus an 11-minute instrumental written by John Skehan entitled "Spring-Heeled Jack" (SHJ is John Skehan's initials backwards); the latter is the longest studio track to date for the band. bluegrass Jam jam band Alt-country folk