The quartet took more than half a second off the 27-year-old record of 41.37sec set by the former East Germany in 1985, beating home arch-rivals Jamaica (41.41sec) while Ukraine took bronze (42.04sec).
The Americans’ 100 metres silver medalist and 200 metres bronze medalist Carmelita Jeter anchored the team to gold with a devastating burst down the final stretch and screamed for joy after she crossed the line.
It was a second gold for 200m champion Allyson Felix while Tianna Madison and Bianca Knight completed the quartet.
It was the United States’ first Olympic gold in the event since the 1996 Atlanta Games and only their first medal since the 2000 Sydney Games, where they took bronze.
The United States took silver in 2:57.05 while Trinidad and Tobago took bronze in a national record of 2:59.40.
Double amputee Oscar Pistorius’s hopes of an historic medal never looked likely as South Africa, who had only reached the final after appealing when they failed to finish their heat, finished eighth and last.
Kenya were disqualified from Thursday’s heat after Vincent Mumo Kiilu running in the second leg of the event was adjudged to have tripped the South African runner in the race.