The 19-year-old broke the deadlock with an 88th minute shot set up by Eden Hazard after a frustrating game in the heat of Rio de Janeiro’s Maracana stadium.
Belgium now have six points from two games and are well placed to take top spot in Group H. The size of the qualification mountain facing Fabio Capello’s Russia now depends on the result of the South Korea-Algeria match later Sunday.
The two sides set off at a furious pace that could not be maintained in the Maracana stadium.
Viktor Faizulin shot from the edge of the penalty area forcing a save from Thibaut Courtois in the Belgian goal.
Belgium’s Napoli striker Dries Martens shot narrowly wide after beating two players. Moments later he broke free again but pushed the ball across the face of the Russian goal.
The game did not improve in the second half although the tension did mount as striker Romelu Lukaku directed some angry words at Belgian coach Marc Wilmots when he was replaced by Origi. Lukaku was substituted for the second straight game after a another disappointing performance.
But Origi, who plays for Lille in France, proved his worth with the goal as Belgium piled on the pressure in the late stages, with fellow substitute Kevin Mirallas hitting the post with a free kick seconds earlier.