Nkomo, 78, was said to have died at St. Anne’s hospital in Harare.
“We have lost our vice-president John Landa Nkomo, he was suffering for a long time with cancer,” said Mugabe “We have lost a real revolutionary, a fighter of freedom, a friend of the people. He will be dearly missed by all of us.”
Nkomo was appointed to one of the country’s two vice-presidencies under Mugabe in late 2009.
Responsible for overseeing financial, economic and environmental policy Nkomo, a former speaker of parliament, was seen as loyal to Mugabe.
He “can be counted on to do Mugabe’s bidding,” a leaked US diplomatic cable dating from 2009 said.
His Ndebele roots made him useful in checking the opposition Movement for Democratic Change in Matabeleland, a western region where the MDC enjoys strong support.
While he was sometimes mooted as a successor to Mugabe, few believe he had a real shot at taking the top post.
But his death will rekindle the succession battle in the ruling ZANU-PF as the country’s power-sharing government nears the end of its tenure.
It is still unclear who Mugabe will appoint to replace Nkomo, or if he will be replaced before elections slated to take place this year.