Famine progresses that way in ever society. Look what was happening at the same time the Viet Minh were launching successful attacks on Japanese convoys.
Every population has a breaking point where the adult men become too weak to fight, but you gotta starve like 85% of the population to make that happen. 2 million dead in North Vietnam didn't put a dent in their forces.