The issue with low birthrates in Japan matters because the average age of the population is going up relative to tge number of births. This means that an increasingly smaller workforce will have to support a growing number of nonworkers.
As it is, family building within Japanese society is hampered by the demands of Japan's workplace culture. People can spend anywhere between 2 to 4 hours a day in commuting, another 8 in working, plus unpaid overtime. Afterwards, no one in a married household feels like sex. This leads both to a very high infidelity rate among married couples, and an abysmally low birth rate. This in turn feeds into tge economic stagnation Japan cannot escape.
Japan is unlikely to take any immigrants from the third world, so this reasoning smacks of "Scary brown people!" but Japan does need to raise it's birth rate, and the first step is breaking the oppressive working culture.