Not necessarily public housing, but just stop insisting on overpricing everything, and stop insisting that all housing should be built for families of 5+, as so much new housing has been built for in the last several decades. The lowest rents in a neighborhood should absolutely not be nearly half the average income in the area, period!
Public transit doesn’t even have to be free, just cheap, and reliable. Too many cities with bus systems can’t get buses to the stops anywhere near on time over half the time, making people late for work when waiting for the bus should have got them there early. Which of course soon results in being fired, and being that much more difficult the get hired again.
The only low cost mental health in most areas is available only after filling out vast amounts of paperwork, and amounts to offering up a few days supply of a drug until you fill out more vast amounts of paperwork, often without any support. This is one of the reasons we so badly want a single-payer health care system, so people with problems just get an appointment, get their treatment, and the doctor gets paid every time, not playing with insurance companies that all want to handle claims differently, and so many who even deny claims for random reasons, leaving sick people without further care.
And yes, all of this, every bit altogether, is far cheaper than shipping all the homeless and other undesirables who will inevitably be attached off to countries that also don’t want them. And yes, all of the Sahara is inside other sovereign countries!