In the aftermath of his win in Dusseldorf, Fury had said: "I'm not sexist. I believe a woman's best place is in the kitchen and on her back. That's my personal belief. Making me a good cup of tea, that's what I believe."
He has also drawn criticism for saying that fellow SPOTY nominee Jessica Ennis-Hill "slaps up well".
But Fury, who refers to himself as the 'Gypsy King' because of his Irish traveller heritage, told BBC Radio 2's Jeremy Vine programme on Monday: "I love my women and what I said goes for my wife alone. She knows her place, I know her place. That's our culture of people.
"That's nothing to do with the world or anybody else and if I was a normal person, I wasn't in the spotlight, no-one would be making a scene about what I say to my wife."
The new world champion has also previously said it would only take the legalisation of paedophilia in addition to the decriminalisation of homosexuality and abortion to see "the devil come home".
In an interview he said: "There are only three things that need to be accomplished before the devil comes home: one of them is homosexuality being legal in countries, one of them is abortion and the other one's paedophilia.
"Who would have thought in the '50s and '60s that those first two would be legalised?"