After Sugar Ray's defeat by Maxim for the Light-Heavyweight title in 1952. Six months later he announced his retirement from the world of boxing he'd made plenty of money and invested large sums of it into his businesses.

Now for the first time in his life, Ray was unemployed and instead of sitting around idly, he become a dancer, which had been his first love as a child. He got gigs in Las Vegas and even toured parts of Europe, making somewhere in the region of $15 000 dollars per week - exceptional for someone with Ray's background of no real formal dance training of which to speak. Initially higher than that of the big Hollywood dancing stars Ginger Rogers, Gean Kelly and Fred Astaire. This was to last over a two and half year period.

While in Paris France Ray received an urgent telegram to return home to the States because his business, Ray Robinson Enterprises - which consisted of; the Cafe, Sugar Ray's Quality Cleaners, his second wife Edna Mae's Lingerie shop, and George Gainford's Golden Glover Barber shop valued at $300,000 dollars - was in trouble. Although Ray still had a large proportion of the money from the two Randolph Turpin bouts which was roughly $250,000, dollars as well as the consolation money from the Maxim bout another $150,000 dollars, little did he know that his problems where about to get worse.

While on tour in Europe Ray had left his business interests in the hands of his business manager. He was the real-estate man who had handled the purchase of all the properties in Ray's businesses including his home and that of his mum's house. Ray glanced through the books and realised that somehow, $250 000 had disappeared. In addition, he was also threatened with foreclosures on all his mortgages, and the Internal Revenue Services (IRS) was all over his back for unpaid tax. For Ray the only way out of this hole was to return to boxing which he did by recapturing the middleweight title for the third time from Carl 'Bobo' Olson. The IRS seized $81 000 dollars, which was the entire purse from that bout. In the return match which Ray won, $90,000 dollars was seized from a purse of $150 000 dollars, thus prolonging his boxing career.