The Beige Bowler, of course, is a reference to the famous Brown Derby club in Hollywood.
The Mann Act is a federal law passed in 1910 prohibiting the transporting of a woman across a state line for immoral purposes.  It was originally intended to fight forced prostitution and “white slavery”, but because of the ambiguity of the word “immorality”, it got used to go after any type of moral turpitude.  The famous black boxer Jack Johnson was arrested and convicted under the Mann Act, as was Chuck Berry and Charlie Chaplin, believe it or not, although Chaplin was acquitted.  The act is still on the books, although it has been amended to make it less broad.

I don’t know if hiring Ginger would count as “immoral”, but having watched her walking down the aisle of the train, I think that Harry would say that what she does with that tail when she walks ain’t hardly moral.

As for the “shoe polish” remark; well, what can I say.  I saw “The Jolson Story” on TV when I was a kid, and it scarred me for life.