Ted Pappageorge, head of the powerful Culinary Workers Union Local 226, called it the "Trump slump." He said visits from Southern California, home to a large Latino population, were also drying up because people are afraid of the administration's immigration crackdown.

"If you tell the rest of the world they're not welcome, then they won't come," Pappageorge said.