Gen X and Boomers are the only ones who can afford to eat at fancy restaurants anymore