Anger about higher prices appears to outweigh the benefits of a hot labor market in the minds of consumers.