Actually it can tear the country apart, but not because of the stupid "destroying our good White culture" argument. If more people come here than there are jobs, it could do either or both of the following:
1) lower the de facto wages of many unskilled jobs, thus reducing the quality of life for the entire demographic of people who work those jobs
2) disrupt the lives of people who already have a living in the U.S., which will create more suffering than there would be if those without jobs remained jobless
Again, this whole argument assumes that there is nothing being done about population control (the real source of this problem), so the available solutions aren't pretty. No matter what we do, there are going to be people starving to death as long as we have more people than there are jobs for. But in the meantime we might as well do what we can to protect the livings of people who already have jobs, because if we don't, we're basically just telling all unskilled workers in the entire world, "Hey, enjoy your job while it lasts, because it could be swiped away from you at any moment."