Why did the Whig Party collapse?

The Whigs were also badly hurt by the short-lived Native American or Know-Nothing party, which was primarily anti-immigrant and anti-Catholic.

That is just part of the answer of course. Here is another account, again too simplistic:

• For a brief time, many Americans supported the nativist Know-Nothing Party, concerned that the Know-Nothings might represent the only truly national party possible, largely united by a general fear of Catholic immigrants.
• But in the end, the issue over slavery proved stronger than fears over non-protestant immigrants, and southerners lined up behind the Democratic party and northerners behind the Republican party.
• Sectional party systems replaced a national party.

This seems to be the definitive detailed account.  Try this book too.  Most of the time, however, parties do not collapse but rather party members fall in line, fearing the costs of the alternatives, including the costs to their careers.

Here is my previous post Why did the Federalist Party collapse?