Are broken links bad for SEO?

I just ran a site audit and found a bunch of broken links on my site. How worried should I be about this? Also, is it just internal broken links I need to worry about, or external ones too?