This problem was solved if I would reboot the system and open Firefox and watch videos first (before playing music). The problem still was recurring, though.
Apparently this is more than an ALSA sound driver fight between Firefox and Amarok. It's actually a bug (see link).
How to fix your Firefox audio: open a terminal/console, and type
ln -s /tmp/.esd-1000/socket /tmp/.esd/socket