Originally Posted by Cassidy
Pretty much this
I've tested this fairly extensively with my
RME Fireface 800
EMU1616M (when it worked)
stock sound on Striker extreme, X58 sabertooth and the probably identical sound in the sabertooth Z87.
And using numerous hi-fi speakers and my adam and focal monitors.
Playback quality does vary a little between sources, but it's very, very similar, even between the stock sound and the RME. (actually I forgot the EMU0202, and maybe the difference can be fairly significant, but it's still tiny compared to the difference between speakers)
Playback quality varies absolutely massively with the monitors/speakers. If you were to use a hi-fi amp in between the source and passives, then the effect of that is in between, with my very limited tests (with a NAD and Rotel somethingoroether), but still making vastly less difference than either the speakers or their placement and the rooms treatment.
You want better playback quality, get better monitors, and/or set up a decent listening position and treat your room.
Similarly, headphones amps do have an effect, but it's pretty marginal and unless you're using very high impedance headphones, tiny compared to the difference between different headphone models. In fact, in my experience it's even smaller than the difference between headphones that are in different phases of burn in (checked while I had access to 3 sets of ATHM50s - mine, antons and my sisters, with the usage frequency probably something like mine >>>>>>> antons > sisters, so they all sound different, and using the headphone amps in my fireface, pro40 and samson s-phone).
TLDR: The biggest differences by far happen with the stuff that actually makes the air move, not in the electronics.
Headphone amp adendum - the question is really whether they have enough oomph to drive high impedance headphones. Low impedance can be driven off nearly anything. Unless you're talking about the amps in mobile devices (which are uniformly dreadful) then they're probably fine.