NDS 4.4 vs 4.6 - An attempt at science...

Are those measurements displayed with the same smoothing, and same vrtical scale? If so, then they are quite significantly different, not least the peak at about 30Hz considerably narrower/sharpened in 4.4 compared to 4.6.

My general impression is that 4.6 has applied some smoothing below about 400Hz, but maybe reduced the smoothing above that - and the effect could very well be significant audibly.

Innocent Bystander posted:

Are those measurements displayed with the same smoothing, and same vrtical scale? If so, then they are quite significantly different, not least the peak at about 30Hz considerably narrower/sharpened in 4.4 compared to 4.6.

My general impression is that 4.6 has applied some smoothing below about 400Hz, but maybe reduced the smoothing above that - and the effect could very well be significant audibly.

My graphs and description could have been better... The upper graph has both grey and orange traces, but they are virtually on top of each other. This is the 4.4 vs 4.6 graph. The other orange graph below is the room measurement from Fuzzmeasure, which I added just to prove the the FFT analysis was yielding broadly similar results to that from Audacity.

Click on the graph and it opens in a separate window which is easier to see,

Dave.

Are, now I see - I mistook the reference to orange to mean the bottom one, not scrutinising the apparently grey one to see hints of orange peeping through at the top end!

So, if I understand, the three plots  are all supposed to be a room measurements, one fed through NDS and the other one direct. What is more interesting to me than the lack of difference between the two firmware versions (though that is the subject), is the difference between NDS and direct. Is the direct through the Mac’s Internal DAC? i wonder what it would look like through the NDS’ DAC. (And what to make of it however that appears may be another question!)

DaveBk posted:

I've been following the 4.4 vs 4.6 debate closely, and also visited another forum member this week to see if I could hear any difference on their system. Based on the test at Graham's I was of the view that something was going on with the bass response and wondered if this was impacting the in room response due to the interaction with the various room modes. I had a little time to myself this morning and decided to try an experiment.... When I was looking at various room acoustic treatments, I bought a measurement mic, a USB amp and Fuzzmeasure, so I could see the impact the treatments were having. I also have Audacity on my Mac.

This morning I remeasured the room response (the big peak at 30Hz is my main room mode). The usual choppy in room response which is much better since treatment, but still interesting to observe is shown in the lower graph. This was a mono, 2 second sweep, played through both speakers and measures at my usual listening position.

Then, I worked out a way of testing the NDS:

First I downgraded to 4.4.

I set up Audacity to record a mono track using the measurement mic, and USB pre amp as inputs. Exactly the same position as before.

I played a 20Hz to 20kHz sine wave sweep from a FLAC track via the NDS and recorded this using Audacity.

I upgraded the NDS to 4.6.

I played the sweep again, with absolutely nothing else changed.

Having trimmed both recordings using Audacity to remove the silence at the beginning and end of the sweeps, I used Audacity's 'Plot Spectrum' to perform a FFT analysis of each. The data from these was exported, and imported into Excel for plotting.

The resulting graphs are shown below, lined up against the in room response measured in Fuzzmeasure. Reassuringly the graphs up to around 2kHz are pretty close, which gave me confidence that the measurement methodology was working.

Looking closely at the 4.4 and 4.6 data, it is virtually identical up to 2kHz. Deviations do increase above this, but having played around with Fuzzmeasure, there is a lot of very 'chaotic peakiness' as frequency increases due to comb filtering effects, so getting an identical plot between graphs is virtually impossible unless you change the smoothing settings.

So, what do I conclude... Well, It seem obvious that 4.4 and 4.6 are reproducing an almost identical spectral power distribution, so my theory of more low end at the expense of mid bass is not supported by measurement. It doesn't explain the different views, but at least my mind is put to rest. I know our brains are far more sophisticated measuring devices, but they are also impacted by a bunch of learned preferences, emotions, psychoacoustics etc., but whatever the answer I'm going back to enjoy 4.6... 

initial bass was better  , then after a few days I concluded the same  no difference

Dave, thanks for your intrepid attempt to find a scientific explanation for 4.6gate. Indeed the orange and grey traces for 4.4 and 4.6 do seem incredibly closely matched. But an audible difference there clearly is, which you have also attested to. Even those like myself who actually prefer 4.6 to 4.4 hear a difference, it just that it is a positive enhancement as far as I am concernred. So I have no doubt that others will also hear a difference, but a retrograde one.

So where does this leave us? Well your measurements do seem to show minimal differences, but this is in your room using your system. You are also a 4.6er, so maybe your room is more 'suited' to 4.6. The rooms of those who prefer 4.4 may be more suited to 4.4, with 4.6 exiting some nodes and nasties that 4.4 fails to excite. Add to this personal preference and prior accustomisation (to 4.4) and you have potential for varied opinions of the SQ produced by the two firmware versions.

I also believe that as you go up the Naim streamer and amplification hierarchy, the system can be more sensitive to subtle changes to the frequency responses produced by difference firmware versions. And of course higher end speakers may also be more sensitive to firmware differences.

Thinking about it there are more reasons to support the reported differences in opinion between 4.4 and 4.6, than not.

All this could explain why Naim rely on extensive listening tests rather than measurements when they are evaluating a new product.

Thanks Nigel. My post was largely motivated by my own subjective view that the bass extension had improved with 4.6, but perhaps to the detriment of mid bass definition. My measurements have proved my ears to be wrong, which highlighted the overall subjectiveness of the  topic we have been discussing. All I have done in proven that 4.6 is not 'broken' in some fundamental sense as some have suggested. I don't think this is a firmware corruption, or anything that significant, just personal preferences being exposed. That's not wrong in any way, but it does reinforce the point that any change can be perceived as positive or negative.

Dave.

Actually Dave, I would suggest your ears (and brain) are right, it is just that you can't 'measure' it.

We believe that 4.6 is not broken but we prefer 4.6 over 4.4. If we could hear 4.6 from the perspective of those who really dislike it (in their room, with their system, borrowing their ears), then we may reach a different conclusion.

It appears from the straw poll you took that the majority prefer 4.6, but that is no consolation to those that do not. Naim appear to be entering the realms of high system (source) sensitivity (and high levels of interaction with the listener's room and system). I fear the advent of the ND555 with take this sensitivity to an even higher level, requiring far more care with future hardware and software developments.

Interesting times!

The perceived difference in sound might be nothing to do with room interactions or frequency response.

I’ve found, I tend to turn up the volume until I can hear the midrange instrument clearly. If I’m listening to a system with poor midrange clarity, I need to turn the volume higher than if the system had better midrange clarity. This increased volume has the effect of increasing the bass output.

nigelb posted:

Actually Dave, I would suggest your ears (and brain) are right, it is just that you can't 'measure' it.

We believe that 4.6 is not broken but we prefer 4.6 over 4.4. If we could hear 4.6 from the perspective of those who really dislike it (in their room, with their system, borrowing their ears), then we may reach a different conclusion.

It appears from the straw poll you took that the majority prefer 4.6, but that is no consolation to those that do not. Naim appear to be entering the realms of high system (source) sensitivity (and high levels of interaction with the listener's room and system). I fear the advent of the ND555 with take this sensitivity to an even higher level, requiring far more care with future hardware and software developments.

Interesting times!

Based on what I have read about the ND555 architecture, I hope Naim have  limited the chance of similar issues being raised. On the NDS, the Sharc DSP is on the same PCB as the DACs, so there must be a greater chance of it injecting noise into the local power lines. The new ND555 puts the Sparc, the ARM and all other high clock rate CPUs in a separate Faraday cage. All that comes out of this is nice clean balanced bits for the DAC. Hopefully, we will be able to enjoy the benefits of new firmware and sleep soundly in the future...