DAW WARS

Screen Shot 2014-07-30 at 20.11.07.png

Digital audio workstation summing engine comparisons

There has been a long standing argument among users of digital audio workstations “DAW’s” that some sound sonically better than others. This has been primarily placed down to the summing engine which is the system of mathematical algorithms that calculate the final audio output based on the sum over the combined signals. However there is a counter argument from some programmers that in the digital domain that, “there can only ever be a singular answer” (Prof Monty Montgomery 2013[online]) and that rounding differences based on the systems could not be the cause of audible differences.

Additionally the quality of the stock effects is said to add significant discrepancies to the audio rather than ideally being transparent, Unless otherwise specifically mimicking an analogue model.

Loudness can be a key factor also “It's almost guaranteed that the same track rendered from any two DAWs will be a few dB different”(Audio Myths & Daw Wars 2009). With an even slightly comparatively louder mix appearing to have higher clarity over all, As humans we often perceive louder to better.

 

This has lead to countless arguments between artists, producers and audiophiles as to which sound best and why. Where as one DAW can become more overwhelming popular against another due to this culture of non critical comparison it has led to constant development updates and in some cases complete rewrites of DAW programs. 

While development and competition can appear to be a good driving factor to further the quality of the digital equipment available there are a growing number of software engineers that feel it to all be largely irrelevant and more about marketing features that will actually make no difference to your work. Prof Monty Montgomery one of the creators of the Vorbis audio compression file format stated for example that “compact audio tape could reach 9bits at best, And the very best studio quality tapes recorded could barely hit 13bit” (Prof Monty Montgomery 2013[online]) Making the chase for all DAW’s to work at 32bit floating point on a platform that experiences much less audible issues such as noise floor and hiss from tape a wasted effort.

 

The intention is to critically evaluate a selection of the most popular DAW’s in a systematic way that offers fair comparison. by using audio stems from a track from various sources such as vocals, instruments and sampled based audio. The audio will be in broadcast .Wav format at 24bit and 44.1 Khz sampling rate which will also be matched within the DAW’s where possible.

these audio files will then be mixed identically to the nearest 0.1dB per track margin via the mixer to fairly assess and discrepancies in the summing engine of the DAW’s. for this all other will be compared against a single control mix created in protools, the other will also have been matched to the number correlation of this mix.

To compare the results firstly they will be accurately measured in volume to the highest peak under 0dBFS and the overall average RMS of 0dBFS.

After this comparison the latter mix that was not the Protools sum will be inverted in phase within Protools. If there are any differences to the audio they will be the only audible sound allowing perception of the differences between the two, Should there be any. (“From a transparency perspective all DAW software is created equal” Scott Fisher 2009[online]) suggesting as far as mixing and summing we shouldn’t find a difference despite the insistence from companies and user that one exists. 

Global factors taken into consideration that could cause discrepancies if varied between DAW’s when compared.

  • Bit Depth of project - 32bit floating (32 bit static if floating is not an option)
  • Sample rate of project - 44.1 KHz
  • Panning Law - Linear
  • Meter weighting of sum audio - using Izotope's Ozone 5 on Peak ballistics and RMS also measured with Ozone 5 with identical settings per final audio sum. 
  • individual audio stems matched against the control mix to neared 0.1 dB
  • separate mix with and without compression and EQ applications incase of exact matches not being possible due to areas such as “knee” in compression and “Quality” in EQ not having strict laws to follow in development and could potentially vary dependant on programmer. Thus separating out a true test of the summing equations of the digital sound against the application effects. 
  • No Dither to be applied as dither algorithms may vary.
  • To be stated if the DAW is floating point of static.

The audio used is the control mix produced as the control from protools. And the same audio file will be processed via each DAW with identical settings where possible and an exact EQ match where possible.

Three common uses for an EQ are applied with a 24dB per Oct slope High pass filter applied at 80Hz removing low end.

A reduction peak has been applied to the 500Hz region with -5dB at a Q ratio of 2.00.

And a boosting peak at 5kHz of 2dB with a Q ratio of 0.50

 

The state of the audio mix for listening use is irrelevant in the test as the aim is to see if there results of multiple DAW’s null each other and to confirms if the perceived differences are actual physical differences within the DAW’s.

(Ozone 5 Metering settings)

Control mix specific details

 

Five channels of audio consisting of guitar, vocals and cello recorded through an Allen and Heath R16 analogue desk into Logic 9 and a sampled guitar layer not going through the desk but directly printed from the sampler.

 

Cello is -1.7dB

Guitar Mic 1 is -3.0dB

Guitar Mic 2 is -5.4dB

Guitar sampled is +0.9dB

Vocals are -1.3dB

 (meter reduction for protools control mix)

(meter reduction for protools control mix)

This mix is without effects and is for a direct comparison on the summing of the different DAW’s. The mix has been matched with the OSX version of the following DAW’s. using 24bit 44.1 .Wav files.

(EQ settings for protools control mix)

 

  • Reason 7
  • Logic X
  • FL Studio 11 (OSX Beta)
  • Reaper 4.7

 

Reason 7

Reason 7’s mix presented peak values of +0.1dB on both left and right and -4.4dB RMB on left with -5.0dB on the right. Reducing the fader around 3.4dB in Protools causes a large amount of cancelation with an reduction of around 40dB, However there is still audio across most of the audio spectrum as low as 100 Hz. This could be the case that reason sum’s to two decimal places and is the case of rounding difference but with an overall reduction of around -40dB would not be noticeable even under perfect conditions.

(EQ settings for Reason 7 EQ mix)

(Reason EQ response comparison)

(Reason 7 EQ response comparisons)

White = Control mix

Red = Reason mix

Yellow = Phase invert of reason mix against control mix

 

The EQ display curve for reason looks very different to the one represented in protools despite the settings being matched as closely as possible with reasons rounding not allowing exact figures such as 5000kHz, Despite this the response of the EQ is very similar differing in the lower end of the spectrum by as much as a dB but the high end varying only slightly and matching perfectly in some areas which is to be expected as the EQ rounding wouldn't allow exact values the high end EQ is boosted by 0.10dB less, And the Q value can only reach 0.70 not the 0.50 used inside protools. If exact values were possible it would very likely being an identical match. However the invert still shows a significant reduction of the control mix showing that there is a large amount of the audio that is identical and cancels out giving a reduction of up to -4dB in the best matched region. this level of difference should be very noticeable to someone actively listening for a difference between two piece of audio as it it changes in volume by a perceivable amount across the frequency spectrum most noticeable in the lower end. 

 

Logic X

Logic X’s mix match gave a result of +0.1dB on both the left and right channels for peak and -4.6dB Left with -5.1dB right for RMS, Presenting a marginally louder mix however anything less than plus or minus 1dB is considered negligible according to mastering engineer Bob Katz who has stated “As a rule-of-thumb, 1 dB is about the smallest level difference listeners can detect in a mix 0.2~0.5 dB in a laboratory setting” Bob Katz (2010)

While the increase in volume is clearly apparent when the mix from Logic X is inverted there is a significant drop in level of the two, And when the fader level of the mix from Logic is reduced by 3.6dB from the additional gain complete cancelation is achieved.

(Logic X EQ response comparison)

(Logic X EQ response comparison)

(Logic X EQ response comparison)

White = Control mix

Red = Reason mix

Yellow = Phase invert of reason mix against control mix

 

Logic X’s channel EQ GUI shows a very similar curve to the ones seen on the protools EQ and also the results are very similar with the red line being the frequency response of the mix produced via Logic X there is one small area that is more than 0.5dB difference around the 250Hz area however all other parts of a frequency spectrum are within less than 0.5dB of difference, these two pieces of audio while marginally different when measured in this way would not be able to be perceived as different even in laboratory conditions by human ear. the phase invert give roughly -6dB in reduction across the spectrum and full cancelation in the high frequency showing that the results from the Logic X Channel EQ are extremely similar to the one produced in protools. 

 

Fl Studio (OSX Beta)

Fl Studio (OSX Beta) Gave peak results of +0.2dB on left and +0.1dB on right with -4.6dB RMS on the left and -5.6dB on the right. a reduction of 3.4dB via the Protools mixer gives a level drop of over 55dB almost complete cancelation and does not show any areas of differing cancelation. Note that FL has a limiter set on the master by standard that increases the gain dramatically and limits the peaks as this is the default FL state upon opening the program i have also tested this.

 

FL Studio (OSX beta) Default Master. the peaks report out as 0.0dB as would be expected when applying a brick wall limiter to audio. the RMS levels are also slightly higher -4.0dB left and -5.2dB right. A user not knowing this is automatically being applied to their audio my unknowingly fall into the trap of feeling that FL produces sonically richer mixes from the perception of louder is automatically better.   

(EQ settings for FL studio EQ mix)

(FL studio EQ response comparison)

FL studio EQ response comparison)

White = Control mix

Red = Reason mix

Yellow = Phase invert of reason mix against control mix

(FL studio EQ compare close up)

FL studio’s EQ2 has added gain across the spectrum of between 0.5dB and 1.0dB as well as adding some other additional artefacts to the audio when compared to the protools EQ comparison. The boost in the high range covers a larger spectrum and so does the cut at 500Hz however the increase and increasing in dB across the spectrum in totally linear taken into account that there is already an additional increase added. due to the different range of spectrum covered and that the EQ is automatically louder very little cancelation happens between the FL studio mix and the control mix. This increase in loudness could possibly be perceived as it falls within the range where a change is perceived. This could be a considerable factor in deciding one DAW can in fact sound better than other on a purely listening basis, Because this particular instance the EQ has increased the level by a perceivable amount and the nature is to perceive the louder of the the two as the better. 

Reaper

Reaper gave a +0.1dB peak onto both channels with a -4.6dB Left for RMS and -5.1dB Right. Identical to Logic X’s meter readings however i was not able to achieve full cancelation via gain reduction to match protools, This is most likely again to the fact Reaper round to two decimal places and on the reference mix it was not possible to exactly match the values with the largest difference being 0.05 of a decibel. The reduction in volume however was between -40 to -45dBFS, With only around the 1 - 2Khz regions still slightly audible. A difference but not one that should be noticeable under even the most ideal of listening conditions.

 (EQ settings for Reaper EQ mix)

(EQ settings for Reaper EQ mix)

 (Reaper EQ response comparison)

(Reaper EQ response comparison)

Reapers EQ matches very closely with the control mix and with a tolerance of 0.5dB across the frequency spectrum. in the comparison shot the scale has been moved down by -10dB to show the complete phase cancellation at the high frequency interesting to note that the lower frequencies experience very little cancelation for the EQ adjustment being very slightly differing from that of the control mix, it would appear a different ratio is working on the HPF in the EQ from reaper causing this.  

Conclusion

There are consistent differences between the outputs from the DAW’s tested, All four of the DAW’s compared against the ProTool control version had increased level of around 3.2dB, This would be a significant difference when comparing the mix’s by ear. The +3dB increase could cause the listener to perceive it as better describing it as having more clarity as stated by Bob Katz(2010) “Louder always sound better” but as shown here once the additional gain is reduced the mix’s vary so insignificantly from each other that they cancelled at a reduction of almost 60dBFS which is considered silence.

Reaper and Reason cause the most difference but in both cases the mix is adjusted to two decimal places instead of the conventional one used by the other DAW’s however despite these differences of up to 0.05 per channel the amount of cancelation between them and the control mix shows such small differences it would be vary rare and difficult for a person to accurately tell the difference once the gain has been matched.

 

The differences can also be attributed to the way DAW’s handle the panning law, where possible linear was used however linear still has a ratio that is not user definable in the DAW and it set at a programers level. for example a mono signal at -20dB would be summed differently depending on the ratio set of the DAW if the ratio were to be -4.5 that -20dB mono would register -24.5 on both the left and right stereo outputs. This could also be the cause for one DAW’s printed audio file being louder than another. 

 

This has shown two key areas that people could mistake for a difference in the summing results of a DAW’s summing engine. Firstly the factor that some will sum louder or quieter than others leading to an instant change in the perception of how the result sounds, while it has been shown to be purely an increase in gain most likely attributed to the pan law of the of DAW and the ratios under which it operates. Secondly the fact some DAW designers have chose to work to a two decimal system for the decibel scale causing tiny differences between comparisons that can not be identically matched due to the rounding system found in the two examples. While there is intact a difference in the audio found upon phase cancelation it is to note that the difference are so small in level that they wouldn't be noticed by human ears even under laboratory conditions.  

 

The possibility however that the argument for one DAW to sound consistently better than another based purely on the summing engine and applied pan law is very unlikely. Most finished music or audio mix’s will have been treated in some way via the DAW’s plugins. most commonly used are equalisation (EQ) and compression, These are available natively in all the tested DAW’s so its fair to compare the channel EQ for the DAW’s against one another by matching boosts cuts and quality (Q) value across a singular piece of audio. Again using Protools as the control mix to reference against. The same process can the be applied to the single band compressor available with the DAW in question. After these tests results a combination of compression followed by EQ can be applied to ensure the result is not related to a sequence of effects processing.

The difference are apparent between each individual workstation and the fact they handle audio process in slightly different ways, Giving results that can be measurable via specific means but regularly fall within the limits where human perception would struggle or find it Impossible to accurately determine a difference. 

The way some DAW’s round up the sum of audio at output leading in some cases to increases of +3dB is the most likely contributing factor to the feeling of one DAW sounding better than another, As we perceive louder as automatically better without the prerequisite for critical listening. This can lead to the inexperienced or less informed to make instant conclusions. 

using protools as the control source was based on the fact that as a digital audio recording package it is the most high recommended among recording professionals. Which makes it interesting that it intact gave the lowest peak dB result, This does not mean it is any more accurate than the others compared but that instead it works on different pan law ratios and most likely rounds down to the nearest 0.0dB instead of up. As shown however when the gains where reduced on the other mix’s in most cases full or near full cancelation where achieved meaning that the audio is actually identical just louder.

To take this further it would be logical to take the audio generated by each DAW and play it under controlled conditions to a group of people who could then subjectively mark which sounded the most pleasing this would help more clearly determine what differences can actually be perceived between the different pieces if any and how the very minor differences in frequency and dB affect the listener if at all. 

A Broader variation of types of audio with loud peaks and low RMS, fast and slow transients should be tested to give a better representation of exactly how the summing and FX processing is effecting the audio.

 

The perception of one DAW sounding better than another clearly has reasons why it may be perceived by an individual as shown in the results here. With the amount of difference show in cases where rounding was not an issue however the results would largely be down to the preference of the individual as apart from the increased final level the audio was the same.

 

Scott Robinson (Unders)