Geo differentials don’t align to market data – Why Not?

Question: (original question from WorldatWork)

I have found that the geo-differential from Seattle might suggest that jobs, as a whole, in Seattle pay 10%-15% above the national market. However, when I ran a comparison of multiple surveys to identify the differential between the Seattle cuts and the National cuts, I found that the median differential, is much lower than the suggested geographic differential, overall.

  • Radford suggested that the Seattle cut vs the National cut had a median differential of only 1%
  • Culpepper suggested the Seattle cut vs the National cut had a median differential of about 7%

Is anybody able to provide some insight as to why these numbers are significantly less than what I have seen in terms of what the geographic differentials suggest?

Answer by Sam Reeve:

There are many factors that could lead to different geographic pay differential results between surveys.

Such as:

  • Different Data Sample.  Just about every survey out there will have a different group of companies that provide compensation data.  They may be from different industries, company sizes and some jobs may be more dominant than others.
  • Different Data Collection Methods.  Every survey has their own survey job descriptions and therefore companies will need to make a judgement call on how to match their roles to the survey.  In addition, not all companies provide quality information and it is difficult for survey companies to determine the good information from the bad.
  • Different Survey Methodology.  In school we learn the right way to use statistics and then some survey companies come up with their own creative spin on the science.  (I am not pointing any fingers here.)  Whether a company uses sound statistics or not, different survey methodologies with yield different results.
  • May have Different Definitions for Pay Elements.  Some survey companies may also define pay elements differently such as base pay versus fixed pay although, to my knowledge, this should not be an issue between Culpepper and Radford.  This is typically a concern with international surveys.

This doesn’t mean that surveys are all bad, it is just a fact that they are not a perfect representation of reality.  The bottom line is that the geographic differential process is inherently flawed.  It is not meant to be perfect.  It is meant to be a cheaper and less administratively burdensome than direct market mapping. (meaning, purchasing local surveys and mapping local jobs)  The key is to find the best source that works for your company and keep consistent with your methodology.  Understand that it is not perfect and check your employee attraction and turnover data to identify problems and adjust as needed.