Skip to main content
blog

How to Choose a Salary Survey: A Compensation Pro’s Working Checklist

Key takeaways

  • Salary surveys differ wildly in quality, methodology, sample, and fit for purpose. Buying based on brand or convenience produces benchmarks that don’t hold up.
  • Seven properties matter most: methodology rigor, sample depth in your functions and geographies, job-match documentation, refresh cadence, peer participation, output flexibility, and total cost.
  • Most enterprise programs need 2-4 surveys: a primary general industry survey, a tech/specialized function survey, a peer-group survey, and optionally a geographic or industry-specific specialty.
  • Five common selection failures: buying what peer companies use, buying the cheapest option, buying every survey, not pressure-testing samples, and not auditing matches.
  • A salary survey is a multi-year investment shaping every comp decision. Pressure-test it like one.

Survey selection rarely gets treated as a strategic decision. Most programs inherit their survey portfolio from whoever ran comp before them, add a new one when a peer company mentions it, or default to whatever shows up first in a Google search. None of those is a methodology. All of them produce the same result: benchmarks built on a data foundation nobody has actually evaluated.

A salary survey is a multi-year investment that shapes every range, every offer, and every equity analysis the comp program produces. This checklist is what to actually evaluate before writing the check.

The hidden truth about salary survey selection

Most salary survey purchases happen for one of three reasons: it’s what the company already has, it’s what a peer company recommended, or it’s what came up in a Google search. None of these is a defensible reason to invest five or six figures in a data source.

The wrong survey produces benchmarks that don’t fit your workforce, ranges that don’t survive transparency scrutiny, and equity analysis built on mismatched inputs. The right survey is the foundation of a defensible comp program. This is the practitioner’s working framework for evaluating a survey portfolio with the seriousness the decision deserves.

What a salary survey is

A salary survey is a structured collection of pay data from participating employers, organized by job, level, industry, geography, and size. Participating employers submit data in a defined format. The survey provider compiles, normalizes, and reports back the results. Participants get access to the report. Non-participants typically can’t.

The participation model matters. The best surveys are participant-only because data quality depends on willing, structured contribution. Non-participation models where you can buy access without contributing tend to have weaker data quality.

Different surveys cover different scopes. General industry surveys offer broad coverage across functions and industries. Industry-specific surveys focus on a single vertical like tech, financial services, or healthcare. Function-specific surveys go deep on engineering, sales, or executive roles. Geographic surveys focus on specific markets or regions. Peer-group surveys focus on a defined set of comparable employers.

The right mix depends on your workforce composition, which is why the evaluation process matters more than picking from a ranked list.

The seven properties that matter

Seven properties separate a survey worth participating in from one that’s expensive table stakes. Strong surveys score well across all seven. The buying decision is mostly about identifying which surveys actually have all seven, not picking between trade-offs.

1. Methodology rigor

How is the data collected? How are jobs matched and validated? How is data normalized? How are outliers handled? A rigorous methodology is documented and consistent. A weak methodology is “we have lots of participants.”

Survey providers with strong methodology can walk through their process clearly in a thirty-minute conversation. Providers with weak methodology pivot to talking about dataset size. That pivot is the tell.

2. Sample depth in your functions and geographies

Total dataset size matters less than sample depth in the functions and geographies you actually employ in. A 500,000-record dataset that’s thin on your specific roles is worse than a 50,000-record dataset that’s deep on them.

Ask for sample counts at the cuts you’d actually use. A provider that can produce those counts on demand is signaling methodology transparency. One that can’t is signaling something else.

3. Job-match documentation

Strong surveys provide detailed job descriptions, scope definitions, and level criteria for every survey job. Match becomes a defensible decision rather than a judgment call.

If the job descriptions in the survey are vague, your matches will be too. The quality of the job-match documentation is the upstream determinant of benchmark quality, and most buyers don’t pressure-test it before signing.

4. Refresh cadence

How often is the data collected and published? Annual is the historical standard. Some surveys are now twice-yearly or continuous. For fast-moving markets like tech, AI, and specialized expertise, the cadence matters considerably more.

Refresh cadence interacts directly with your aging methodology. A survey that refreshes annually needs aggressive aging to stay current. A survey that refreshes more frequently produces benchmarks closer to current market reality from the start.

5. Peer participation

Does the survey include the employers you actually care about: your direct competitors, your aspirational peers, your hiring market? If the participant list doesn’t include the companies you’d benchmark against, the data is less useful regardless of overall quality.

Ask for the participant list. Reputable providers will share it in at least summary form. Surveys without strong peer participation in your relevant cuts are surveys producing data that doesn’t reflect your competitive reality.

6. Output flexibility

Can you slice the data by the cuts you need: industry, geography, size, function, level? Can you produce aged data, blended cuts, custom percentiles? Or are you locked to a fixed set of canned views?

A rigid survey forces your methodology to match its limits. A flexible survey supports your methodology. The flexibility shows up in pricing, but the cost is usually worth it in extraction time saved across a full planning cycle.

7. Total cost

Survey costs include participation time, the work to submit data, license fees, and comp team time spent extracting useful results. The all-in cost varies dramatically. Cheap surveys with rigid output often cost more in extraction time than expensive surveys with flexible output. Price the full investment, not just the license.

How many salary surveys does an enterprise program need?

Most enterprise programs benefit from two to four surveys. The structure tends to look similar across programs regardless of industry.

  • A primary general industry survey provides broad coverage across functions and comprehensive geography. 
  • A primary tech or specialized function survey adds depth in high-movement functions. 
  • A peer-group survey provides direct competitive intelligence among the companies that actively hire from and to your workforce. 
  • An optional geographic or industry-specific specialty survey fills the gap if there’s significant exposure to a non-standard market.

More than four surveys produces redundancy without adding signal. Fewer than two produces single-source risk where benchmarks are only as good as one provider’s methodology and sample.

Where survey selection goes wrong

Survey selection fails in predictable ways, and the failures share a common root: skipping deliberate evaluation in favor of a faster shortcut. Five patterns come up consistently.

  1. Buying the survey a peer company uses. Their workforce isn’t your workforce and their needs aren’t yours.
  2. Buying the cheapest option. Survey cost is a small fraction of the total cost of bad benchmarks. Buying down on the survey usually costs more in downstream remediation.
  3. Buying every survey in the category. The fifth survey adds noise more than signal. Diminishing returns set in well before most programs realize it.
  4. Not pressure-testing the sample. Looking at total record counts rather than sample depth in the specific cuts that matter to your program.
  5. Not auditing the matches. Trusting vendor match recommendations without comp team review produces benchmarks built on assumptions nobody has verified.

When most of these are true simultaneously, the survey portfolio is producing benchmarks that look defensible until someone applies real pressure.

A working framework for salary survey selection

Six steps to convert the survey decision from inheritance to deliberate choice. They have to happen in sequence because each one informs the next.

1. Define what you need

List your top functions, top geographies, and top peer companies. Identify where current benchmarks feel weakest. The gap analysis is what determines the right surveys, not vendor reputations or relationship longevity.

The gap analysis should answer three specific questions: which functions consistently produce benchmarks the team doesn’t fully trust, which geographies have thin data underneath the benchmarks, and which peer companies aren’t well-represented in current surveys.

2. Get sample data

Request sample reports for your most critical roles in your most critical geographies. Compare across providers. The differences in match quality, level definition, and statistical depth are usually obvious once you’re looking at them side by side.

Sample data is more revealing than any vendor pitch. A survey that can produce a defensible benchmark for your hardest role in your hardest geography is worth serious consideration. One that can’t won’t hold up at scale.

3. Audit the methodology

How is data collected? How are jobs validated? How is normalization handled? A thirty-minute conversation with the survey provider’s methodology team tells you more than a thirty-page brochure.

The conversation should cover data collection validation, job-matching protocol, outlier handling, and minimum sample size requirements for reportable cuts. Providers with strong methodology answer these questions clearly. Providers without it pivot to talking about dataset size or client logos.

4. Check the participant list

Are the companies you care about participating? Pay particular attention to direct competitors you actively compete with for talent, aspirational peers whose pay practices you want to track, and the companies that hire from and to your workforce. If those employers aren’t in the survey, the data won’t reflect the market you’re actually competing in.

5. Negotiate output flexibility

Survey providers will often offer custom cuts, aging support, or extended report access if asked. The flexibility is more valuable than a discount on the base license. Ask before signing.

6. Plan participation seriously

Submitting data is a real investment of comp team time. Quality submission produces quality data for everyone, including you. Plan for the participation work before the contract is signed, not after.

How to implement a survey portfolio refresh

Phase one: portfolio audit (months one to two)

Document the current survey portfolio. For each survey, assess how it’s being used across functions, geographies, and the decisions it informs, what the actual quality of the output is in terms of sample depth, match documentation, and refresh cadence, what the total cost is including license, participation time, and extraction time, and whether the survey is still earning its place or has become legacy infrastructure nobody wants to retire.

Phase two: gap and overlap analysis (months two to three)

Identify gaps where current surveys don’t provide adequate coverage. Identify overlaps where multiple surveys provide redundant data without adding signal. Build the case for additions, removals, and renegotiations before approaching any vendor.

Phase three: selection and transition (months four to six)

Evaluate candidate surveys for the identified gaps using the six-step framework. Renegotiate or sunset surveys that aren’t pulling their weight. Plan the transition to the refreshed portfolio including participation logistics and any downstream workflow updates the comp team needs to make.

What tends to go wrong

Five failure modes come up consistently in portfolio refresh projects, and most of them trace back to letting relationships or habits override the evaluation.

  1. Skipping the gap analysis. Refreshing the portfolio without identifying actual gaps produces a different portfolio, not a better one.
  2. Buying based on relationship. Long-standing relationships with survey providers can override the objective evaluation in ways that are hard to see clearly from inside the relationship.
  3. Underestimating participation cost. Submitting data well takes significant comp team time. Programs that don’t plan for it end up submitting poor data, which weakens everyone’s results including their own.
  4. Ignoring methodology transparency. Providers that resist methodology questions are telling you something important. Take the signal seriously.
  5. Treating selection as a one-time decision. Survey quality drifts over time. Periodic re-evaluation, every two to three years at minimum, is necessary to keep the portfolio earning its cost.

Measuring survey portfolio effectiveness

A deliberate survey portfolio and an inherited one can look similar from the outside until you dig into the numbers. Here are five metrics worth tracking to tell them apart.

  1. Coverage rate by function. What percentage of your workforce has adequate survey coverage in their function? Gaps here are direct inputs to the next gap analysis.
  2. Sample depth at your relevant cuts. Sample counts at the geography, function, and level cuts you actually use in planning. Low counts at the cuts that matter are the clearest signal a survey isn’t earning its place.
  3. Match quality across portfolio. Match defensibility for your top functions across each survey. Weak match documentation in a survey the team relies on is a risk that compounds over time.
  4. Total cost of ownership. License plus participation time plus extraction time. The all-in number often looks different from the license fee alone.
  5. Benchmark trust score. The comp team’s qualitative assessment of how much each survey can be trusted for each function. If the team has low confidence in a survey’s output for a critical function, that’s a data point worth acting on.

Building your survey portfolio strategy

For programs that have inherited their portfolio and never formally re-evaluated it, a six to nine month project covers the ground. Here’s what that looks like in practice.

  1. Audit the current portfolio across use, quality, cost, and necessity before touching anything else.
  2. Run the gap and overlap analysis to identify where coverage is thin and where it’s redundant.
  3. Evaluate candidate additions using sample data, methodology audit, and peer participation check rather than vendor reputation alone.
  4. Renegotiate or sunset legacy surveys that aren’t pulling their weight. Keeping surveys out of inertia is a cost the program pays in benchmark quality every cycle.
  5. Plan the transition carefully including participation logistics, comp team training, and any downstream workflow updates that need to happen before the next planning cycle.

The survey is the foundation. Treat it like one.

Salary survey selection is one of the highest-leverage decisions a comp program makes, and it gets treated like a procurement checkbox more often than not. The wrong surveys produce benchmarks that don’t survive transparency, equity scrutiny, or audit pressure. The right surveys produce a foundation the rest of the program can build on.

The seven properties, methodology, sample depth, match documentation, refresh cadence, peer participation, output flexibility, and total cost, are the working checklist. Run any survey purchase through it deliberately. Most TR teams inherit their survey portfolio. Few re-evaluate it. The teams that do, every few years and methodically, consistently produce stronger benchmarks than the teams that don’t.

Want to see what market pricing looks like when it’s built to get the most out of your survey data? See how Bettercomp approaches it.

Learn why the “why” behind your data sources matters more than ever for a resilient compensation strategy.

Frequently Asked Questions

A salary survey is a structured collection of pay data from participating employers, organized by job, level, industry, geography, and size. Participants submit data in a defined format and the survey provider compiles and reports back. Salary surveys are the primary source of external market data for compensation programs.

A salary survey is the data source. A salary benchmarking tool is software that consumes survey data and produces benchmarks. Some vendors provide both. Most modern programs use survey data from one set of providers and benchmarking software from another.

Enterprise salary surveys range from $5,000 to $50,000 or more per year depending on coverage, geography, function, and participation status. Participants typically pay less than non-participants. The license fee is a small fraction of the total cost. The larger costs are participation time and comp team time spent extracting useful results.

It depends on the specific function mix, geographic spread, and peer group. There’s no single best survey for tech. Most enterprise tech programs use a combination of general industry, tech-specific, and peer-group surveys to triangulate. The right mix is the one that produces strong sample depth in your specific functions and locations.

Annual is the historical standard. Some surveys are now published twice yearly or continuously. For fast-moving markets like tech, AI, and specialized expertise, more frequent refresh cadence is increasingly important to keep benchmarks current.

Participate when possible. Participation provides better pricing, better data fit because your data is included, and often better support from the provider. Non-participation access is available for many surveys but typically costs more and produces lower data quality because the contributing companies have less stake in methodology rigor.

Most enterprise programs benefit from two to four surveys: a primary general industry survey, a primary tech or specialized function survey, a peer-group survey, and optionally a geographic or industry-specific specialty. More than four produces redundancy without adding signal. Fewer than two produces single-source risk.

A thirty-minute conversation with the survey provider’s methodology team tells you more than a thirty-page brochure. Ask about data collection validation, job-matching protocol, outlier handling, and sample size policies. Providers with strong methodology answer these questions clearly. Providers without it pivot to talking about dataset size.

A participant survey is available only to employers who contribute their own data. A non-participant survey can be purchased without contributing data. Participant surveys tend to have higher data quality because contributors have a vested interest in methodology rigor. Non-participant surveys are more accessible but typically produce weaker benchmarks.