Author: Dr. Hunt

Choosing an assessment vendor can be a confusing process. There are literallyhundreds of vendors selling all kinds of products to help you hire employees.The majority of these vendors will assure you that their products are perfectfor your company or can be easily modified to fit your needs. While theappropriate use of assessment tools can provide significant ROI, using the wrongtool can result in substantial losses of time and revenue. The key to ensuringthat you reap the benefits of assessment tools while avoiding the risks is touse a well-planned and thorough vendor-selection process.

    Effective vendor selection requires three basic steps:

Step 1: Create a team and a process
    Unless you are addressing a very simple and straightforward staffing need, itis a good idea to create a formal team and project plan to oversee thedeployment of assessment tools. The functions of this team include determiningassessment tool requirements, researching and gathering information fromvendors, reviewing and evaluating vendor solutions, and outlining processes forongoing use and support of assessment tools once they have been implemented. Theteam should include representatives from the major stakeholder groups that willbe affected by the use of assessment tools.

    In addition to HR, consider including personnel from IT, legal, andoperations. It is also useful to include someone with expertise in assessmentdesign and validation to help provide guidance during the process.

Step 2: Perform a needs analysis
    Before speaking with any vendors, clearly define your reasons for usingassessment tools. Outline specific business issues you wish to address and theoutcomes you expect. Consider operational issues such as where you plan to useassessments, how the assessments will be administered, how you will interpretthe results and handle assessment data, and who will provide ongoing support andtraining for the assessment tools. Define constraints such as budgets, personnelresources, and technology requirements.

    This needs analysis should clearly define what the assessment tools must doin order to be successful. Failure to conduct a good needs analysis will resultin vendors telling you what you need instead of allowing you to be in control ofthe process.

Step 3: Conduct a formal request for proposal (RFP)
    The assessment-tools market is rapidly changing, with new solutions beingintroduced virtually every month. It is always a good idea to get proposals froma range of vendors before deciding on any single solution. The RFP that you sendto vendors should detail the specific business objectives you want to achieve,which jobs you want to support with assessment tools, the number and location ofcandidates, and any technology requirements.

    The RFP should also ask vendors to supply some very specific information thatwill help you select the best one for your needs. The following 10 questions canbe used to evaluate a vendor’s solutions and their ability to help you meetyour goals.

1. What assessment techniques do you provide? It is important to understandthe types of assessments offered by a vendor. Do they sell primarilyqualifications screens, knowledge tests, talent measures, or background checks?Explore which assessments reflect their core capabilities and which ones theyoffer through partnerships or outsourcing. Ensure that the assessments offeredby the vendor align with your business objectives.

2. How do you determine what are the most critical aspects of jobperformance? To provide effective and legally sound assessment tools, a vendormust be able to clearly define the critical factors that influence job success.This process of defining job performance is commonly known as job analysis. Manyvendors will cut corners in this up-front work, so it is important that theyclearly demonstrate the processes they use to develop links between theirassessment tools and job performance.

3. What validity data can you provide for this product? Validity dataprovides the proof that an assessment tool actually predicts job performance.All assessment vendors should have summaries of validity data that are readilyavailable to prospective clients. This data should provide evidence of thetool’s performance on jobs similar to those for which you plan to use it. Lookfor “hard numbers” linked to well-defined measures of performance (e.g.,supervisor ratings, tenure).

    Avoid vendors that rely heavily on vague anecdotal statements about theirtool’s perceived effectiveness and impact. Ask for references from specificclients that you can contact to learn more about the effectiveness of theirassessments.

4. Do you have evidence regarding the legal defensibility of this tool? Youhave a right to know if a vendor’s product has ever been challenged in court.It is also important to determine if the tool displays adverse impact (i.e.,members of certain protected classes do not perform as well on it as members ofthe majority group).

5. How much consulting and customization is required to get your systemconfigured for our organization, and how much time will this take? Some systemsare built so that they can be used “off the shelf” with very littlecustomization, while others require more time and effort to configure. In mostcases, taking time to create a customized system offers advantages in accuracyof prediction and higher levels of legal defensibility. However, it alsoincreases development costs.

6. What delivery methods are available? Paper and pencil, telephone, and theInternet are the three main ways to deliver assessment tools to applicants. Manycompanies offer all three options, but differ widely in their level of expertisewith each method. The technology around telephone and Internet screening can bea major source of problems, but it can also offer significant benefits incomparison to paper-and-pencil delivery methods. It is a good idea to askvendors for specific examples of using similar technology to deliver theirassessment tools.

7. How easy will the results be for recruiters and HR personnel to interpret?You want a system that provides high-level results that are easy for non-expertsto understand but also allow in-depth information for the purposes ofdocumentation and more detailed investigation. Some assessments can even be usedfor both candidate evaluation and development of newly hired employees.

8. Does the system consider the needs and feelings of the test taker? Theexperience of applicants is a critical factor in defining the success of anassessment tool. Upsetting applicants is bad recruiting and can lead to legalaction. What steps has the vendor taken to ensure that candidates will perceiveits tools to be job relevant and culturally unbiased?

9. How is the system priced? It is important to clearly understand the feeschedule as it relates to the manner in which you plan to use the test. Askvendors to outline up-front fees for system design and configuration, as well asongoing usage fees.

10. What ongoing support will you receive? What services does the vendorprovide to ensure the ongoing effectiveness of its assessment tools? Does itprovide quarterly or semi-annual reviews of the performance of its assessmenttools, including EEOC summaries? What sort of ongoing customer and help-desksupport is offered? You want to look for vendors that will provide activeongoing support and will not simply disappear after you have bought their tools.

    You should also consider the financial and personnel stability of thecompany. Some assessment companies depend heavily on the expertise of one or twokey individuals. Try to find out if this is the case, and ask what transitionplans the companies have in place should these individuals leave.

    In general, the RFP process should be approached as a dialogue, not aninterrogation. Most assessment vendors are ready and willing to respond toformal RFPs. However, be respectful of the time it requires to respond to yourquestions, and do not ask for solutions that you are not serious aboutimplementing. It is also helpful to let vendors know in advance about what sortof budget constraints you have. It will save both parties a lot of time, and youmay be pleasantly surprised by the alternative solutions they will propose.

Workforce Online, December 2002 — Register Now!

It’s tough to determine the precise return on investment provided by theuse of assessment tools. Though it’s problematic to estimate the actual valueof a good hire–or the cost of a bad one–there are formulas available that willhelp.

    Such tools are most effective for jobs with large annual hiring volumes orhigh levels of turnover, or in cases in which the financial value of performanceis high. That’s the amount of revenue generated by a single employee, a figureoften assumed to be about 2.5 times the average employee salary.

    In general, staffing-assessment tools are used for jobs with high turnover orgrowth, or where the difference between low and high employee performance has amajor impact on the company’s bottom line. Many of the jobs that meet thesecriteria are either high-volume hourly positions in manufacturing, retail, orcustomer service, or high-impact positions such as first-line supervision, storemanagement, sales, information technology, and senior leadership.

Determining how much to budget for the use of assessment tools
    Costs for staffing-assessment tools fall into two general categories: initialsetup costs and ongoing or “per usage” fees. Setup costs depend primarily onthe nature and design of the assessment tools, whether customization will berequired, what technology will be used, the need for hiring-manager or recruitertraining, and the number of candidates to be assessed. For common jobs thatdon’t require highly specialized skills, it may be possible to set up a simplebut effective “off the shelf” assessment system for less than $20,000.

    On the other hand, a highly tailored, validated, Web-enabled assessmentsystem can easily exceed $500,000 in development and deployment costs. When usedto support high-volume or high-impact positions, these tailored systems quicklyreturn this initial investment through increased employee performance, tenure,and staffing efficiency.

    Vendors offer a wide variety of costs and pricing structures around ongoingor “per usage” fees. Fees typically range from as little as $3 per candidatefor simple prescreening measures to more than $500 per candidate for jobsimulations or talent assessments. Unlimited-use licenses are also an option forhigh-volume staffing processes. Per-usage prices are constantly changing as themarket evolves, and ultimately the only way to ensure a fair price is to shoparound.

ROI to expect from the use of assessment tools
    Most companies do not track data at the level of detail needed to compute theROI generated by staffing-assessment tools. This includes how differences inemployee performance affect profitability, the true cost of turnover caused bywork-flow disruptions, and loss of intellectual capital, or staffing costs.Without this data, it is not possible to calculate the exact value of assessmenttools. Nevertheless, some estimate of ROI is needed to generate a business casefor the use of assessment tools. The following spreadsheets can help inestimating four types of ROI provided by assessment tools:

1. The value of better hiring decisions

   
   

2. The value of avoiding “catastrophic” hires

   
   

3. The value of reduced turnover

   
   

4. The value of increased staffing efficiency

   
   

    The spreadsheets include some “industry standard” estimates in case youdo not have access to all the data required to compute the ROI.

Spreadsheet #1: ROI Provided Through Better Hiring Decisions
    The main value of assessment tools comes from improving the averageperformance of newly hired employees. Imagine the impact if the averageperformance of each employee in your company improved by 5 percent. Extensiveprivate and public research has shown that well-designed staffing-assessmenttools can provide such results.

    The following spreadsheet estimates the potential ROI to be gained by usingstaffing-assessment tools to increase employee performance. Note: The ROIestimates provided by this spreadsheet may seem unrealistically large. However,they accurately reflect the long-term impact that assessment tools can have byimproving the average performance of an entire workforce.

    You will need the following data to use this spreadsheet:

Hires (N)
The number of people hired per year due to growth and turnover for theposition(s) for which you are using assessment tools.

Tenure (T)
The average number of years that employees work in the position(s). Indicatetenure using decimal values (e.g., if the average tenure is 9 months, set thisvalue at .75).

Value of High Performance (Zx)
The differences in revenue generated by high- versus low-performingemployees. This is commonly set at 40 percent of the average employee salary.

Increased Hiring Effectiveness (rxy)
An estimate of how much the use of assessment tools will improve the qualityof hiring decisions. Effectiveness ranges from 0 (random hiring) to 1.0 (perfecthiring).

    Accurately calculating hiring effectiveness is a highly complex mathematicaltask. However, the table below can be used to calculate a very rough estimatefor this value.

    To calculate Increased Hiring Effectiveness, subtract the effectiveness ofthe assessment methods you are currently using from the assessment methods youare considering deploying. For example, if you currently use UnstructuredInterviews (effectiveness of .02) but are considering using measures of Ability(effectiveness of .25) and Personality (effectiveness of .15), then theIncreased Hiring Effectiveness would be equal to (.25 + .15) – .02 = .38.

    Note: This table provides only assessment tools designed to predict superiorperformance. It does not include assessment tools that add value primarily byreducing administrative time (e.g., qualifications screens) or reducing the riskof catastrophically bad hires (e.g., background verifications).

Per Usage Assessment Cost (Cy)
    How much the use of assessment tools will increase the cost of evaluatingcandidates. Rough cost estimates are provided in the table above. The costs ofinterviews in this table are associated primarily with time spent by recruitersand hiring managers conducting the interviews.

Selection Ratio (SR) (default value: 5)
    The number of candidates you typically assess before making a hiringdecision. It is usually somewhere between 3 and 10. If you do not have thisstatistic, we suggest setting it at 5.

Formula 1: ∆$performance = [(N) (T) (rxy) (Zx)] – [(N)(Cy)(SR)]

Spreadsheet #2: ROI Provided by Avoiding Catastrophic Hires
    Although good employees can be a company’s greatest assets, the wrongemployees can be a company’s largest liabilities. Certain assessment toolssuch as drug screens and background checks reduce the risk of hiring employeeswho may engage in counterproductive activities such as theft, violence, orsabotage. The following spreadsheet estimates the savings associated with usingassessments to avoid hiring individuals who are likely to engage incounterproductive activities. You will need the following data to use thisformula:

Hires (N)
The number of people hired per year due to growth and turnover for theposition(s) for which you are deploying the assessment tools.

Cost of Bad Hire (CBH) (default value: $7,500)  
The average loss incurred by hiring an employee who engages in theft or othercounterproductive behaviors. It should include legal and security fees incurredas a result of counterproductive behavior. We have conservatively set this valueat $7,500, based on retail theft statistics.

Percentage of Catastrophic Hires Avoided (HA) (default value: .05)
The percentage of candidates screened out through background checks who wouldhave engaged in employee theft had they been hired. Industry statistics suggestthat around 10 percent of background verifications uncover somethingsubstantially negative about candidates. If we estimate that half of thesecandidates would in fact engage in counterproductive behavior, this value can beset at 5 percent.

Assessment Cost (Cy) (default value: $45)
How much the use of assessment tools will increase the cost of evaluatingcandidates. Most background checks and drug screens cost between $25 and $100,with an average probably around $45. It is assumed that these assessments areconducted late in the staffing process as a final check prior to employment.

    Saving due to Assessment = (N * CBH * HA) – N*Cy

Spreadsheet #3: ROI Provided by Reduced Turnover
    Many assessment tools are specifically designed to help organizations reduceturnover. When designed properly, these tools can reduce turnover by 10 percentor more. This spreadsheet estimates the financial value of using assessmenttools to reduce turnover. You will need the following data to use this formula:

Hires (N)
The number of people hired per year for the position(s) for which you aredeploying the assessment tools.

Annual Turnover Rate (TR)
The percentage of the workforce that currently leaves each year due toturnover.

Average Time to Fill (TF)
The average number of weeks required to fill a vacant position.

Value of Performance (VP)
An estimate of the cost of the annual revenue generated by employees in thisposition. This is commonly set at 2.5 times the average employee salary.

Hiring Cost (HC) (default value: $3,000)
The average costs associated with hiring an employee. These include timespent by recruiters and managers sourcing and screening candidates, time andexpense invested in training new hires, and any coming-on-board costs such asrelocation or orientation. Industry studies place typical hiring costs at $3,000for hourly employees and $10,000 for exempt employees. We have set the defaultvalue at $3,000.

Assessment Cost (Cy) (default value: $30)
How much the use of assessment tools will increase the cost of evaluatingcandidates. The cost of assessment measures specifically designed to reduceturnover commonly ranges between $10 and $75, depending on the design. Anaverage cost might be set at $30.

Selection Ratio (SR) (default value: 5)
The number of candidates you typically assess before making a hiringdecision. It is usually somewhere between 3 and 10. If you do not have thisvalue, we suggest setting it at 5.

    Value of Reduced Turnover = (N * TR * ((TF * VP)/52) + HC) * .10) – (Cy *SR * N)

Spreadsheet #4: ROI Resulting from Reduced Administrative Costs
    Perhaps the most visible short-term benefit of using assessment tools is thereduced administrative time spent screening candidates. The followingspreadsheet gives a very general estimate of ROI provided by reducingadministrative costs.

    You will need the following data to use this formula:

Hires (N)
The number of people hired per year for the position(s) for which you aredeploying the assessment tools.

Time Spent with Candidates (TC)
Average number of hours spent evaluating candidates who are not hired. Thisincludes time spent reviewing résumés, coordinating and conducting interviews,and holding recruitment conversations.

Average Cost of Recruiting Time (RT) (default value: $50)
Estimate of the hourly cost of recruiters and hiring managers who spend timewith unqualified candidates. An average cost for this factor might be $50 perhour.

Hiring Cutoff (HC) (default value: .50)
The number of people who will pass the assessment tool you use. The actualnumber depends on a lot of factors, but might be set at 50 percent for thepurpose of this exercise. This would mean that the assessment tools would screenout half of the candidates because of failure to meet key job requirements.

Assessment Cost (Cy) (default value: $20)
The degree to which the use of new assessment tools will increase the costsassociated with assessing candidates. Most reductions in administrative costscome from the use of relatively inexpensive prescreening questionnaires. Anaverage cost for these might be set at $20.

    Selection Ratio (SR) (default value: 5)

    The number of candidates you typically assess before making a hiringdecision. It is usually somewhere between 3 and 10. If you do not have thisvalue, we suggest setting it at 5.

    Value of Reduced Admin = (N * TC * RT * HC * SR * .50) – (Cy * SR * N)

Workforce Online, March 2003 — Register Now!