A great deal of scrutiny is necessary when police and fire departments look to promote individuals to higher rank. The process varies across the country and there is no standard solution, no dominant best practice. Labor agreements, state and local laws and public accountability all factor into the practices adopted in the promotional process of public safety professionals. As our agencies face financial pressures and demands for transparency, those of us in the testing procession find ourselves at a crossroads. Are the resource intensive evaluation processes defined by an assessment center necessary today, or are there better ways to promote police officers and firefighters?

Promotional Exams generally consist of three elements, a written test, an oral interview board and in many cases an observation in an “Assessment Center.” An Assessment Center is broadly defined to as a standardized evaluation of behavior based on multiple evaluations including: job-related simulations, interviews, and/or psychological tests. Job Simulations are used to evaluate candidates on behaviors applicable to the most critical aspects (or competencies) of the job. When it comes to police and fire, this evaluation increasingly has grown to be a live simulation. In the case of an assessment center for police sergeant as an example, typical duties include employee-counseling, shift-meeting, and citizen-interview exercises. An observation of the candidate in any of these situations can be indicative of how that candidate may perform in the actual situation and thus an Assessment Center recreates the setting and sees the candidate in action.

When Properly designed and administered, assessment centers can come close to real-life and provide keen insight into a candidate while under pressure. While there is not a great deal of empirical data to support the reliability of the assessment center method, its success may be demonstrated by the growing number of police agencies that have chosen it over other testing methods. In the case of fire departments, the assessment center that contains a “burn building” puts the candidate under real danger so that judges may evaluate supervisory, managerial, and administrative potential under stress. Assuming the assessment center includes a thorough job analysis of the position being evaluated, and that the proper guidelines were following during its design and administration, assessment centers can be easier to defend if challenged assuming the judging panels provide age, gender and ethnic diversity.

Assessment centers are designed to test what a person can do, not just what they know. A candidate may have high intellect and be well educated, but since most positions being tested are supervisory, the assessment center seeks to assess make critical decisions under pressure and display leadership ability. This one-time view of the candidate can be insightful but is not perfect. As with any process involving humans, judging has its shortcomings and professionalism is defined as 24/7, not just a staged performance under the lights.

Assessment Centers are expensive, costing much more than traditional testing. An industry average seems around $2,000 for each candidate to administer an assessment center. The more rigorous the test, the higher the cost. Often assessment centers involve vendors transporting elaborate sets and staging areas to the district, or candidates must travel to existing training and testing locations. Candidate instructions and preparation may be inconsistent. Some candidates may have already been exposed to the center from a previous attempt at promotion, been coached by other candidates or may have come from another district’s evaluation which was similar. Assessment center role players perform inconsistently or may be absent, affecting the outcomes. Judges/ Assessors may be biased and may fail to conduct themselves in a professional manner and may have bias. The assessment scenario may be far from the actual duties of the job and therefore may be only addressing one dimension of the candidate. A candidate may possess the ability to perform, the assessment center has no relevance to the willingness or motivation to do the all the aspects of the job.

Without an Assessment Center, a review process will require a strong review process which seeks to understand, and quantify the skills, knowledge, experience, temperament and overall professionalism of the candidate. An assessment center does not consider recommendations, degrees, special training and assignments. This is where strong written tests and oral review boards come in. Those in supervisory roles need to know the rules, need to act deliberately and in most cases, have time to prepare, just like they would for written tests. An oral review board can be viewed as the court of public opinion or an actual jury providing insight into the candidate under scrutiny. Applying both knowledge obtained on the job and knowledge from training and educational material is the goal of the written exams and oral review boards. Review boards can be recorded, just like an assessment center and in many aspects, the observations made through questioning may be as insightful as observations made through role play.

As they say, the devil is in the detail. Three measures (written test, oral review board and assessment center) are usually better than two (written test and oral review board); however, three inconsistent measures done poorly is worse than two very well-done measures. In today’s tight budget environment, the cost-benefit analysis should include quality of the process and quality and integrity of those involved in the process. In today’s competitive and litigious environment, the promotional process should be well thought out and your processes designed to have no bias. it’s worth doing right and any attempt to do an any part of the promotional process “on the cheap” is likely to produce unsatisfactory results.

While this cost can be diminished if you are able to do it “in-house” without using a consultant or contractor, you put your process under additional scrutiny with regards to bias. A fresh, outsider view from an organization that specializes in the assessment process is a prudent investment. We hear often of retired chiefs taking on this role, but do the retired chiefs deliver written tests, oral review boards or even assessment centers build on data and past-performance? We at McCann Associates use both current and retired chiefs on our boards, but we apply modern test practices to our work. The old adages of “time is money” and “good service isn’t cheap, cheap service isn’t good” do apply in developing a sound and defendable promotional process. Because in the end, “you can pay me now, or pay me later” so we recommend you invest in a good process up front, so you can avoid legal, political and public relations problems later.