I
tried one of those First Generation DISC-type instruments
and the report really seemed to describe me pretty
well. Doesn't that mean the test is a good one? Not
necessarily. All personality tests work to some degree.
The question for you is whether the information is
specific enough to drive serious business decisions.
Many simple instruments rely on a psychological experience
known as the "P.T. Barnum Effect." Experiments have
demonstrated that when personality reports are written
in fairly general terms, most people judge them to
be accurate representations of themselves. This "Effect"
is the trick behind the "try it and see if it is accurate"
approach. It is not that the information is wrong,
but that it is seldom adequate for serious business
decisions. In fact, one expert described such reports
as being "just right enough to be dangerously wrong."(Generations
of Assessment Tools)
What
should I look for in selecting an instrument? First of all, be clear on what you want the
instrument to do. Instruments are designed for specific
uses. Ask to see example of how the instruments were
used. (E.g. There are a variety of HermanCheckStart case
studies included.)
Second,
consider your resources. Who will be using the information?
Certain instruments provide excellent information
but considerable training is required to use the information
effectively. This can be a problem if you want the
information to be useful to a wide range of people.
HermanCheckStart produces a unique form of report writing,
known as a virtual interview. This type of report
requires no interpretation or training and offers
specific information on job behaviors(HermanCheckStart
Job Categories).
Third,
consider the logistics of your application. Do you
want the instrument available online? On paper? On
Screen? Do you want to process the reports in-house?
Fourth,
how much time do you have? Traditionally, higher quality
instruments required as much as 1-2 hours to complete;
the only shorter instruments were of substantially
less quality. HermanCheckStart was the first in a new Generation
of higher quality assessment tools that require less
than 15 minutes to complete.
How
do I know an instrument is valid?
All instruments are valid for some purpose. Validity
is not an intrinsic characteristic. Assessments are
validated for a specific use within a specific population.
(E.g. DISC-type instruments were never intended to
be used for hiring decisions, but as a vehicle to
discuss communication styles.) Be certain that the
products you choose were developed for the purpose
you need.(HermanCheckStart
Technical Manual pdf document)
I
have heard that I should ask to see the technical
manual for the instrument. What is that?
A
thorough technical manual describes the development
of the instrument. It will generally include the objectives
of the instrument's use; the concepts behind its design;
the methodology used in the development process; and
the statistical data upon which the instrument's information
is based. Unfortunately, the existence of a technical
manual, regardless of how thick or complex it may
be, is not a guarantee of quality. Knowing that most
buyers are not trained to understand psychometric
terminology and statistical data, a number of companies
have put quite a bit of creative energy into providing
an impressive technical manual for some rather unimpressive
products. You would do well to consider the author
of these manuals and who was the technical expertise
behind the development of the instrument. Fortunately,
many of the newer instruments are offering user-friendly
versions of their technical manuals, which are designed
to educate the users and provide a guide to making
a sound decision. CheckStart's Technical Manual is
an example of this innovation. It was authored by
Dr. Leonard Goodstein, former President of the American
Psychological Association and co-author of Personality
Assessment, one of the principal textbooks on the
subject of assessments.(HermanCheckStart Technical Manual
pdf document)
We
have used a First Generation instrument for years
and everyone really likes it. Why should we change?
First of all, it is important to separate the emotional
feelings of familiarity from the pragmatic issues
of effectiveness. Think back to your first computer.
Was it a 386; 486; Mac SE? Do you still use it? Of
course not! Did it stop working? Probably not. You
just changed to newer technology because it could
do things the old computer could not. Assessment technology
is the same thing. Fifth and Sixth Generation instruments
are simply capable of providing levels of information
undreamed of with earlier instruments.(Generations
of Assessment Tools)
What
do you think about instruments that rely on success
profiles or benchmarking of jobs?
Success
profiling was a method for studying large populations
of homogenous workers to determine the characteristics
that were common to those workers producing a superior
level of performance. This methodology is the core
strategy of many popular assessment instruments, and
it is very appealing to businesses who want to "hire
more people like that one." This strategy can be effective
in the right circumstances, with a large population
of people in the same jobs, working in the same conditions,
with the same resources, with the same management,
in a stable environment. However, when the jobs vary,
or the management varies, or the environment changes
frequently, or other things are different, the success
profiling concept becomes problematical.
The
other limitation is in the collection of data. Most
companies have difficulty agreeing on exactly what
top performance is. The answer for some assessment
companies is to refer buyers to their years of research
and many thousands of people tested. They then provide
success profiles based upon this data. Experience
shows that while these ready made success patterns
can be helpful in some cases, they seldom deliver
the results that the buyer expects.
Small
businesses also have difficulty using the benchmarking
concept, since they have many positions with only
one or two employees, and therefore lack a robust
sample for the study. Job requirement questionnaires
may be used to collect data on such positions, but
it can be difficult. More importantly, small business
employees must generally function in many roles, depending
upon the demands of business. The jobs are rarely
stable enough to develop reliable profiles.
Even
when success profiles are successfully created, they
must be revisited on a regular basis to ensure that
they still match the positions. With the rapid pace
of change in today's business world, this can be as
often as every six months.
We
were considering creating our own tests. Wouldn't
that be better than buying something off the shelf?
It
depends on what you are trying to measure. If there
is some unique skill, ability, or set of knowledge
that is critical to successful performance in a particular
job, and there is not an existing instrument that
measures that, it may be necessary to construct one.
However, if job success is more dependent upon a unique
combination of fundamental characteristics of behavior
and abilities, it is much better to use established
tools. The major assessment instruments, such as those
in the Fourth, Fifth and Sixth Generations, required
years to develop; thousands of people to participate
in the normative studies; many hundreds of thousands
of dollars; and the expertise found in a relatively
small number of psychometric experts. It is usually
more economical and more effective to buy that level
of expertise. While producing "customized" tests may
offer a surface appeal, it is rarely a wise expenditure
of resources with so many other options available.
