To produce and finalize a requirements profile, every management audit begins by working with the client to define the success-critical requirements. This can be done e.g. by identifying situations that are essential for success (the so-called critical incident technique) or by analysing the tasks and the behavioural competences required to fulfil the purpose of the target position (so-called target-task analysis) and/or the requirements of strategic challenges (so-called strategy deployment).

Work samples and simulations are used to account for the fact that behavioural skills cannot be rated viably simply by “speaking about” past experience or possible responses in imagined scenarios (cf.biographical interview), but by using (cf. multi-modality) reflecting critical incidents in a relatively concrete, tangible format. This makes the available competences directly observable. The method is particularly helpful when “bridging the gap” to reality after the simulated situation. A review of the performance can include questions like “What does this say about you?”, “How would you behave in real life?” etc. to get a complete picture of the participant.

The Assessment Centre (or: AC) method was first used in the 1920s, later establishing itself as a favoured concept in modern HR management that allows the facilitators to get to know people in detail in a comparatively efficient and high-quality procedure. In essence, the AC method means that multiple appraisal modules/simulations are used to apply a range of perspectives on the given requirements and allow trained observers to arrive at a valid rating and appraisal. Typical elements include semi-structured interview, a strategy-oriented or more operationally biased case study, communicational-interactive exercises (simulated meetings with employees, colleagues, or customers; presentations), psychometric tests and questionnaires, optional preparatory self-assessments (cf. 360° feedback) and – interim– feedback at the end. The results are ideally processed in written reports that are made available to the participants.
(cf.: Schuler, H. (1998). Psychologische Personalauswahl. Einführung in die Berufeignungsdiagnostik. 2nd ed., Gottingen: Hogrefe, p. 118: Assessment centre is the name of multiple techniques that combine several aptitude diagnostics tools and performance-relevant tasks; Martin Kleinmann, M. (2003). Assessment-Center. Reihe Praxis der Personalpsychologie. Hogrefe, p. 1: Assessment Centres are multiple diagnostic procedures designed to record the systematic behavioural performance or shortcomings of people. In them, multiple observers rate the performance of one or more participants in terms of the chosen requirements dimensions along defined rules.

A fully coordinated and specified rating standard is an essential element of good diagnostics that is often given only little attention or care (many procedures apply a blanket comparison with other averages or norms chosen at random). The expected development, that is, the level of development of the relevant, success-critical competences required for the target position, needs to be defined at the start of every selection or appraisal process. Defining specific key behaviours is a helpful, if labour-intensive approach. When it is used, the levels of development of a competence are reflected in key behaviours that characterize or describe the increasing complexity of the item in question. These combined then form so-called “behaviour-based rating scales” (cf: Smith & Kendall. (1963). Re translation of expectations: An approach to the construction of unambiguous anchors to rating scales. Journal of Applied Psychology)

How can “personality” be structured? This has been one of the frontline questions of personality psychology for many decades. In current research and in business practice, a system of five traits has established itself and has been re-confirmed in multiple investigations: 1. Neuroticism, 2. Extraversion, 3. Openness, 4. Conscientiousness, and 5. Agreeability. Each dimension can be further split down into six facets to constitute the so-called “Little 30”.

Development centres (often known by their abbreviation: DC) usually refer to assessment procedures – typically for groups – that are not designed for selection purposes, but as status quo appraisals to identify any needs for additional qualification and development. Sometimes, they use specialised modules in their design that are aimed at feedback and immediate learning or training (e.g. – often video-based – feedback after a first communicational-interactive simulation, such as a simulated meeting with an employee) as well as dedicated training sequences at the end of the first day in multi-day procedures. The methodology of ACs and DCs is mostly similar.

What makes good diagnostics has been known for many years. These insights that were accepted by broad swaths of the scientific community were collated in so-called process norms at the end of the last millennium, which have been giving practitioners a very good guideline for distinguishing good from less good management diagnostics (cf. DIN. (2002). DIN 33430: Anforderungen an Verfahren und deren Einsatz bei berufs- bezogenen Eignungsbeurteilungen. Berlin: Beuth). The process steps defined in this standard are also binding for all work of aestimamus. Individual steps might have to be revised and adjusted to account for the unique processes and circumstances at the client company. On a European level, the related standards are defined in ISO 10667-1 and -2.

Since the mid-1980s, the format for individual appraisals built around the assessment centre method has become a very well-established and – in its added value for the commissioning company and the participants alike – highly accepted state-of-the-art toolkit used to support personnel decisions. At its heart, it is distinguished from group assessments in that it is shaped with the individual participant in mind. There is a much better ratio of effort to results quality, not least because the candidate can be observed and assessed in a very intensive setting in the typically one-day procedures. The confrontation with a single participant is naturally much more intensive than can be the case for multiple participants (e.g. groups of twelve people) assessed in three-day group procedures. The apparent disadvantage is that the approach allows no group exercises or discussions, which is, however, compensated for by the fact that group work often plays a relatively minor role in the critical incidents on the high-profile management level (critical incident technique) (cf. Flanagan, J. C. (1954). The critical incident technique. Psychological Bulletin). Typically, an individual AC includes a semi-structured interview, a conceptual-strategic simulation exercise (e.g. a case study) and other simulations on the communicational-interactive level (simulated meetings with employees, clients, or colleagues or presentation scenarios), supported by selected psychometric tests (personality inventories and/or performance tests) and first feedback. In terms of the underlying methodology, there is no intrinsic difference between individual assessments and management audits or appraisals. Oftentimes, the actual terminology used is determined by internal or external marketing considerations.

When appraisal procedures have no or only little relevance for real-life practice from the point of view of the participants, when they seem to make no sense for whatever reason on the part of the person assessed in them, the participants will tend to display a different type of behaviour than would be the case if they thought the procedure sensible and useful. Not least when one considers the need to shape management diagnostics processes with marketing considerations in mind, this means that procedures should be used that are highly plausible and meaningful for the participants. Providers of such procedures can then hope that the participants are much more likely to show more authentic behaviour. When the point of the procedure seems obscure at first, the best response is to inform the participants in depth and transparently about the benefits, the limitations, and the uses of such a process at an early point. Sometimes, a participant will already have a fully formed opinion about such procedures even, or especially if he or she has no actual experience with similar processes.

Management audits and assessment procedures often use this appraisal module that allows the observers to see and rate individual, expertise and experience-based approaches to essential “core processes of management and leadership”. For this purpose, the participants are given information about a company, usually in print or with some electronic support. They are usually asked to imagine themselves as having started a new job at an as yet unfamiliar business (which is hardly ever the case in real life), where they have only limited time to get a fuller picture of a) the general situation and b) the specific operational challenges at stake (which can be a common occurrence when taking over a new function). Typically, there is no one ideal solution: many roads lead to Rome and the point is for the observers to see the individual solution chosen by the participant and to discuss the results with the participant to bridge the gap back to his or her real life. “How would you proceed in similar situations?”, “What would you do if you could start this task all over again?”, “What does that say about you?”. These are the questions that can be discussed with the participant in the end. On top of this, the case study included in a management audit offers a great opportunity to observe and rate essential criteria that are harder to grasp or have less face validity in other appraisal formats: analytical capacities in the sense of the speed when processing the information; the reduction of complexity; strategic thinking and responses; and decision-making, risk taking, and focus on action.

The former claim of management diagnostics was to avoid that candidates who seemed suitable at the surface later revealed themselves to be bad choices. This led and still leads to a strong focus on deficiencies in management diagnostics, which is often still reflected in the mindsets of the assessors. Whenever there is a large number of potential applicants or candidates, nothing speaks against this approach, but this does not apply in times of scarcity like today. Now is the time for a different perspective and a new self-perception, i.e. keeping the number of apparently unsuitable, but actually very suitable candidates as low as possible. The implication is obvious: the focus shifts more to dormant potential than to past achievements and successes or current competences.

The suitability (“quality” in this sense) of the test methods used in management diagnostics is measured by means of defined and generally accepted criteria (cf. “Objectivity“, “Reliability”, “Validity”). In this respect, the usefulness and fit of appraisal instruments can be assessed quite well when their provider has published the relevant research data. Test procedures that have no or only obviously biased (unusually positive or affirmative) supporting research should be used with great caution.

Selection and appraisal processes use a range of interview types: from the fully structured interview, in which every question is predetermined, via semi-structured forms, to the free-flowing, explorative interview. Each variant has its purpose – depending on the goal it pursues. In traditional management audits, semi-structured approaches are most common, which often includes considerable exploratory elements aimed at developing hypotheses about the interviewee. Oftentimes, the interview starts by asking the participant to outline his or her past professional biography in brief, then moves to the present (e.g. asking about the reasons for the interest in the target position), before moving on the future and the participant’s visions and ambitions. The procedure frequently ends with open-ended questions on aspects that are important for the point of view of the participant).

The word “competence” has become the catch-all term for skills, capabilities, even attitudes and convictions (although it is distinguished from the construct “experience”) to describe relevant requirements relating to the successful completion of a task or set of tasks. In very few cases do competences actually refer to distinct, clearly defined constructs; rather, the pool of criteria in use as competences holds many overlaps, redundancies, or even duplicate terms for virtually identical constructs (cf. empathy vs. personal instinct). The level of abstraction can also differ considerably (social skills vs. powers of persuasion). On the most detailed level, competences get translated into so-called key behaviours. Company- or job-(group)-specific competences are typically clustered in the form of a model of competence.

Company- or job-(group)-specific competences are typically clustered in the form of a model of competence. A virtually infinite number of competences is available for their design, although practitioners tend to choose from more or less 80 to 100 sensible and distinct constructs. A model of competence is helpful for actual practice when it reflects the requirements as determined by the corporate strategy and the criteria of relevance for the target function/group of functions/job family. Many companies are interested in reflecting their “uniqueness” as an organization in the form of a “unique competence proposition” (resembling their search for a “unique selling proposition (USP)”). The end result is oftentimes a model that is good for marketing purposes, but inherently of less value for HR developers or selection and development processes, not least because of the often colourful terminology. This is not to say that such unique models of competence should be rejected out of hand, but care should be taken to ensure that practically meaningful constructs are at least hidden under the “marketing skin”.

How well is the trait that we want to test or measure actually tested or measured by our chosen means? Does our computer-based test of analytical capacities actually test the participants’ analytical capacities or their affinity to working with computers? Such are the questions that need to be asked to make sure that the chosen tests actually hit the intended target.

Unambiguous and undisputed causal relationships cannot be proved. Appraisals therefore have to work with correlations of probability. The extent to which different traits relate to each other is expressed in the correlation coefficient, ranging from -1.0 to +1.0. In the case of a perfect positive (or negative) relationship, the correlation coefficient is +1.0 (or -1.0); if no link is visible, it lies at 0.0. In management diagnostics, the key point of interest is to identify the variables that can predict professionally relevant and prospectively successful behaviour. The stronger the correlation between the variables, the better the inference about these.

The management appraisal (cf. management audit, individual assessment) can be seen as a diagnostic method belonging to the group or “family” of assessment centres. The technical term has established itself since the mid-90s. There are two schools of thought about the terminology: it is either used when referring to a larger group of executives or managers whose individual and aggregate group competences are to be compared to changing, newly relevant requirements or to the status quo of a benchmark group (responding to the common wish for external benchmarks). The second use of the term focuses on the target group: the term is used when the appraisal is aimed at a rating of actual or presumed “top” executives, for whom the role-plays and group discussions often used for traditional assessment centres are downplayed in favour of the interview method. In the end, the quality of any management audit, as is the case for any assessment procedure, depends on the shape of the process (cf.DIN 33 430) and – albeit not exclusively – on the experience and competence of the consultants conducting it.

In the German-speaking countries of Europe, the term “management audit”, which is hardly used in the Anglo-American world, has established itself since the mid-90s. We recommend equating it with the term //management appraisal//, which is more generally recognized internationally. For more information cf. //management appraisal//). Suffice to say that the terminology is less relevant than the quality of the procedure. In practice, a certain terminological flexibility is required to account for the established and accepted vernacular.

In the end, any systematic attempt to evoke a certain behavioural response represents a test. A single question in an interview can thus be considered a test. A role exercise or a graphological trial – to name good and less good methods – are tests. Good or meaningful tests in management diagnostics are procedures that allow a valid rating on the basis of the outcome, by ensuring an objective, reliable, and valid (cf. quality criteria) observation, rating, and prognosis. The definitions of the TBS-TK (test rating system of the Test Process Advisory Council of the German Psychologists Federation defines “test” as follows: In psychology and among members of the public who are not psychologists, tests have a very broad meaning: the term is used for virtually all psychologically diagnostic procedures. In the stricter sense, a psychological test refers to a specific sub-group of such psychologically diagnostic means, but the term “test” can serve as an umbrella term: there are thus intelligence and general performance tests, especially personality inventories, objective personality tests, and projective procedures, as well as standardized interviews and other survey methods for workplace diagnostics” (cf. TBS-TK Testbeurteilungssystem desTestkuratoriums der Föderation Deutscher Psychologenvereinigungen, Stand und Perspektiven, Version of 28.9.06).

“Objectivity” is one of the foremost quality criteria and describes the extent to which a test result is independent from the person conducting, processing, or interpreting the test. Procedures where the data is processed with automated templates are therefore relatively “objective” (e.g. traditional “IQ” tests or tests conducted by computer or web application). By contrast, procedures in which the assessors have comparatively many opportunity to influence the outcomes (e.g. case studies or role exercises) are less objective.

Which dispositions shape and influence the behaviour that a person shows? Is there one, more or less unchanging “core”? This is the question asked by personality psychologists. A common distinction is made between perspectives that focus on behaviour or on competence. In management diagnostics, personality and behavioural competences are often treated separately, even if both perspectives cannot exist without each other.

Appraising (past) performance (e.g. in the form of target agreements) and (current) competence (as reflected in concrete, representative work samples for the target position) also allows a prediction of future behaviour and future professional success. This fact is recognized in the practice of management diagnostics by including the personal and professional biography of participants (potentially with the added data gleaned from references) e.g. in interviews and then evaluating actual behaviour in suitable simulation exercises. The prediction of future behaviour and future success can be made more accurate by focusing specifically on particularly meaningful (predictive) variables. This can include competences of particular relevance for the client’s organization or the target function (e.g. attitudes, convictions, personality traits and characteristics, but also specific aspects of experience). To further support this, it helps to put particular weight on especially predictive variables (often called the “drivers” or “indicators” of potential): these refer to analytical or cognitive faculties, aptitude for learning, or a person’s creative will and leadership ambitions – which are particularly relevant for managerial positions. These variables gain particular importance when a lack of current experience (e.g. for junior managers) allows only a restricted assessment of past performance or when the current competences are generally weaker or as yet absent. Identifying these hidden talents or hidden champions who yet have very limited competence, but the potential for developing it in the near future, is one of the key challenges in a time of scarce resources.

Reliability refers to the extent of “accuracy” with which a trait is assessed. Diagnostics professionals try to ascertain the extent of reliability e.g. by comparing the ratings of multiple assessors (the so-called “interrater reliability”), by repeating ratings (so-called “retest reliability”), or by applying multiple ratings at the same time (so-called “parallel test reliability”). As a general rule, procedures with better reliability are to be preferred, but only if they actually cover the intended object or construct (cf. Construct validity) and if that construct is actually valid for occupational success (cf. Criterion Validity).

Often termed “role-plays”, a term which should not be used in this context. Rather, this appraisal format refers to a practice simulation in which the participant is given an opportunity to demonstrate current behavioural competences or his or her unique approach to critical incidents on a communicational-interactive level. Such an exercise always has a certain artificial element, which can, however, be reduced or qualified by applying a set of forces: e.g. embedding the simulation in the overarching scenario of the case study, applying professional role actors, or introducing a well-arranged occasion for the scenario in which each participant can envision himself or herself and will encounter only few disorienting elements of artificiality. (A negative example would be overly brief instructions that refer to a long, but practically not established backstory or prior contacts between the simulated parties)). Experienced consultants are required to ensure the objectivity and reliability that a meaningful procedure requires. It is their decisive and highly important job to shape their actions and responses on the basis of the applicable requirements profile to allow the participant to actually display the required competences. The consultants should therefore use behaviour that helps to shape the complexity and increasing difficulty of the exercise and matches the constellations created by the participant’s behaviour.

The quality criterion of validity refers to the extent to which the criterion that is to be tested or evaluated is actually tested and evaluated (so-called “construct validity”) and to the extent to which it allows a prediction of future behaviour (so-called “criterion validity”). An absolutely reliable prognosis of future success that removes all doubts and risks cannot be achieved only by considering the competences and potential of a potential candidate – which is fortunate. Even when many important environmental factors (the team, the market, the industry etc.) are considered, our world remains complex enough that the future could not be predicted “simply” by identifying and rating a few relevant competences of the actors in it.