Abstract
To validate an interpretation or use of test scores is to evaluate the plausibility of the claims based on the scores. An argument-based approach to validation suggests that the claims based on the test scores be outlined as an argument that specifies the inferences and supporting assumptions needed to get from test responses to score-based interpretations and uses. Validation then can be thought of as an evaluation of the coherence and completeness of this interpretation/use argument and of the plausibility of its inferences and assumptions. In outlining the argument-based approach to validation, this paper makes eight general points. First, it is the proposed score interpretations and uses that are validated and not the test or the test scores. Second, the validity of a proposed interpretation or use depends on how well the evidence supports the claims being made. Third, more-ambitious claims require more support than less-ambitious claims. Fourth, more-ambitious claims (e.g., construct interpretations) tend to be more useful than less-ambitious claims, but they are also harder to validate. Fifth, interpretations and uses can change over time in response to new needs and new understandings leading to changes in the evidence needed for validation. Sixth, the evaluation of score uses requires an evaluation of the consequences of the proposed uses; negative consequences can render a score use unacceptable. Seventh, the rejection of a score use does not necessarily invalidate a prior, underlying score interpretation. Eighth, the validation of the score interpretation on which a score use is based does not validate the score use.
Journal Information
The Journal of Educational Measurement (JEM) is a quarterly journal that publishes original measurement research, reports on new measurement instruments, reviews of measurement publications, and reports about innovative measurement applications. The topics addressed are of interest to those concerned with the practice of measurement in field settings as well as researchers and measurement theorists. In addition to presenting new contributions to measurement theory and practice, JEM also serves as a vehicle for improving educational measurement applications in a variety of settings.
Publisher Information
The National Council on Measurement in Education (NCME) is a professional organization for individuals involved in assessment, evaluation, testing, and other aspects of educational measurement. Members are involved in the construction and use of standardized tests and performance-based assessment, assessment program design and implementation, and program evaluation. NCME is incorporated exclusively for scientific, educational, literary, and charitable purposes. These include: (1) the encouragement of scholarly efforts to advance the science of measurement and its applications in education and (2) the dissemination of knowledge about the theory, techniques, and instrumentation available for measurement; procedures appropriate to the interpretation and use of such techniques and instruments; and applications of educational measurement in individual and group contexts. NCME members include university faculty; test developers; state and federal testing and research directors; professional evaluators; testing specialists in business, industry, education, community programs, and other professions; licensure, certification, and credentialing professionals; graduate students from educational, psychological, and other measurement programs; and others involved in testing issues and practices.
Rights & Usage
This
item is part of a JSTOR Collection.
For terms and use, please refer to our Terms and Conditions
Journal of Educational Measurement © 2013 National Council on Measurement in Education
Request Permissions
Attracting sufficient numbers of appropriately qualified applicants is an example of a staffing outcome goal.
True or False
Which of the following people determine whether an applicant will be extended a job offer?
hiring manager
staffing personnel
HR manager
department supervisor
Assigning a high-performance sales associate to work with the company's most important client is an example of
selection
strategic staffing
deployment
matchmaking
Interviewing job candidates to asses their fit with the job and organization is part of ___
deploying
attracting
selecting
employee profiling
Strategically evaluating the company's current lines of business, new businesses it will be getting into, businesses it will be leaving, and the gaps between the current skills in the organization and the skills it will need to execute its business strategy is _______
recruiting
sourcing
workforce planning
succession planning
Reducing the turnover rate of high performers is an example of a staffing process goal.
True or False
Under which of the following circumstances does a company prefer to "churn" rather than keep existing employees?
when technology is developing very rapidly
when the training period provided is short
when competition in the market has increased
when the existing employees are overqualified
when technology is developing very rapidly
The primary goal of ______ is to get the right people interested in working for an organization or in a specific job, then persuade them to apply and ultimately accept the job offer if they are extended.
selecting
sourcing
recruiting
employee branding
Greg's Bakery chain is planning to diversify into producing and selling candy. It has opened a new factory to support this plan, and the facotry is ready to be staffed. The top management at Greg's has decided to offer the best salaries in the industry to its latest employee additions. In this situation, which of the following staffing goals should Greg's follow?
provide an extensive training period
hiring the best qualified employees
hiring a large number of employees
hiring high-salaried employees
hiring the best qualified employees
When a firm determines that it will need to hire 50 customer service representative within the next three months, it has engaged in______
attracting
placement
competency modeling
work force planning
When Mary hires someone because they share the same interests, which of the following has occurred?
stereotyping
prejudice
hiring manager bias
like-me bias
The WARN act of 1988 applies to
employers with employees who work less than 20 hours per week
employers with up to 100 employees, not including part-tine workers
regular local, federal and state government entities that provide public services
private, public, and quasi-public entities which operates in a commercial context
private, public, and quasi-public entities which operates in a commercial context
Comparing the percentages of men, women, or minorities in various job categories to see if men, women, or minorities are disproportionately represented in certain workforce categories is an example of a ______
concentration statistic
stock statistic
hiring yield statistic
flow statistic
A company fails to do a background check that would have revealed that a person it has hired has the potential to harm others. The employee then physically harms a customer during disagreement. The company could be found guilty of _______
Negligent referral
negligent hiring
violating the WARN ACT
fraudulent recruitment
Anyone who submits a resume or an employment application through a website is considered an applicant
True or False
It is legal to compare an applicant's scores only to members of his or her own racial subgroup and set separate passing or cutoff score for each subgroup.
True or False
A BFOQ is an
analysis of a firm's competitive advantage
requirement for establishing a case of discrimination
exception to employment-at-will relationship
characteristic that is essential to the performance of a specific job
characteristics that is essential to the performance of a specific job
A hotel chain that is recruiting front desk receptionists receive 200 applications from males and 100 applications from females. 100 out of 200 men are hired and 25 out of 100 women are hired. Which of the following statements is true of the hotel chain's hiring policy?
There is evidence of disparate impact against both males and females
There is evidence of a disparate impact of the hiring process on males as a group
There is evidence of a disparate impact of the hiring process on females as a group
There is no evidence of any disparate impact
There is evidence of a disparate impact of the hiring process on females as a group
Which of the following is true for affirmative action plans?
An affirmative action plan does not need to be formalized
An affirmative action plan should exclude all non-minorities
An affirmative action plan should be temporary
An affirmative action plan should not be remedial in nature
An affirmative action plan should be temporary
The uniformed Services Employment and Reemployment Rights Act_____
provides job training to members of the uniformed services to assist them in transitioning to civilian jobs
requires that a job board be maintained for the sole use of veterans transitioning to civilian jobs after their service ends
ensures that veterans are equally represented in the workforce
ensures that members of the uniformed services can return to their civilian employment after their military service ends
ensures that members of the uniformed services can return to their civilian employment after their military service ends
Which of the following are employees of a company who take on the operation of certain functions, or staff an entire office or factory on a contractual basis for a client or company?
independent contractors
contingent workers
leased workers
at-will employees
Facially neutral means that all employees and applicants are treated consistently, regardless of their protected characteristics, such as their sex and race.
True or False
Comparing th percentages of men, women, or minorities employed in a job category with their availability in the relevant population of qualified people interested in the position is an example of ______
bona fide occupational qualifications
concentration statistic
stock statistic
flow statistic
Which of the following is a characteristic of independent contractors?
they do not receive benefits from the employer
they have continuous relationship with the employer
they do not pay their employment taxes, the employer does
they control the processes and results of their work
they do not receive benefits from the employer
The age discrimination in employment act of 1967 protects people _______
50 years of age or older
40 years of age or older
60 years of age of older
If a company makes compromises to a recruitment that it does not intend to keep, it could be found guilt of _______
violating the WARN ACT
negligent referral
fraudulent recruitment
negligent hiring
The same job analysis techniques can be used effectively for staffing, compensation, and training purposes.
True or False
The first step in conducting a job analysis is to _____
write the job description and person specification
communicate the purpose of the job analysis to the job experts
collect critical incidents
get the support of top management
get the support of top management
A summary of the characteristics of someone able to perform the job well is a ________
job analysis
job description
job requisition
person specification
Alen and Jane are the best economnists that FlexMoney Inc., a consulting firm has. However, Flexmoney is in urgent need to hire more economists and Alex and Jane are told to come up with all the criteria that they think will influence the ability to succeed in this field. This should give the higher management a good idea of the nature of the job and the kind of people they should be looking to hire. Which job analysis method is being used by FlexMoney?
task inventory method
job elements method
critical incidents method
Position Analysis questionnaire method
When Mary's Macaroons conducted a _______ on its customer service positions, it found that 35% of the customer service employee's time was being spent on administrative duties, not focusing on customer service.
competency analysis
job rewards analysis
job analysis
value chain analysis
The balance between the intrinsic and extrinsic rewards an employee receives by working for a particular employer in return for their job performance is the _________.
job's total compensation package
employee value proposition
work-life balance
employee value proposition
Competency modeling is a job analysis method that identifies the worker competencies necessary for high performance.
True or False
When a main job duties have already been determined, a _______ job analysis would be required.
reactive
inductive
ideosyncratic
deductive
Early retirement programs are a common way of dealing with temporary employee surpluses.
True and False
A ratio analysis assumes that the ratio between the number of employees needed and certain business metrics is highly variable.
True or False
Anna is the manager of human resources at Alcatec Inc. She is comparing the number of employees required over the next six months versus the capabilities of her existing workforce adjusted for attrition. She is trying to identify any gaps between the two. Which step of the workforce planning process is discussed here?
monitor and revise the forecasts and action plans
articulate the firm's strategic staffing decisions
conduct a workforce analysis
develop and implement action plans
conduct a workforce analysis
Which of the following is more likely to warrant changes in an organization's compensation policy and induce it to offer above-market wages?
restructuring of jobs to make them more process oriented
decreasing unemployment rates
decrease in the number of employees quitting jobs
reduction in the number of companies in its industry
decreasing unemployment rates
A transition analysis can account for multiple moves.
True or False
Why is it usually best to supplement the ratio, scatter plot, and trend forecasting methods with managerial judgments?
because top-down approaches need analytical validation
because firms need to keep their fingers on the pulse of their labor markets
because judgment forecasting is the most accurate method
because historical trends and relationships can change
because historical trends and relationships can change
If a company employs two office assistants for every nine architects (a staffing ratio 2:9) and it plans to expand and hire eighteen new architects, how many new office assistants will it need to hire?
6
4
8
2
Filing lower-level positions from the local labor market and higher-level positions from the regional of national labor market is an example of___________.
geographic targeting
global staffing
job fairs
labor market analysis
A successful executive, who is happy with her current position, does not look for information about other jobs, but might be tempted by a great opportunity is a
passive job seeker
semi-passive job seeker
active job seeker
non-active job seeker
The ongoing process of recuting, evaluating, developing, and preparing employees to assume other positions in the firm in the future is _______.
a qualifications inventory
resume database building
job posting system
succession management
An organization that is currently composed of primarily white males may discriminate against minorities and women if it uses which recruiting source?
job fairs
college recruiting
employment referrals
search firms
Recruiting sources should be prioritized based on staffing goals and the results of the recruiting source effectiveness analysis.
True or False
Internet data mining involves searching the internet to locate active job seekers with characteristics and qualifications needed for a position.
True or False
Recruiters are seen as setting the standard for talent for the company.
True or False
When an applicant believes that an assessment method is unfair because it does not give him an opportunity to demonstrate job-relevant skills _____.
distributive fairness is low
procedural fairness is low
interactional fairness is high
internactional fairness is low
interactional fairness is high
Amy was unhappy with the way she was treated when she applied for the position of a shop-floor assistant at a local retail outlet. When her friend Carrie hears about this, she stops shopping at that store. This is an example of _________.
spillover effect
groupthink
cognitive dissonance
incorrect signaling
Having recruiters evaluate candidate for their fit with several different positions is an example of ______.
interpersonal skills
applicant attraction
workforce development
multiple assessments
Failure to measure important aspects of the attribute to be measured results in _____ error.
deficiency
random
contamination
standard
Assessing whether two interviewers rate the same candidate's interview the same is an example of test-related reliability.
True or False
False positives occur when a weak applicant is erroneously classified as being a good hire.
True or False
The correlation coefficient between a selection test and job performance is 0. This means the selection test _________.
perfectly predicts job performance
is positively related to job performance
has adverse impact on job performance
and job performance are unrelated
and job performance are unrelated
How can an organization with a positive employer branding image increase the quality of its applicant pools?
by using external agencies to recruit for them
by following a targeted recruiting effort
by reducing recruitment flaws through training
by following unconventional methods to recruit
by reducing recruitment flaws through training
The uniform Guidelines on Employee Selection procedures were created to______
outline the enforcement of the Americans with disabilities act
assist organizations in complying with federal law prohibiting discrimination in hiring
assist organizations with fewer than 10 employees to comply with Title V11
validate employee selection methods
assist organizations in complying with federal law prohibiting discrimination in hiring
When a company hired Marcus who turned out to be a poor performer, which assessment outcome occurred?
false negative
true positive
true negative
false positive
The fit between a person's abilities and the deamns of the job and the fit between a person's desires and motivations and the job are _______
person-objective fit
supplementary fit
person-job fit
person-group fit
Having a low selection ratio means_______
hiring most people who apply for a job
hiring only a few applicants
having a small number of applicants per job
having unacceptable adverse impact
hiring only a few applicants
Behavioral interviews are more reliable and substantially better than_____ at predicting job performance.
situation judgement tests
structured interviews
situation interviews
unstructured interviews
Supplementary fit occurs when a person has characteristics which are very different from those that already exist in the organizations.
True or False
Both true positive and true negative assessment outcomes are desirable.
True or False
When an employee quits because he/she is passed over for promotion, what has occurred?
a negative stakeholder reaction
external assessment
a breach of employee ethics
successsion management
a negative stakeholder reaction
Skills inventories are databases of the skills, competencies, and other relevant characteristics of each employee.
True or False
Equal Employment Opportunity liability is often greater with internal versus external assessment and staffing efforts.
True or False
Which of the following would be required to provide critical skills to employees that might be needed in the future.
competition
training
evaluation
promotion
A nine box matrix requires assessing employees on________
tenure and current job performance
current job performance and future potential
past job performances and future potential
past job performance and current job performance
current job performance and future potential
After choosing the position on which to focus, the first general step in a succession management project is to ___________
identify each interested candidate's strengths, weaknesses, and succession readiness
write a person specification linked to the firms business strategy
check that the job description outlines the current and future competencies, a person needs to perform at a high level
create a plan to continually and systematically improve the capabilities of all identified succession candidates
check that the job description outlines the current and future competencies, a person needs to perform at a high level
It is sometimes permissible to hire a non-qualified minority candidate over a qualified non-minority candidate.
True or False
Using high cut scores_________
can reduce adverse impact to a minimum in every case
is more appropriate for firms focusing on filling vacancies in the short-term
increase the number of false negative hiring outcomes
is inappropriate for firms with a talent strategy of hiring only the best
increase the number of false negative hiring outcomes
Allowing a candidate's high score on an interview to make up for a low score on a personality test is an example of the ________.
Compensatory approach
contingent approach
normative approach
multiple hurdles approach
Averaging candidates' scores from an interview, job knowledge test, and work simulation to derive a summary score to use in comparing candidates is an example of _________.
contingent weighting
unequal weighting
normative weighting
unit weighting
In the selection procedure of a firm, an applicant must obtain a minimum passing score on two different assessment methods before being allowed to continue in the hiring process. What type of assessment plan is being used?
supplemental
bona fide
contingent
multiple` hurdles
Deferring a new hire's start date can be an alternative to ______.
negligent hiring
using an assessment center
reneging
conducting a reference check
What is optimal turnover?
it sees any employee turnover as a loss for the company
it produces the highest level of long-term business improvement
it is the lowest turnover that a firm can achieve
it produces lowest levels of long-term productivity
it produces the highest level of long-term business improvement
Holding managers accountable for turnover of the employees under them is an effective retention strategy because _____.
employees cannot complain about miscommunication
employees have most grievances about their supervisors
managers know which employees to retain
managers become stressed about being laid-off
managers know which employees to retain
Having terminated employees sign a severance agreement that includes a release stating that the departing employee gives up some or all rights to sue reduces the risk of future litigation.
True
False
When a poor performer leaves, functional turnover has occurred.
True or False
Workforce redeployment applies the ____ principals used to optimize inventory management, planning, and production to optimize the utilization of a firm's employees.
competitive advantage
supply chain
product innovation
customer management
______ is a direct cost of staffing.
lower competition
lower productivity
lost business opportunities
lost market share
In terms of evaluating staff systems, lower applicant quality is a ________.
lagging indicator
staffing scorecard criterion
staffing metric
leading indicator
The choice of criteria in a balanced scorecard should be based on the company's strategy and goals, not on the the challenges the company anticipates.
True of False
Which of the following is a long-term staffing metric?
the average time-to-start (by position, source, and recruiter)
the number of diverse hires coming from each recruiting source
employee job success measured by recruiting source and recruiter
the number of high quality new hires coming from each recruiting source
employee job success measured by recruiting source and recruiter
The average time to start is a short-term staffing metric.
True or False
Having administrative assistant job candidates perform a set of exercises including taking phone messages, typing reports, scheduling meetings, and filing is an example of ______.
structured interview
job knowledge test
assessment center
clinical assessment