New York City (N.Y.C) has issued a Final Rule on Automated Employment Decision Tools (AEDTs). The date of enforcement is now July 5, 2023.
The Details
Beginning on July 5, 2023, an employer or employment agency (employer) that uses an Automated Employment Decision Tool (AEDT) to substantially assist or replace discretionary decision making in an employment decision (e.g., screening candidates for employment or promotion) must:
- Provide notice to candidates and employees that the AEDT is being used;
- Conduct an audit for AEDT bias within one year of using the AEDT; and
- Make the audit information available to the public.
As a way of background, an initial version of this Rule was published in 2022. The city is adding rules to implement new legislation regarding AEDTs.
The Final Rule
The Final Rule defines several terms.
Under the Final Rule to substantially assist or replace discretionary decision making means:
- Relying solely on a simplified output with no other factors considered; or
- Using a simplified output: 1) as one of a set of criteria where such output is weighted more than another criterion in the set; or 2) to overrule conclusions derived from other factors, such as human decision making.
The Final Rule also defines simplified output as a prediction or classification as specified in the definition for machine learning, statistical modeling, data analytics, or artificial intelligence that may take the form of a:
- Score, such as rating a candidate’s estimated technical skills;
- A tag or categorization, such as categorizing a candidate’s resume based on keywords, assigning a skill or trait to a candidate, or recommendation (whether a candidate should be given an interview); or
- Ranking (such as arranging a list of candidates based on how well their cover letters match the job description).
Note: This does not include output from analytical tools that translate or transcribe existing text, such as converting a resume from a PDF or transcribing a video or audio interview.
Under the Rule, for AEDT purposes, machine learning, statistical modeling, data analytics, or artificial intelligence means a group of mathematical, computer-based techniques:
- That generate a: 1) prediction (expected outcome) for an observation, such as an assessment of a candidate’s fit or likelihood of success, or 2) a classification (an assignment of an observation) to a group, such as categorizations based on skill sets or aptitude; and
- In which a computer (at least in part) identifies the inputs, the relative importance placed on those inputs, and, if applicable, other parameters for the models to improve the accuracy of the prediction or classification.
Bias Audit
The Final Rule requires an employer or employment agency to conduct an audit when they use an AEDT to assess employees or candidates for employment or promotion. An employer or employment agency may not use (or continue using) an AEDT if more than one year has passed since their latest audit.
Under the Final Rule, for purposes of the required audit, an independent auditor must be used and be a person or group with the capability to exercise objective and impartial judgment on all issues within the scope of an audit.
The independent auditor cannot:
- Be involved in the use, development or distribution of the AEDT;
- Have an employment relationship with either the employer or an employment agency that seeks to use the AEDT during the audit; or
- Have a direct or indirect material financial interest in using, or continuing to use, the AEDT during the audit.
Data Use
The bias audit must use historical data of the AEDT. An individual employer or employment agency may rely on a bias audit of an AEDT that uses the historical data of other employers or employment agencies if the employer or employment agency provided historical data from its own use of the AEDT to the independent auditor conducting the bias audit or if such employer or employment agency has never used the AEDT.
Employers and employment agencies may rely on a bias audit that uses test data if insufficient historical data is available to conduct a statistically significant bias audit. However, they must explain why historical data was not used.
Calculations
Under the Final Rule, the bias audit must include the calculation of the impact ratio for each EEO race/ethnicity category, sex category (male/female) and the intersectional categories of sex, ethnicity and race categories. The impact ratio is calculated in one of two ways depending on whether the AEDT provides a scoring rate or selection rate for candidates.
Specifically, scoring rate is defined as a rate at which individuals in a category receive a score above the sample’s median score where the score has been calculated by an AEDT. Selection rate is defined as the rate at which individuals in a category are either selected to move forward in the hiring process or assigned a classification by an AEDT.
Where the AEDT selects candidates to move forward in the hiring process, a bias audit must calculate the selection rate and impact ratio for each for each sex and race/ethnicity category and the intersectional categories of sex, ethnicity and race. The impact ratio will be calculated by dividing the selection rate for a category by the selection rate of the most selected category.
If the AEDT scores candidates for employment, the bias audit must calculate the median score for the full sample of applicants, the scoring rate for individuals in each category, and calculate the impact ratio. These calculations must be done for each sex, race/ethnicity and intersectional categories. Where the AEDT scores candidates for employment, the impact ratio is calculated by dividing the scoring rate for a category by the scoring rate for the highest scoring category.
The independent auditor may exclude a category that represents less than 2% of the data being used for the bias audit from the required calculations for impact ratio. However, the independent auditor’s justification for the exclusion, as well as the number of applicants and scoring or selection rate for the excluded category must be included in the bias audit.
See the text of the Rule for more examples.
Publishing Audit Results
Before using an AEDT, an employer or employment agency in N.Y.C. must make the following publicly available on the employment section of their website in a clear and conspicuous manner:
- The date of the most recent AEDT audit and a summary of the results, including:
- The distribution date of the AEDT.
Note: Employers may meet the publishing requirements with an active hyperlink to a website containing the required summary of results and distribution date, if the link is clearly identified as a link to the results of the audit.
Recordkeeping
An employer must keep the summary of results and distribution date posted for at least six months after its latest use of an AEDT in an employment decision.
Notice and Posting Requirements
Employers are not required to provide an alternative selection process, but they are required to include instructions for how an individual can request an alternative selection process or a reasonable accommodation under other laws, if available.
The Final Rules also provide for notice requirements and provide various ways that the notice requirements may be complied with depending on whether the notice is being given to a candidate for employment or an employee seeking promotion.
For candidates for employment who reside in the city, employers may comply with the notice requirement if they (at least 10 business days before the use of an AEDT) provide notice on the employment section of its website in a clear and conspicuous manner, in a job posting or via U.S. mail.
Employers and employment agencies may comply with the notice requirements for employees who reside in the city and are seeking promotion by providing such notice at least 10 business days before use of an AEDT in a written policy or procedure given to employees, in a job posting or via U.S. mail.
Employers and employment agencies must also either post in a clear and conspicuous manner on the employment section of its website information about its data retention policy, the type of data collected for the AEDT and the source or such data or provide instructions for making a written request for such information as well as an explanation as to why disclosure of such information would violate local, state, or federal law or interfere with a law enforcement investigation.
Next Steps
By July 5, 2023, employers and employment agencies covered under the law that currently use or are considering using an AEDT should:
- Evaluate if their use of an AEDT requires further action under the Final Rule.
- Follow the proper protocols under the law for all bias audits.
- Provide the required notice to employees, candidates and the audit results to the public in the timeframes specified under the Final Rule.
Please contact your dedicated service professional with any questions.