Section 1 - Description
Section 2 - Scope
Section 3 - Requirements
Section 4 - Penalties
Section 5 - Status
The bill, which was introduced to the state Assembly in January 2023, requires both developers and deployers of the tool to submit annual impact assessments. The bill also requires certain notifications to be given, from deployers to subject users and from tool developers to deployers.
The bill applies to both developers and deployers of automated decision tools which are involved in decisions that have a material impact on an individual. These types of decisions considered in the bill are:
- Essential utilities
- Family planning
- Health care and health insurance
- Financial services
- Criminal justice
- Legal services
- Access to benefits or assignment of penalties
The bill defines an ADT as a machine-based system that can make predictions, decisions or recommendations according to a human-specified objective.
The bill requires impact assessments to be completed by both the developer and the deployer of the tool, before or on Jan 1st, 2025 and annually thereafter.
The requirements fall into three categories:
1. Impact assessments
The impact assessment must also include an analysis of potential disparate impacts on the basis of sex, race, color, ethnicity, religion, age, national origin, limited English proficiency, disability, veteran status, or genetic information. The assessment must also include a description of the tool, covering its purpose, data collection, what safeguards were put in place, and more.
- The deployer of the tool must notify potential subjects that an ADT is being used, as well as the purpose of the tool.
- Tool deployers must accommodate requests from subjects to be assessed by an alternative method, if technically feasible.
- Tool developers must provide deployers with a statement of the intended use of the tool and documentation about its limitations, what data was used and how it was evaluated.
Lastly, both developers and deployers must develop a governance strategy with the technical and administrative capabilities to map, measure, manage, and govern the reasonably foreseeable risks of discrimination from the use of ADTs. All developers and deployers are subject to this requirement, except companies with fewer than 25 employees who didn’t deploy a tool that impacted 1,000 or more people last calendar year.
Failing to submit an impact assessment to the state’s Civil Rights Department within 60 days of its completion would result in a fine of no more than $10,000 for the offending developer or deployer. Each day the model is used without submitting the impact assessment results in another fine.
The bill has not yet passed. If and when it does, it would enter enforcement on January 1st, 2025.