Artificial Intelligence and Automated Systems 2022 Legal Review

• Safe and Effective Systems
• Algorithmic Discrimination Protections
• Data Privacy
• Notice and Explanation
• Human Alternatives, Consideration, and Fallback

For more details, please see our Artificial Intelligence and Automated Systems Legal Update (3Q22).  The principles apply broadly to “ automated systems that … have the potential to meaningfully impact the American public’s rights, opportunities, or access to critical resources or services.”  “Automated systems” are themselves defined very broadly, encompassing essentially any system that makes decisions using computation.[9]  The Blueprint therefore stands in contrast to the draft EU AI Act, which is generally limited in scope to an identified list of high-risk AI.[10]  The Blueprint is intended to further the ongoing discussion regarding privacy among federal government stakeholders and the public, but its impact on the private sector is likely to be limited because—unlike the wide-ranging EU AI Act, which is inching towards an implementation date—it lacks prohibitions on AI deployments and details or mechanisms for enforcement.  The Blueprint is accompanied by supporting documentation, including a set of real-life examples and a high-level articulation of how the five principles can “move into practice.”[11]

2.  National Institute of Standards and Technology (“NIST”) Risk Management Framework

• Organizations must cultivate a risk management culture, including appropriate structures, policies, and processes.  Risk management must be a priority for senior leadership.
• Organizations must understand and weigh the benefits and risks of AI systems they are seeking to deploy as compared to the status quo, including helpful contextual information such as the system’s business value, purpose, specific task, usage, and capabilities.
• Using quantitative and qualitative risk assessment methods, as well as the input of independent experts, AI systems should be analyzed for fairness, transparency, explainability, safety, reliability, and the extent to which they are privacy-enhancing.
• Identified risks must be managed, prioritizing higher-risk AI systems.  Risk monitoring should be an iterative process, and post-deployment monitoring is crucial given that new and unforeseen risks can emerge.

NIST plans to publish AI RMF on January 26, 2023.

NIST is also leading federal regulatory efforts to establish practices for testing, evaluating, verifying, and validating AI systems.  In March 2022, NIST released a document titled, “Towards a Standard for Identifying and Managing Bias within Artificial Intelligence,” which aims to provide guidance for mitigating harmful bias in AI systems.[14]  The guidance makes the case for a “socio-technical” approach to characterizing and mitigating bias in AI, noting that while computational and statistical sources of bias remain highly important, broader societal factors —including human and systemic biases—that influence how technology is developed should also be considered.  The guidance also recommends a human-centered design process, and draws out organizational measures that can be deployed to reduce the risk of potential bias, including monitoring AI systems, providing resource channels for users, implementing written policies, procedures, and other documentation addressing key terms and processes across the AI model lifecycle, and fostering a culture of internal information sharing.

3.  FTC

a)  FTC Explores Rulemaking to Combat “Commercial Surveillance”

Notably, the ANPRM solicited public input on algorithmic decision-making, including the prevalence of algorithmic error, discrimination based on protected categories facilitated by algorithmic decision-making systems, and how the FTC should address algorithmic discrimination through the use of proxies.[17]  The FTC is undertaking this rulemaking under Section 18 of the FTC Act (also known as “Magnuson-Moss”),[18] a lengthy and complicated hybrid rulemaking process that goes beyond the Administrative Procedure Act’s standard notice-and-comment procedures.[19]  In light of these procedural hurdles, any new proposed rules likely will take considerable time to develop.  The ANPRM notes that, if new rules are not forthcoming, the record developed in response to the ANPRM nevertheless will “help to sharpen the Commission’s enforcement work and may inform reform by Congress or other policymakers.”  The inclusion of algorithmic decision-making in the scope of the potential rulemaking underscores the FTC’s continued focus on taking the lead in the regulation of automated systems at federal level.

b)  FTC Report Warns About Using Artificial Intelligence to Combat Online Problems

In December 2020, as part of the 2021 Appropriations Act, Congress tasked the FTC with conducting a study and reporting on whether and how AI could be used to identify, remove, or take other appropriate action to address a variety of online harms (scams, deepfakes, child sexual abuse, terrorism, hate crimes and harassment, election-related disinformation, and the traffic in illegal drugs and counterfeit goods).  Congress also required the FTC to recommend reasonable policies and procedures for using AI to combat these online harms, and any legislation to “advance the adoption and use of [AI]” for these purposes.

In its June 16, 2022 report,[20] the FTC advised that, while AI can be used as a tool to detect and remove harmful material online, there are significant risks associated with its use.  In particular, the FTC cautioned that because AI systems rely on algorithms and inputs created by humans, and often have built-in motivations geared more towards consumer engagement rather than content moderation, even supposedly neutral systems can disproportionately harm minorities while threatening privacy and free speech.  Additionally, the FTC stated that while many companies currently use AI tools to moderate content, they “share little information about how these systems work, or how useful they are in actually combating harmful content.”[21]  The FTC therefore advised that there needs to be more transparency before the government can understand how AI tools work in the real world.  Although the Commission acknowledged that major tech platforms and others are already using AI tools to address online harms, the report’s final recommendation is that Congress should avoid laws that would mandate or overly rely on the use of AI to combat online harms and instead conduct additional investigation into other tools that might also be helpful in moderating online content.  In his dissenting statement, Commissioner Phillips noted that the report “has no information gleaned directly from individuals and companies actually using AI to try to identify and remove harmful online content, precisely what Congress asked us to evaluate.”[22]

Further, on June 22, 2022, Senators Ed Markey (D-MA), Elizabeth Warren (D-MA), Brian Schatz (D-HI), Cory Booker (D-NJ), Ron Wyden (D-OR), Tina Smith (D-MN), and Bernie Sanders (VT) sent a letter to FTC chair Lina Khan urging the FTC to “build on its guidance regarding biased algorithms and use its full enforcement and rulemaking authority to stop damaging practices involving online data and artificial intelligence.”[23]  The letter cites the National Institute of Standards and Technology’s study that Black and Asian individuals “were up to 100 times more likely to be misidentified” by biometric surveillance tools than white individuals, and asks the FTC to use its authority to combat “invasive and discriminatory biometric surveillance tools,” including facial recognition tools.

4.  CFPB

The Consumer Financial Protection Bureau (“CFPB”) published guidance in May 2022 for financial institutions that use AI tools.  The CFPB guidance addresses the applicability of the Equal Credit Opportunity Act (“ECOA”) to algorithmic credit decisions and clarifies that creditors’ reporting obligations under the ECOA extend equally to adverse decisions made using “complex algorithms.”

The U.S. Equal Employment Opportunity Commission (EEOC) has been pursuing an initiative that seeks to provide guidance on algorithmic fairness and the use of AI in employment decisions.

On May 12, 2022, more than six months after the Equal Employment Opportunity Commission (“EEOC”) announced its Initiative on Artificial Intelligence and Algorithmic Fairness, the agency issued its first guidance regarding employers’ use of AI.  The EEOC’s non-binding, technical guidance provides suggested guardrails for employers for the use of AI technologies in their hiring and workforce management systems.

On May 5, 2022, the EEOC filed a complaint in the Eastern District of New York alleging that a software company providing online English-language tutoring to adults and children violated the Age Discrimination in Employment Act (“ADEA”) by denying employment as tutors to a class of plaintiffs because of their age. Specifically, the EEOC alleges that the company’s application software automatically denied older, qualified applicants by soliciting applicant birthdates and automatically rejecting female applicants age 55 or older and male applicants age 60 or older.  The EEOC seeks a range of damages, including back wages, liquidated damages, a permanent injunction enjoining the challenged hiring practice, and the implementation of policies, practices, and programs providing equal employment opportunities for individuals 40 years of age and older.

The EEOC’s guidance outlines best practices and key considerations that, in the EEOC’s view, help ensure that employment tools do not disadvantage applicants or employees with disabilities in violation of the Americans with Disabilities Act (“ADA”).  The guidance provides three ways in which an employer’s tools could be found to violate the ADA:  (1) by relying on the tool, the employer fails to provide a reasonable accommodation; (2) the tool screens out an individual with a disability that is able to perform the essential functions of the job with or without an accommodation; and (3) the tool makes a disability-related inquiry or otherwise constitutes a medical examination.

The Artificial Intelligence Training for the Acquisition Workforce Act (AI Training Act) was signed into law by President Biden in October 2022.  The bi-partisan Act takes a risk management approach towards federal agency procurement of AI and cleared the Senate in late 2021 after being introduced by Sens. Gary Peters (D-Mich.) and Rob Portman (R-Ohio).  This bill requires the Office of Management and Budget (OMB) to develop an AI training program to support the informed acquisition of AI by federal executive agencies, and ensure agencies and individuals responsible for procuring AI within a covered workforce are aware of both the capabilities and risks associated with AI and similar technologies.

2.  National Defense Authorization Act 2023

3.  The Algorithmic Accountability Act of 2022 (H.R. 6580)

4.  Digital Platform Commission Act of 2022 (S. 4201)

5.  American Data Privacy and Protection Act (H.R. 8152)

6.  Health Equity and Accountability Act of 2022 (H.R. 7585)

Introduced in the House on April 26, 2022, the Health Equity and Accountability Act of 2022 (H.R. 7585) aims to address algorithmic bias in the context of healthcare.  The Bill would require the Secretary of Health and Human Services to establish a “Task Force on Preventing AI and Algorithmic Bias in Healthcare” to develop guidance “on how to ensure that the development and [use] of artificial intelligence and algorithmic technologies” in delivering care “does not exacerbate health disparities” and help ensure broader access to care.  Additionally, the Task Force would be charged with identifying the risks posed by a healthcare system’s use of such technologies to individuals’ “civil rights, civil liberties, and discriminatory bias in health care access, quality, and outcomes.”  The bill was referred to the Committee on Energy and Commerce.

In September 2022, a public hearing was held to clarify SDAA’s requirements and objectives.  Commenters focused on the expansive definition of “algorithmic eligibility determination” or “algorithmic information availability determination” in the bill, which, as drafted, applies to any determination based “in whole or significant part” on an “algorithmic process that utilizes machine learning, artificial intelligence, or similar techniques.”[30]  These broad definitions—which mirror the rights-based approach in the AI bill of rights and contrast with the EU AI Act’s risk-based hierarchy—could potentially include virtually any automated process and therefore create both significant uncertainty about the scope of the bill and the prospect of burdensome audit and disclosure obligations even for low-risk processes.  We will continue to track the progress of this bill, as well as forthcoming opportunities to participate in public hearings and submit comments.

2.  Colorado Law “Protecting Consumers from Unfair Discrimination in Insurance Practices” (SB 21-169)

In July 2021, Colorado enacted SB 21-169, “Protecting Consumers from Unfair Discrimination in Insurance Practices,” a law intended to protect consumers from unfair discrimination in insurance rate-setting mechanisms.[31]  The law requires insurers to test their big data systems—including external consumer data and information sources, algorithms, and predictive models—to ensure they are not unfairly discriminating against consumers on the basis of a protected class, and to demonstrate to the Division of Insurance how they are testing their data and tools to ensure they do not result in unfair discrimination.  The legislation directs the regulator to work with stakeholders during the rulemaking process regarding how companies should test for bias and demonstrate compliance.  The latest stakeholder meeting took place on December 8, 2022.

Similar laws attempting to regulate insurers’ use of consumer data and algorithmic processing have since been proposed in Indiana,[32] Oklahoma,[33] Rhode Island,[34] and New Jersey.[35]  We will continue monitor Colorado’s stakeholder process, as well as state legislative and regulatory activity seeking to impose requirements with respect to insurers’ use of external consumer data, information sources, and algorithms.

3.  California Department of Insurance Issues Bulletin Addressing Racial Bias and Unfair Discrimination

D.  Employment & HR

1.  New York City Artificial Intelligence Law

On September 19, 2022, the New York City Department of Consumer and Worker Protection (“DCWP”) proposed rules in an attempt to clarify numerous ambiguities in New York City’s Automated Employment Decision Tools (AEDT) law, which was originally expected to take effect on January 1, 2023.[42]  New York City’s law will restrict employers from using AEDT in hiring and promotion decisions unless it has been the subject of a bias audit by an “independent auditor” no more than one year prior to use.[43]  The law also imposes certain posting and notice requirements to applicants and employees.  The DCWP’s proposed rules are currently under consideration and may well invite more questions than answers as uncertainty about the requirements lingers.  The proposed rules attempted to clarify certain key terms, specify the requirements for and provide examples of bias audits, and outline several different ways by which, if passed, employers may provide the advance notice to candidates and employees regarding the use of an AEDT.[44]

Emphasizing the ambiguities in both the law and proposed rules, commenters at the first public hearing, held on November 4, 2022, advocated for a delay in the law’s enforcement on the basis that employers would not have enough time to come into compliance with finalized rules before the January 1, 2023 effective date.  On December 12, 2022, DCWP announced that it would delay enforcement of the law to April 15, 2023.  At the end of December 23, 2022, DCWP issued revisions to the proposed rules, which included a new definition for an “independent auditor,” a slightly narrowed definition of AEDT, and information about conducting a bias audit using historical or test data.  In light of the high volume of comments it has received, DCWP held a second public hearing on January 23, 2023.[45]  We are continuing to monitor the law and proposed rules for further updates.

2.  New Jersey Bill to Regulate Use of AI Tools in Hiring Decisions, A4909

On December 5, 2022, New Jersey lawmakers introduced a bill to regulate the “use of automated tools in hiring decisions to minimize discrimination in employment.”[46]  The bill is similar to the initial draft of the New York AI law and imposes limitations on the sale of AEDTs, including mandated bias audits, and requires that candidates be notified that an AEDT was used in connection with an application for employment within 30 days of the use of the tool.  The bill has been referred to the Assembly Labor Committee.

3.  California

In March 2022, the Fair Employment & Housing Council released proposed regulations intended to clarify that the state’s current employment discrimination regulations apply to automated-decision systems, defined as any “computational process, including one derived from machine learning, statistics, or other data processing or artificial intelligence techniques, that screens, evaluates, categorizes, recommends, or otherwise makes a decision or facilitates human decision making that impacts employees or applicants.”[47]  Under the proposed regulations, actions that are based on decisions made or facilitated by automated-decision systems may constitute unlawful discrimination if the action results in disparate impact, imposing liability on employers as well as third-party vendors that use, sell, or administer covered employment-screening tools.  At a remote public workshop on March 25, 2022, the Council did not set a timeframe for adopting the proposed regulations.

The Workplace Technology Accountability Act, AB 1651, proposed in April 2022, would restrict electronic monitoring of workers to situations where there is a “business necessity,” provide access to the data collected, and mandate specific risk management requirements:  algorithmic impact assessments and data protection impact assessments for automated decision tools and worker information systems to identify risks such as discrimination or bias, errors, and violations of legal rights.[48]  The bill was referred to the Committee on Labor and Employment but was pulled in November 2022 ahead of a scheduled vetting by the Assembly Privacy Committee.

E.  Intellectual Property

2.  Copyright Issues

Novel copyright issues continue to emerge in 2022 as technology companies release AI tools and features to the public.  With the deployment of large-scale AI systems such as Chat GPT-3 and DALL-E 2 in 2022, there has been increasing attention paid to potential copyright issues, including authorship of AI-generated works, whether the outputs of sophisticated machine learning systems can infringe copyrighted works, and the use of copyrighted materials as training data for machine learning.  On October 28, 2022, the U.S. Copyright Office (USCO) revoked an earlier registration for an artist’s partially AI-generated graphic novel, stating that “[c]opyright under U.S. law requires human authorship. The Office will not knowingly grant registration to a work that was claimed to have been created solely by machine with artificial intelligence.”[51]  Earlier this year, the USCO Review Board affirmed a decision of the USCO denying registration of artwork generated by an AI algorithm created by Dr. Stephen Thaler, mirroring his attempts to argue that his DABUS AI system is eligible to be granted a patent.[52]

II.  EU POLICY & REGULATORY DEVELOPMENTS

The Committee of the Permanent Representatives of the Governments of the Member States to the European Union approved the final version on November 18, 2022,[57] and the EU Council formally provided its updated consensus draft (the “general approach”) on December 6, 2022.[58]  The consensus proposal limits the definition of AI systems to “systems developed through machine learning approaches and logic- and knowledge-based approaches.”  On December 14, MEPs reached an agreement to delete the provision of the AI Act that would allow providers of high-risk AI to process sensitive data to detect biases in algorithms.[59]

The adoption of the general approach allows the Council to enter negotiations with the European Parliament (known as “trilogues”) once the latter adopts its own position with a view to reaching an agreement on the proposed regulation.  The European Parliament, which is still working through a slew of compromise amendments, will likely vote on the final text in the first quarter of 2023, possibly by the end of March.[60]  Following this vote, discussions between the Member States, the Parliament and the Commission are expected to commence in April, so further negotiations can be expected during 2023.[61]  Reports suggest EU lawmakers anticipate that the Act could be approved by the end of 2023, though it would not come into force until a later time.

B.  Draft AI Liability Directive and New Draft Product Liability Directive

The draft Product Liability Directive (“PLD”) establishes a framework for strict liability for defective products across the EU—including AI systems—meaning claimants need only show that harm resulted from the use of a defective product.  Notably, the mandatory safety requirements set out in the draft AI Act can be taken into account by a court for the purpose of determining whether a product is defective.

The AI Liability Directive (“ALD”), which would apply to fault-based liability regimes in the EU, would create a rebuttable “presumption of causality” against any AI system’s developer, provider, or user, and would make it easier for potential claimants to access information about specific “High-Risk” AI Systems—as defined by the draft EU AI Act.  Of particular significance to companies developing and deploying AI-related products is the new disclosure obligation related to “High-Risk” AI systems, which could potentially require companies to disclose technical documentation, testing data, and risk assessments—subject to safeguards to protect sensitive information, such as trade secrets.  Failure to produce such evidence in response to a court order would permit a court to invoke a presumption of breach of duty.

The PLD and ALD will be subject to review and approval by the European Council and Parliament before taking effect.  Once implemented, Member States will have two years to implement the requirements into local law.  We are monitoring developments closely and stand ready to assist clients with preparing for compliance with the emerging EU AI regulatory framework.

C.  Digital Services Act

D.  The EU Parliament Adopts Special Report on AI

E.  EDPS Opinion on Negotiating Directives for Council of Europe’s AI Convention

___________________________