TPRC 2024: The 52nd Research Conference on Communications, Information and Inter

Link to paper

Abstract:
In its Blueprint for an AI Bill of Rights, the White House lists “notice and explanation” as one of five principles fundamental to protecting the American public as artificial intelligence (AI) is deployed. The Blueprint states “[y]ou should know that an automated system is being used and understand how and why it contributes to outcomes that impact you.” In its description of the notice/explanation principle, The White House emphasizes the importance of plain language explanations about AI use. Furthermore, a company should describe how it plans to use AI systems, how the systems work, and explain any risks to consumers. While AI transparency is associated with the possibility of auditable and accountable algorithmic systems, more research is needed to develop best practices. This project assesses the extent to which notices from 40 companies align with The White House call for AI transparency.

This study adapts a data privacy transparency assessment model to assess the AI transparency of 40 companies. The sample: 10 social media, 10 eCommerce, and 10 brick-and-mortar companies, and 10 banks. AI transparency was assessed via qualitative content analysis of AI transparency materials (via privacy policies) from company websites. Building on previous studies addressing data privacy transparency, the assessment involved assigning full, half, or zero stars on ten AI transparency criteria, including: whether transparency materials are accessible via company websites, and presented in plain language - assessed by Flesch-Kincaid grade reading level analysis, whether references to applicable laws/regulations are provided, whether information about how AI systems work and connections between AI systems and company decision-making are explained, whether the risks of AI use are explained, whether companies disclose details about data retention policies, and data storage/processing, and whether company AI transparency materials are posted elsewhere online.

Findings suggest companies provide privacy-related components of AI transparency but have yet to start disclosing details about the use and implications of algorithmic, automated systems. The average score across the 40 companies is 2.95/10 stars, with the average across social media companies (3.35/10), which is higher than e-commerce (2.75/10) and brick-and-mortar companies (2.75/10), as well as the banks (2.95/10). YouTube/Google had the highest score across the sample with 4.5/10 stars, and Alibaba and Disney+ had the fewest stars with 1.5/10. Each company sampled provided access to privacy materials via its homepage. All companies also provided information about applicable laws/regulations. Most companies provided details about data storage/processing and about half describe data retention policy. Few provided details about how AI systems work, how AI systems link to company practices, or risks of AI use. Most companies provided some form of information about AI or machine learning on a site away from the privacy policy. To ensure the auditability and accountability of AI systems, companies are encouraged to improve upon these transparency efforts by better-aligning with the calls for AI transparency in The White House Blueprint for an AI Bill of Rights. Accessible and plain language notices are recommended, as is the inclusion of information about how AI systems work at each company, and the associated implications and risks of automated decision-making that may result from digital service use.