How state officials think about implementing and managing generative AI

Generative AI in state governments: A team approach: NASCIO President Jim Weaver advocates for a collaborative strategy in managing generative AI implementation, rather than relying on a single chief AI officer.

• At the NASCIO 2024 Annual Conference, Weaver expressed skepticism about the need for chief AI officers, viewing them as potentially adding unnecessary bureaucracy.
• Weaver, who is also the North Carolina CIO, suggested that a team-based approach would be more effective in driving value and managing the complexities of generative AI adoption.

Creative applications and challenges: State officials at the conference explored innovative uses for generative AI while acknowledging the complexities involved in its implementation.

• Weaver proposed using generative AI for legislative code analysis, helping IT departments identify relevant clauses in seemingly unrelated bills.
• Missouri Deputy CIO Paula Peters highlighted the challenges of implementing chatbots, emphasizing the need for thorough preparation, including indexing millions of documents and rigorous training and vetting.
• The importance of getting AI implementations right the first time was stressed, as public trust can be easily shaken by inaccurate AI-generated responses.

Budgetary considerations: States must carefully evaluate the long-term financial implications of adopting AI technologies.

• Beyond initial purchase costs, states need to factor in ongoing maintenance expenses for AI systems and solutions.
• Projecting per-month costs for AI use cases can be challenging, requiring careful planning and budgeting.
• Indirect costs, such as potential increases in public records requests facilitated by AI, may necessitate additional staffing and resources.

Infrastructure challenges: On-premise AI implementations present unique infrastructure demands that states must address.

• Generative AI systems require significant electrical power, potentially straining local power grids.
• Water for cooling AI systems is another critical consideration, with Weaver noting that many data centers are not equipped for this requirement.

Vendor selection and transparency: Nebraska CIO Matt McCarville emphasized the importance of transparency in AI procurement for government agencies.

• Regardless of pricing or discounts, governments cannot afford to purchase AI systems without understanding their inner workings.
• CIOs need to comprehend algorithm functionality, potential biases, and error rates, especially when handling public data.

Broader implications for state governance: The adoption of generative AI in state governments raises important questions about resource allocation, infrastructure readiness, and public trust.

• States must balance the potential benefits of AI with the need for responsible implementation and management.
• The team-based approach advocated by Weaver suggests a shift towards more collaborative and interdisciplinary AI governance in state governments.
• As AI technology continues to evolve, state officials will need to remain adaptable and vigilant in addressing new challenges and opportunities in public sector AI adoption.