Holding A.I. and Tech Accountable

Last December, San Francisco experienced a massive blackout that impacted the majority of the City and left us in the darkness for hours, and – for parts of San Francisco – for days. This impacted our street and traffic lights, causing autonomous vehicles (AVs) to stop in place, jamming up intersections and blocking emergency response vehicles.

According to the San Francisco Fire Department (SFFD), this was not the only time that AVs have blocked our roadways and impeded emergency response. Since April of last year, SFFD has documented at least 31 instances where response has been delayed due to an AV blocking the road. In addition to this, according to the San Francisco Chronicle, SFFD’s EMTs have seen an increase in emergency calls from AVs to address passengers who have fallen asleep in the driverless cars. Not only do the delays in response jeopardize our public safety, the non-life threatening safety calls are a drain on our limited public resources.

Although there have been some attempts at AV regulation at the state level, our medical and fire response have been left out of these regulations. In order to include all of our first responder agencies in oversight, I am drafting legislation to hold AV companies accountable for the emergency situations they create or exacerbate on our public roads. This legislation will work to accomplish two goals. The first will help identify the public safety risks created by faulty AV technology, in order to create a full picture of the safety situations on our roads. The second will allow the City to recover our costs in instances where an AV has caused a false emergency report or impedes emergency responses.

As we continue to see the use of A.I. and other technological tools grow, whether it’s through AVs or other technologies, San Francisco must have guardrails to protect our residents and workers, to ensure A.I. is serving the public, not just profiting corporations, especially if it impacts public safety. My job is to ensure that we have a people-centered approach to regulate A.I. Just recently, we have seen news of a robot boss firing a human worker in our City. This is in addition to reports of A.I. agents hacking rival companies without authorization and, indeed, without the knowledge of their human programmers.

These incidents prove that we have to pause data center development and push for regulation and mandate human oversight to hold these companies accountable, prevent these occurrences and ensure there are consequences for these violations. Under S.F. Mayor Daniel Lurie’s initiatives, the City is welcoming more investment by A.I. companies downtown. This is when we also need to do more to ensure that A.I. and tech companies who are profiting from public roads and spaces in San Francisco are paying their fair share and making an equal contribution to our City. We must have a plan that’s people-centered so we don’t have to play catch-up later and repair damage done by A.I. technologies.

Connie Chan represents District 1 on the San Francisco Board of Supervisors. She can be reached at 415-554-7410 or chanstaff@sfgov.org.