Collaborative research on AI safety is vital | Artificial intelligence (AI)

On Geoffrey Hinton’s concerns about the dangers of artificial intelligence (‘The Godfather of Artificial Intelligence’ reduces the chances of technology wiping out humanity within the next 30 years, December 27), I believe these concerns can best be mitigated through collaborative research on AI safety with the participation of regulators.

Currently, Frontier AI tested after development using “red teams” who try their best to achieve a negative result. This approach will never be enough; AI must be designed for safety and assessment purposes – this can be achieved by drawing on knowledge and experience from well-established safety-related industries.

Hinton doesn’t seem to think that existential threat from AI is being intentionally encoded – so why not force conscious avoidance of that scenario? While I do not share his views on the level of risk humanity faces, the precautionary principle suggests that we must act immediately.

In traditional safety-critical fields, the need to build physical systems such as aircraft limits the speed at which safety can be impacted. Frontier AI has no such physical “speed limiter” when deployed, and this is where regulation has a role to play. Ideally, a risk assessment would be carried out before deployment, but current risk metrics are insufficient – for example, they do not take into account the application sector or the scale of deployment.

Regulators need the ability to “recall” used models (and large companies that develop them should include mechanisms to stop specific uses), as well as support for risk assessment work that provides leading indicators of risk, not just lagging indicators. In other words, the government needs to focus on post-market regulatory controls while supporting research that will provide regulators with information to enforce pre-market controls. It’s a difficult task, but a necessary one if Hinton is right about the level of risk humanity faces.
Professor John McDermid
Institute for Secure Autonomy, University of York