UK AI Safety Fears Grow as Labour Weighs Risks and Benefits

Artificial Intelligence (AI) safety is emerging as a major concern within the UK government, with senior Labour figures increasingly worried about the potential for powerful AI systems to be exploited by extremist groups, cybercriminals and other malicious actors. As the technology advances, ministers face growing pressure to strengthen safeguards without sacrificing the economic and public-service benefits that AI could deliver.

The debate has intensified around concerns that increasingly capable AI models could help attackers breach computer systems, develop dangerous biological agents or carry out sophisticated operations that were previously beyond their reach. At the same time, policymakers are grappling with questions about how quickly the government should adopt AI and whether excessive caution could leave the UK behind other countries.

During the Labour Party conference, several senior figures reportedly identified AI as a growing source of anxiety alongside the cost of living and the country’s fragile economic outlook. One cabinet minister warned that AI could soon dominate political debate, reflecting a wider recognition that the technology is becoming an issue of national security, economic competitiveness and public trust.

The central challenge for the government is increasingly clear: how can Britain benefit from AI while preparing for threats that may develop faster than existing laws, security systems and international agreements can respond?

Why AI Safety Is Becoming a Major Concern in Westminster

Concern about AI has moved beyond debates among technology specialists and into the centre of government discussions. Senior Labour figures are increasingly focused on the possibility that advanced systems could concentrate enormous capabilities in the hands of individuals or organisations seeking to cause harm.

One cabinet minister described the most worrying scenario not as a science-fiction vision of killer robots but as the prospect of highly capable AI technology becoming available to malicious actors. Such systems could potentially help people carry out complex activities, including cyberattacks or the development of dangerous technologies, with fewer technical barriers than before.

The concern is not limited to the possibility of AI acting independently. A major risk is that people could use AI tools to increase the speed, scale or sophistication of activities they already conduct. Criminal networks, hostile state-linked groups and extremists could attempt to exploit the technology to identify vulnerabilities, automate parts of an attack or generate convincing material for manipulation.

Another senior Labour figure reportedly warned that the government was falling behind the public in responding to concerns about AI Safety. The observation points to a widening challenge for policymakers: as people encounter AI in their workplaces, schools and daily lives, expectations about safety, transparency and accountability are changing.

Political leaders must therefore respond to two related pressures. They need to address the risks identified by security experts, while also demonstrating that government institutions understand how rapidly the technology is developing.

Energy Secretary Ed Miliband was among the prominent voices raising concerns at the Labour conference. He described the potential dangers of AI as daunting and argued that the financial interests of technology companies should not come at the expense of children or humanity.

His remarks reflected a broader political question about the relationship between commercial innovation and public protection. AI companies have strong incentives to develop increasingly capable products, but governments are responsible for ensuring that technological progress does not create unacceptable risks for society.

AI Minister Warns of Potential Attacks on British Government Systems

The debate gained momentum when AI minister Kanishka Narayan warned that it was likely, if not inevitable, that rogue actors would attempt to hack the British state using increasingly sophisticated capabilities. His comments placed Cybersecurity at the centre of the government’s immediate AI concerns.

Government departments and public institutions hold large quantities of sensitive information and operate systems essential to everyday life. These include services connected to healthcare, taxation, public administration and national security. An attack that compromises important systems could disrupt public services, expose confidential information or create wider economic consequences.

AI can potentially assist cyber attackers by helping them analyse software, identify weaknesses, generate malicious code or automate parts of reconnaissance. It can also help create more convincing phishing messages and other deceptive communications, making it harder for individuals and organisations to distinguish legitimate activity from an attack.

However, the risks must be understood in context. AI does not automatically make every cyberattack successful, and attackers still face technical barriers, security controls and the need to exploit real vulnerabilities. The concern is that more capable tools could lower the effort required for certain activities and allow less experienced attackers to attempt operations that previously demanded greater expertise.

Narayan said the UK was looking to strengthen its forthcoming cybersecurity legislation in response to the changing threat environment. Such measures could form part of a wider effort to improve the security of critical systems, strengthen organisational preparedness and ensure that institutions can respond to emerging forms of digital attack.

The policy challenge is to create protections that remain effective as attackers and defenders adopt new technology. Rules designed around earlier generations of cyber threats may need to be reconsidered as AI changes how vulnerabilities are discovered, how attacks are organised and how quickly malicious activity can spread.

MI5 Chief Highlights the Growing Cybersecurity Contest

Security concerns received further attention after MI5 chief Ken McCallum warned that AI was intensifying the contest between cybersecurity attackers and defenders. His assessment reinforced the view that the technology could change the capabilities available to both sides.

For defenders, AI can help analyse large volumes of security data, detect suspicious activity, identify unusual behaviour and support incident response. These applications could allow security teams to identify threats more quickly and manage complex systems more efficiently.

Attackers can also use the technology to improve their operations. AI-assisted tools may help criminals refine their tactics, search for weaknesses and produce more persuasive social-engineering attempts. As both sides develop new capabilities, security organisations face pressure to improve their systems continuously rather than relying on static defences.

McCallum also highlighted the potential threat posed by extremist groups seeking to use AI to develop or deploy biological weapons. These weapons involve biological agents or toxins intended to cause disease, injury or death.

AI systems that provide scientific information could potentially be misused to explore dangerous biological concepts or assist with parts of research that would otherwise require substantial specialist knowledge. The concern is that increasingly capable models might lower some barriers to understanding complex scientific processes.

Nevertheless, there is an important distinction between obtaining information about a dangerous biological agent, designing a potentially harmful organism and successfully producing and weaponising it. Biological research involves practical, technical and logistical challenges that cannot necessarily be overcome through AI-generated instructions alone.

Security experts are therefore concerned about the potential for AI to make certain stages of harmful activity easier, even though the extent of the risk varies by application and remains an important subject of assessment.

Labour’s Change of Leadership and the Debate Over Big Tech

The increased political attention has also prompted questions about the government’s relationship with major technology companies. Liam Byrne, chair of the parliamentary committee responsible for business, innovation, science and trade, argued that the change from Keir Starmer’s administration to Andy Burnham’s government had contributed to a shift in the political approach to AI safety.

Byrne said the previous administration had been too close to large technology companies, while the new government appeared less aligned with their interests. Another senior Labour figure similarly suggested that some members of the previous administration had placed too much faith in the technology sector’s approach to innovation.

These are political assessments rather than established findings about the effectiveness of either administration’s policies. They nevertheless highlight a persistent tension in AI governance: technology companies possess much of the expertise and computing infrastructure needed to develop advanced systems, but their commercial priorities do not always align perfectly with the public interest.

Companies need to innovate, attract customers and compete internationally. Governments, meanwhile, must consider the consequences of AI for national security, employment, privacy, public services and democratic institutions.

Close cooperation between the two sides can help policymakers understand the technology and develop workable standards. However, relying entirely on voluntary industry commitments may leave questions about accountability, independent testing and enforcement unresolved.

The debate is therefore shifting towards how governments can work with AI developers while retaining sufficient oversight to assess risks independently and intervene when necessary.

OpenAI, Anthropic, DeepMind and Meta Face Questions Over AI Testing

Byrne’s parliamentary committee held an evidence session involving senior civil servants responsible for AI security and representatives of major AI developers, including Anthropic, OpenAI, DeepMind and Meta.

A key issue was whether governments should be able to require testing of new AI models before or around their release. The discussion reflects growing concern that developers may introduce systems with powerful capabilities before regulators have had sufficient opportunity to assess their potential misuse.

Testing can help identify weaknesses, examine how a model responds to dangerous requests and evaluate its ability to assist with sensitive tasks. Depending on the system, assessments may include cybersecurity evaluations, biological risk testing and attempts to bypass safety restrictions.

Mandatory testing could give governments a more formal role in determining whether certain models present unacceptable risks. However, the design of such requirements would matter considerably. Regulators would need to establish which systems require testing, what standards should apply, who conducts the assessments and how the results should influence release decisions.

Companies may also argue that poorly designed requirements could delay useful products, impose disproportionate costs on smaller developers or expose sensitive technical information. Policymakers must weigh these concerns against the potential consequences of releasing systems that have not been adequately assessed.

One possible area of cooperation is the development of shared testing standards that allow companies and governments to assess advanced models using comparable methods. International coordination could also reduce the risk of inconsistent rules that encourage developers to move sensitive activities to jurisdictions with weaker oversight.

For the UK, the challenge is to establish safeguards that are credible enough to address security concerns without unnecessarily restricting beneficial research and innovation.

US-UK Cooperation Faces Questions Over Access to Advanced AI Models

International cooperation is another major issue for British policymakers. The UK has sought to work with the United States on AI security, including the testing and monitoring of advanced models. However, concerns have emerged over whether American technology companies would be able to share their latest systems with Britain’s AI Security Institute.

Reports suggested that the administration of US President Donald Trump had asked AI companies not to share their newest models with the UK institute for testing. Narayan subsequently said that cooperation had been retained, indicating that the two countries continued to engage on the issue.

The episode illustrates how AI security has become entangled with international politics, technology competition and national security priorities. Governments want access to advanced systems so that their capabilities can be evaluated, but companies must also consider restrictions, commercial interests and political pressures in the countries where they operate.

Byrne argued that the UK should take advantage of the current moment to strengthen its regulatory approach. He also pointed to the possibility that changes in the balance of power in the US Congress following the midterm elections could alter the political environment surrounding AI oversight.

The broader dilemma is that Britain wants to maintain access to leading AI technology while ensuring that adequate safeguards are in place. Restricting cooperation too heavily could limit the UK’s ability to study important developments, but depending excessively on decisions made by foreign governments and private companies could weaken its ability to manage risks independently.

Byrne said technology companies themselves had reasons to avoid releasing systems capable of causing serious harm, but they also had to navigate the political priorities of the US administration. He argued that the British government needed to make decisions based on the country’s interests rather than allowing uncertainty over international relations to delay action.

International cooperation will remain important because AI systems, digital attacks and scientific information can cross borders quickly. No single country can address every aspect of the threat in isolation.

Why Open-Source AI Models Are Raising Additional Safety Concerns

Another focus of the debate is the development of open-source or openly available AI models. Unlike some commercial systems that are accessed through controlled services, certain models can be downloaded, modified and adapted by users.

Greater accessibility can encourage research, competition and innovation. It can also allow developers, businesses and researchers to customise systems for particular tasks without relying entirely on a small number of technology companies.

However, the same accessibility creates a challenge for safety oversight. Once a model is widely distributed, its original developer may have limited ability to monitor how it is used or prevent modifications that remove built-in safeguards.

Geoffrey Irving, a former chief scientist at the UK’s AI Security Institute, warned that openly available models could already be powerful enough to cause serious harm. He also suggested that such systems could approach the capabilities of leading commercial models relatively quickly.

Irving’s concern centres partly on the possibility that safety restrictions in some open models may be absent, incomplete or relatively easy to remove. A user who modifies a model could potentially weaken protections intended to prevent assistance with harmful activities.

The speed at which open models improve complicates the work of regulators. A model that appears less capable than a leading commercial product at one point may become substantially more useful after further research, training or adaptation.

At the same time, openness is not inherently dangerous, and not every openly available model has the same capabilities or risk profile. The potential for misuse depends on factors such as model performance, accessibility, safeguards, the resources required to operate it and the specific tasks it can support.

Regulators must therefore avoid treating all models as identical. Risk-based approaches could distinguish between systems with limited capabilities and those that can materially assist with high-impact cyber or biological threats.

Could AI Help Create Biological Weapons?

AI-enabled biological threats have become an important part of the debate over the future of advanced models. As AI systems become more capable of processing scientific information, security experts are examining whether they could help malicious actors explore dangerous biological techniques or design harmful agents.

The concern is that AI might reduce the amount of specialist knowledge required to investigate certain scientific questions. A sufficiently capable system could potentially help users navigate technical literature, understand biological processes or identify approaches that would otherwise be difficult for a non-specialist to evaluate.

However, biological weapons present challenges that go beyond designing a dangerous pathogen. Successfully producing, handling and deploying biological agents requires access to materials, suitable facilities, practical expertise and the ability to overcome numerous scientific and operational barriers.

Experts have therefore highlighted the gap between the theoretical design of a harmful biological agent and the ability to turn that design into an effective weapon. The existence of that gap does not eliminate the risk, but it is important when assessing how likely different forms of misuse may be.

Effective safeguards could involve evaluating whether AI models provide meaningful assistance with dangerous biological tasks, monitoring changes in their capabilities and developing controls for particularly sensitive applications.

International collaboration is also important because biological risks are not confined to national borders. Governments, scientific institutions and technology developers may need shared procedures for identifying emerging threats and responding to them without unnecessarily restricting legitimate scientific research.

Is the Risk of AI Wiping Out Humanity Credible?

Concerns about cyberattacks and biological threats have also revived wider questions about whether increasingly advanced AI could pose an existential risk to humanity. Some members of the AI research community worry that systems with exceptionally high capabilities could eventually behave in ways that humans cannot reliably control.

These scenarios differ from more immediate forms of misuse. Cybersecurity risks can involve identifiable attempts to compromise existing systems, while biological risks concern the potential misuse of scientific knowledge. Existential-risk scenarios involve much more uncertain possibilities about the future behaviour and capabilities of advanced AI.

Irving shares concerns about the potential for powerful AI systems to create severe risks. Other experts, however, argue that policymakers should prioritise threats that are already visible and supported by stronger evidence.

Professor Elena Simperl, co-director of the Institute for Artificial Intelligence at King’s College London, challenged the emphasis on existential risk, arguing that the case remained theoretical and lacked sufficient evidence to establish it as a credible immediate threat.

Simperl pointed instead to more immediate concerns, including hacking and evidence suggesting that AI products could become highly engaging in ways associated with the addictive design of social media platforms.

The disagreement highlights an important challenge for governments deciding how to allocate limited resources. Preparing for extreme future scenarios may be prudent when the consequences could be severe, but policymakers must also address harms that are already occurring or can be demonstrated more directly.

A balanced strategy would assess different risks according to their evidence, potential impact, likelihood and urgency. Cybersecurity protections, safeguards against dangerous biological applications and measures to protect users can be pursued alongside research into longer-term questions about advanced AI systems.

Concerns Over the UK’s Defence AI Capabilities

AI safety is also linked to Britain’s ability to defend itself against emerging forms of digital and military threats. Some defence sources have expressed concern about the cancellation of an AI programme at the government’s Defence Science and Technology Laboratory, based at Porton Down.

The concern is that reducing dedicated research could limit the country’s ability to develop advanced capabilities and understand how adversaries might use AI in future conflicts. As militaries explore AI-assisted intelligence analysis, autonomous systems and other applications, defence organisations need expertise in both the technology’s potential benefits and its vulnerabilities.

AI could help military planners process large volumes of information, support logistics and improve the analysis of complex operational environments. It could also be used by adversaries to enhance cyber operations, automate certain tasks or develop systems that challenge existing defences.

The debate is therefore not simply about whether the military should adopt AI. It is also about how research, testing and risk management should be organised so that new capabilities can be developed responsibly.

The Ministry of Defence disputed the suggestion that AI investment was being neglected. A spokesperson said AI was transforming defence across industry and the battlefield and should be integrated across different areas of work rather than treated as a standalone programme.

The ministry also pointed to a planned investment of £400 million in AI-focused capabilities over four years, intended to prepare the UK for future warfare.

The contrasting positions reflect two different concerns: whether the government is investing enough in specialised AI research and whether AI development is better managed through a broader programme that embeds the technology across defence activities.

Determining the effectiveness of either approach will depend on implementation, research capacity, workforce skills and the ability to translate investment into operational capabilities while maintaining appropriate oversight.

Could AI Safety Rules Slow Britain’s Economic Progress?

Not everyone in government is primarily concerned that AI safeguards are too weak. Some officials worry that an excessive focus on potential dangers could discourage the public sector from adopting technology that might improve productivity and address longstanding problems.

AI could support a range of public services, including medical administration, document processing, information management and the analysis of operational data. In healthcare, for example, carefully implemented systems could assist with administrative tasks and help organisations manage information more efficiently.

Such applications could be particularly valuable in services under pressure from rising demand, limited resources and long waiting lists. However, technological adoption does not automatically translate into better outcomes. Systems must be evaluated, integrated into existing workflows and monitored to ensure that they are reliable and appropriate for the tasks they perform.

Public institutions also need to consider privacy, data security, transparency and accountability. An AI system that produces incorrect information or makes an unreliable recommendation can undermine trust, particularly when it affects access to essential services.

The former cabinet secretary Gus O’Donnell argued that ministers needed to pay attention to the risk of being left behind as well as the need for safeguards. Speaking in connection with a Global Government Forum report on AI produced with technology company Cognizant, he expressed optimism about the technology’s potential while emphasising the importance of managing risks and avoiding mistakes that could damage public confidence.

His argument reflects a broader concern that governments must learn to use AI effectively rather than treating it exclusively as a security problem. If public institutions are unable to adopt useful technology, they may miss opportunities to improve services, reduce administrative burdens and make better use of limited resources.

At the same time, rushing to deploy poorly tested systems could create costly failures and undermine the public’s willingness to accept future innovations. The objective is therefore not simply to adopt AI faster, but to build the institutional capacity needed to use it safely and effectively.

Why the UK’s G20 Presidency Could Matter for AI Regulation

International cooperation is likely to play an important role in the UK’s efforts to manage AI risks. Britain’s forthcoming G20 presidency offers an opportunity to put the issue before a group of major economies and encourage discussions about shared approaches to safety, testing and accountability.

AI development is international by nature. Companies can develop models in one country, make them available to users elsewhere and face misuse involving infrastructure or victims across several jurisdictions. Differences in national rules can complicate efforts to assess risks and respond to harmful activity.

International coordination could help establish common principles for evaluating advanced AI systems, sharing information about emerging threats and improving cooperation between security agencies and technology developers.

However, reaching agreement among major economies will not be straightforward. Countries have different regulatory traditions, security priorities and economic interests. Some may place greater emphasis on preventing misuse, while others may prioritise rapid development and maintaining a competitive advantage.

The UK will therefore need to balance its ambitions for international leadership with practical proposals that governments and companies can implement. Shared testing methods, improved cybersecurity cooperation and clearer procedures for assessing high-risk models could provide areas for progress.

Domestic policy will remain important regardless of the outcome of international discussions. Britain will still need effective cybersecurity protections, appropriately resourced institutions and a clear approach to the adoption of AI across government.

The Challenge for Labour: Protect the Public Without Falling Behind

The growing focus on AI has exposed a difficult policy choice for the Labour government. On one side are concerns that increasingly capable systems could enable cyberattacks, facilitate dangerous scientific misuse or create risks that existing safeguards cannot adequately manage. On the other are the potential gains in productivity, public-service delivery and economic competitiveness.

These objectives do not necessarily conflict. Effective safety measures can help build confidence in AI, making it easier for organisations and members of the public to use the technology. Clear rules can also give developers a better understanding of the standards expected of them.

The difficulty lies in deciding which systems require the most scrutiny, how testing should be conducted and when the government should intervene. Requirements that are too weak may fail to prevent serious harm, while poorly targeted rules could impose unnecessary costs or delay beneficial applications.

Policymakers must also distinguish between different categories of risk. Cybersecurity attacks, biological misuse, privacy violations, potentially addictive applications and long-term concerns about highly advanced AI require different forms of assessment and response.

The government’s approach will need to combine technical expertise with independent oversight, cooperation with the private sector and a willingness to update rules as capabilities change. It will also need to ensure that public institutions have the skills and resources to evaluate AI systems rather than relying solely on assurances from their developers.

For Labour, the political stakes extend beyond regulation. Voters are likely to judge the government on whether it can protect essential services and national security while ensuring that Britain benefits from technological change.

The emerging debate in Westminster suggests that AI is becoming a central issue in how the UK approaches economic policy, defence and public administration. The challenge is no longer simply whether artificial intelligence should be adopted, but how the country can prepare for its risks while making productive use of its capabilities.

As ministers consider cybersecurity legislation, defence investment, model testing and international cooperation, the central question will remain whether the UK can move quickly enough to address emerging threats without missing the opportunities that AI offers. Striking that balance will be essential to maintaining public confidence and ensuring that technological progress serves the wider public interest.