The use of AI to abuse others is theoretically prevented by internal guardrails designed to prevent misuse. These guardrails have so far been found inadequate. The question then becomes whether they will ever be adequate.
Chester Wisniewski, Field CTO of applied research at Sophos, thinks not. “Any system to prevent abuse, but seemingly designed to let the system operate autonomously, will most likely always be able to be bypassed.” He hopes it will become harder as OpenAI and other AI researchers learn more about how jailbreaking is performed, “but it will still be possible.”
John Bambanek, principal threat hunter at Netenrich, also has doubts. “The fundamental cybersecurity problem is how to perform automation on untrusted inputs – and we are nowhere on solving that.”
“I doubt it is possible to create a GPT model that can’t be abused,” adds Mike Parkin, senior technical engineer at Vulcan Cyber. “The challenge long term will be keeping threat actors from abusing the commercially available AI engines. Ultimately though, it will be impossible to keep them from creating their own and using them for whatever purposes they decide.”
But there are some hopeful thoughts. Stephanie Aceves, senior director of product management at Tanium, accepts the task is like achieving cybersecurity by turning the computer off and locking it in a vault – but the task should not preempt the attempt. “Risk should not be a showstopper,” she said. “Rather it should be an input to the policies, programs, and guardrails we develop.”
Polyakov doesn’t believe that much can be done to prevent the misuse of AI without a legal framework. But he believes that AI developers can do more to protect their own security. “It will be hard work and a continuous cat and mouse game, but it’s certainly possible to make AI much more secure.”
He further believes that making the systems more secure will have the byproduct of making them more difficult to misuse. “What is more important and amazing is that in making AI models more secure, you may make them more robust and accurate as a byproduct.”
He warns that this should be done sooner rather than later. “The earlier companies start initiatives, the better they will protect their systems and have a competitive advantage. Sometimes the goal is not to be 100% secure but to be more secure than your neighbor.” Criminals tend to attack the easiest target.
But it may already be too late. “It will never be possible to create a large language model that cannot be abused,” suggests Andy Patel, senior researcher at WithSecure. “Prompt injection aside, many underhanded uses of NLG [natural language generator] models rely on generating text designed to persuade or trick people. Prompts designed to create this type of content, by nature, don’t trigger safety filters or the model’s refusal policy.”
He points to developments outside of OpenAI. “Recent ML advances have precipitated models such as Alpaca, a 6B parameter model that works approximately as well as GPT-3, and that can be run on a laptop. Those models are good enough to generate content that could be used for malicious purposes. I would expect bad guys to be more interested in those models than GPT-4 – at least in the near future. Eventually GPT-4-strength models will be available to all in the same way.”
Privacy is one of the areas considered most at risk from an unfettered use of AI, and an ethical or unethical implementation of the technology will drive the extent of privacy abuses. SecurityWeek spoke to Christina Montgomery, chief privacy & trust officer, and AI ethics board chair at IBM. “The technology is clearly moving faster than society’s ability to build reasonable guardrails around it, and there’s still not enough transparency around how other tech companies are protecting the privacy of data that interacts with their systems,” she said.
The solution must come from both government and industry: government in strong regulations, and industry in a clear ethical use of AI. “There is a real need for our government leaders to work with the private sector on effective, risk-based AI regulation – where the tightest regulatory control focuses on the AI applications with the greatest risk of societal harm.” But while AI must be regulated, privacy must also be protected. “We need a consistent, national privacy law in this country,” she added.
She acknowledges that ChatGPT has guardrails ins place, but with clear shortcomings. “That makes proper oversight even more important, especially in a consumer context.” At the enterprise level, she believes the solution must lie in greater use of the principles of ethical use. “We’re at an early stage in public exposure to AI – and I firmly believe that every company involved in this work has an obligation to strengthen trust in the technology.”
She notes that IBM has established ethical principles to the development and use of AI, including questioning not whether something can be done, but whether it should be done. “Our focus is on developing technologies, including generative AI tools, with responsibility and ethics at the forefront and then urging other private sector developers to do the same.”
Finally, she adds, “People need to see more companies leading by example, putting ethics, responsibility, and people’s interests first. ChatGPT is a reminder that these technologies are getting more powerful and that the era of move fast and break things must end.”
Absent regulation, however, privacy abuses will likely continue. “The reason big tech companies collect so much data is to have the training data to create tools like GPT4 in the first place,” comments Bambanek. “We are only scratching the surface of the risks this poses, mostly because the primary use case for these companies is advertising. As long as people are willing to tolerate the privacy invasions for cheap/free service, there isn’t much that will slow this down.”
Aceves is more optimistic. “The short answer is yes, something can be done,” she says. “Organizations like The Cyber Collective are leading the way in educating the average person and initiating change in our current policies.”
Steve Wilson, CPO at Contrast Security, tasked ChatGPT to answer some of the questions SecurityWeek was asking, as if it were an ethics professor. “While it might not create entirely new ideas, it can produce novel combinations of existing knowledge and concepts,” responded ChatGPT about ChatGPT.
But that ‘existing knowledge may be false and incomplete. “AI models like ChatGPT can indeed learn and propagate inaccuracies or biases present in the training data,” continued the fake professor. The old adage of ‘garbage in, garbage out’ still applies to the new technologies.
“To address this issue, AI developers must continuously improve the training process by curating diverse, high-quality datasets and incorporating methods to mitigate bias.” But this is the conundrum underlying all ethical attempts to remove bias — it is led by people with existing, perhaps unconscious, biases of their own.
On March 31, 2023, the Italian data protection regulator blocked ChatGPT over concern that it is unlawfully processing personal data protected by GDPR. OpenAI has 20 days to respond, but the potential to hallucinate (give out false information) adds a further complication if it spreads false information about European residents. Then comes the issue of the European ‘right to be forgotten’. How does AI forget?
A further complication comes through other software that uses OpenAI’s GPT. Microsoft has invested billions in OpenAI, and is clearly very close to the company. ChatGPT has been incorporated into Bing and can be used via Skype.
If it is found that these internet searches gather PII on GDPR-protected citizens from untrusted internet websites, will Italy block Microsoft?
The Future of Life Institute published an open letter on March 29, 2023: “We call on all AI labs to immediately pause for at least 6 months the training of AI systems more powerful than GPT-4.” The letter cites the Asilomar AI Principles, a recognized list of AI governance principles – #20 of which states, “Advanced AI could represent a profound change in the history of life on earth, and should be planned for and managed with commensurate care and resources.”
Response from within the security industry to the letter has been varied. Dan Shiebler, head of machine learning at Abnormal Security, commented, “Personally, I don’t think this letter will achieve much. The cat is out of the bag on these models. The limiting factor in generating them is money and time, and both will fall rapidly. We need to prepare businesses to use these models safely and securely, not try to stop the clock on their development.”
Chenxi Wang, founder and general partner at Rain Capital, is in favor of a pause. “A pause in the AI fever is needed, not just from the business standpoint, but also from the point of view of security and privacy. Until we understand how to assess data privacy, model integrity, and the impact of adversarial data, continued development of AI may lead to unintended social, technical, and cyber consequences.”
But in the final analysis, we will not be able to halt the continued evolution of AI. Two of humanity’s greatest drivers, greed and curiosity, will push AI development forward. Our only hope is that we can control it. The genie is out of the bottle, and isn’t offering any wishes.
Related: Cyber Insights 2023 | Artificial Intelligence
Related: White House Unveils Artificial Intelligence ‘Bill of Rights’
Related: Bias in Artificial Intelligence: Can AI be Trusted?
Related: The Starter Pistol Has Been Fired for Artificial Intelligence Regulation in Europe
