In a world where not all employees at any given AI research lab believe that AI presents a large and present danger, I think insider threats are a huge factor to consider.
Yes, there's a lot of disagreement about policies regarding model opensourcing especially. It seems likely to me that some employees at large AI labs (Google Brain Deepmind, Meta, Microsoft Research, etc.) will always disagree with the overall policy of their organisation. This creates a higher base rate risk of insider threats.
Summary
Insider threats are security risks caused by an organisation's staff. Careless and intentional insider threats cause 25+% of cyber breaches.[1]
In a survey of existential risks involving cybersecurity and AI, I identified key actors in the AI supply chain to protect: compute manufacturers, frontier AI labs, and AI regulatory agencies.
These actions can help key actors reduce insider threats:[2]
Details on Individual Factors
It's difficult to identify demographics factors that correlate with insider threats.
Next, there are correlations between "dark personality traits" and staff who intentionally cause insider threats.
Instead, organisations can proactively monitor and encourage employee wellbeing.
The last point is important since insider threats are more often caused by employee error than malicious intent.[1] Extremified employee monitoring programs will reduce privacy, trust, and wellbeing.[4] Proactive wellbeing supports have low side effects.
Details on Organisational Factors
First, employees who work together are best able to observe each others' wellness. Thus, an organisation's first priority is to increase "see something, say something" behaviour amongst all employees. What gets in the way?
Separately, some more tangible organisational policies also have an impact.
Sadly, an organisation can easily do all the right things on paper but cause a distrustful, inefficient, and toxic workplace. Though intangible, culture is critical. Even one remark by managers can have a large impact:[5]
Details on Technical Factors
The general theme is that technical safeguards are necessary, but not sufficient. Many solutions are reactive ways to prevent insiders from causing damage.
There are many standard defences to briefly name: spam filters to prevent social engineering against staff; principle of least privilege to limit any one employee's access to sensitive information; zero trust architectures to prevent colleagues' access from being exploited; network segmentation, proxies, and firewalls to prevent damages from spreading; intrusion detection systems/anomaly detection systems to spot suspicious activities like sensitive data being exported.
While there are many improvements being researched to the above standard technologies, here are some improvements relevant to insider threats:
To start, it's common to monitor device-specific data for outliers. This data reveals some information about the user (unique keystroke patterns, common actions on the device, etc.). Still, it doesn't reveal much about user motivations. Complementing device-based data sources with HR data reveals employee motivations.
A similar conclusion is possible with psychometric data. That said, psychometric data sources are ethically-contentious and publicly unavailable to develop defensive tools with. Specifically, psychological questionnaires may be seen as cumbersome or overbearing by employees. Whereas automated data collection tools like social media crawlers may be seen as privacy violations.[4]
These "contextual" data sources can make other insider threat detection systems more adaptive. For example, access management often has static policies set for each team.[6] If an employee has a low trustworthiness score due to some recent logins at suspicious times, a dynamic access management system could temporarily revoke the employee's access to certain sensitive documents.
As seen, the general trend with improving technical defences against insider threats at an organisation is to get more holistic (and human) data which is used to adapt defences over time.
Personally, I've been intrigued to learn about all these human-focused best practices to reduce insider threats. I'm hoping to get more primary data by interviewing cyber security staff at AI labs and compute manufacturers. Any suggestions on who to reach out to are much appreciated!
G. Bassett, C. D. Hylender, P. Langlois, A. Pinto, and S. Widup, “2022 Data Breach Investigations Report,” Verizon Communications Inc., 2022. Accessed: Nov 15, 2022. [Online]. Available: https://www.verizon.com/business/resources/T501/reports/dbir/2022-data-breach-investigations-report-dbir.pdf
Black, Marigold, et al. Insider Threat and White-Collar Crime in Non-Government Organisations and Industries: A Literature Review. RAND Corporation, 2022, https://doi.org/10.7249/RRA1507-1.
L. Liu, O. De Vel, Q. -L. Han, J. Zhang and Y. Xiang, "Detecting and Preventing Cyber Insider Threats: A Survey," in IEEE Communications Surveys & Tutorials, vol. 20, no. 2, pp. 1397-1417, 2018, doi: 10.1109/COMST.2018.2800740.
J. Love and F. Schmalz, ‘Companies Now Have Many Tools to Monitor Employee Productivity. When Should They Use Them?’, Kellogg Insight. Available: https://insight.kellogg.northwestern.edu/productivity-monitoring. [Accessed: Oct. 04, 2023]
A. Moore, S. Perl, J. Cowley, M. Collins, T. Cassidy, N. VanHoudnos, P. Buttles-Valdez, D. Bauer, A. Parshall, J. Savinda, E. Monaco, J. Moyes, and D. Rousseau, "The Critical Role of Positive Incentives for Reducing Insider Threats," Carnegie Mellon University, Software Engineering Institute's Digital Library. Software Engineering Institute, Technical Report CMU/SEI-2016-TR-014, 15-Dec-2016 [Online]. Available: https://doi.org/10.1184/R1/6585104.v1. [Accessed: 5-Oct-2023].
J. Crampton and M. Huth, ‘Towards an Access-Control Framework for Countering Insider Threats’, in Insider Threats in Cyber Security, C. W. Probst, J. Hunker, D. Gollmann, and M. Bishop, Eds., Boston, MA: Springer US, 2010, pp. 173–195. doi: 10.1007/978-1-4419-7133-3_8. Available: https://link.springer.com/10.1007/978-1-4419-7133-3_8. [Accessed: Oct. 05, 2023]