We recently published "Practical Attacks on Machine Learning Systems", which has a very large references section - possibly too large - so we've boiled down the list to five papers that are absolutely essential in this area. If you're beginning your journey in ML security, and have the very basics down, these papers are a … Continue reading Five Essential Machine Learning Security Papers
This paper collects a set of notes and research projects conducted by NCC Group on the topic of the security of Machine Learning (ML) systems. The objective is to provide some industry perspective to the academic community, while collating helpful references for security practitioners, to enable more effective security auditing and security-focused code review of ML systems. Details of specific practical attacks and common security problems are described. Some general background information on the broader subject of ML is also included, mostly for context, to ensure that explanations of attack scenarios are clear, and some notes on frameworks and development processes are provided.
Introduction The work presented in this blog post is that of Ewan Alexander Miles (former UCL MSci student) and explores the expansion of scope for using machine learning models on PE (portable executable) header files to identify and classify malware. It is built on work previously presented by NCC Group, in conjunction with UCL’S Centre … Continue reading Machine Learning for Static Analysis of Malware – Expansion of Research Scope
In this blog post, I’m going to recount the story of my quest to train OpenAI’s large language model, GPT-2, to create a virtual doppelganger of myself and my peers. Machine learning is one of those buzzwords that, sometimes, lives up to its reputation. As an information security professional, my go-to hobby has typically been … Continue reading Impersonating Gamers With GPT-2
(Or, “Can large language models generate exploits?”) While attacking machine learning systems is a hot topic for which attacks have begun to be demonstrated, I believe that there are a number of entirely novel, yet-unexplored attack-types and security risks that are specific to large language models (LMs), that may be intrinsically dependent upon things like … Continue reading On the malicious use of large language models like GPT-3
Outline 1. Introduction2. How does MT19937 PRNG work?3. Using Neural Networks to model the MT19937 PRNG3.1 Using NN for State Twisting3.1.1 Data Preparation3.1.2 Neural Network Model Design3.1.3 Optimizing the NN Inputs3.1.4 Model Results3.1.5 Model Deep Dive184.108.40.206 Model First Layer Connections220.127.116.11 The Logic Closed-Form from the State Twisting Model Output3.2 Using NN for State Tempering3.2.1 Data Preparation3.2.2 … Continue reading Cracking Random Number Generators using Machine Learning – Part 2: Mersenne Twister
Outline 1. Introduction2. How does xorshift128 PRNG work?3. Neural Networks and XOR gates4. Using Neural Networks to model the xorshift128 PRNG4.1 Neural Network Model Design4.2 Model Results4.3 Model Deep Dive5. Creating a machine-learning-resistant version of xorshift1286. Conclusion 1. Introduction This blog post proposes an approach to crack Pseudo-Random Number Generators (PRNGs) using machine learning. By cracking … Continue reading Cracking Random Number Generators using Machine Learning – Part 1: xorshift128
For the past few years, NCC Group has been an industry partner to the Centre for Doctoral Training in Data Intensive Science (CDT in DIS) at University College London (UCL). CDT is composed of a group of over 80 academics from across UCL in areas such as High Energy Physics, Astrophysics, Atomic and Molecular Physics, … Continue reading Research Paper – Machine Learning for Static Malware Analysis, with University College London