A collaborative ensemble of reinforcement learning agents for automated network penetration testing
Open AccessOrganizations use network penetration testing to assess preparedness for cyber-attacks. Existing penetration testing methods lack scalability, automation, and learning capabilities to facilitate adapting to new scenarios. Some methods require extensive human operation, are only useful for planning, lack scalability, or require perfect information about a network. This research proposes an automated Reinforcement Learning-based system capable of emulating human penetration testing. Popular Reinforcement Learning methods are designed to maximize a learning agent’s rewards from decisions made based on state/action pairs. The proposed method maximizes the agent’s ability to explore the network and the number of hosts discovered and compromised. The method's effectiveness in network exploration and host exploitation provides a solid foundation for future research. The goal is to develop an intelligent, automated system capable of scaling to perform penetration testing of complex networks with multiple subnets, services, and potential exploits. Our research demonstrates the superior performance and scalability of ensemble reinforcement learning in automated penetration testing, particularly in larger real-world enterprise networks, compared to traditional reinforcement learning algorithms. The novelty of our method is two-fold. First, to our knowledge, this is the first study to implement an ensemble of diverse types of Reinforcement Learning algorithms to the problem of automated network penetration testing. Second, existing reinforcement learning ensemble methods use each algorithm’s decision in a weighted or voted-in manner to determine the final decision regarding which action the agent should take. Our ensemble focuses on calculating the Q-values that the agent will use when determining what action to take regardless of the action each of the individual algorithms would have taken given its own calculated Q-values.
- All rights reserved
Notice to Authors
If you are the author of this work and you have any questions about the information on this page, please use the Contact form to get in touch with us.