Current penetration testing or red-teaming activities to evaluate the cyber resilience of a system mostly rely on subsets of known vulnerabilities or procedures. Despite the use of these techniques, it is hard to assess the process and risks falling into repetitive patterns, which do not effectively validate the resilience of the system against potential zero-day attacks. Countermeasures in the system-under-test are often left out of the cyber resilience evaluation phase. We propose CERERE-an automated framework designed to measure and test the cyber resilience of complex IT systems, such as critical national infrastructure and military networks. CERERE simulates the effects of attacks on the system regardless of exploitation methods. The framework consists of war gaming exercises where attacker and defender modules interact in a simulated test environment to allow a dynamic evaluation of resilience. The attacker module uses heuristic algorithms to generate kill chains, while the defender module leverages AI-based algorithms to simulate defense strategies. CERERE has been validated by evaluating the resilience of a given scenario and identifying the optimal configuration of responses and countermeasures. This paper was originally presented at the NATO Science and Technology Organization Symposium (ICMCIS) organized by the Information Systems Technology (IST)Scientific and Technical Committee, IST-209-RSY- the ICMCIS, held in Oeiras, Portugal, 13–14 May 2025.