Software Hacking and Protection

Contents

In this course, we focus on the technical knowledge and skills, so-called hacking skills, required to attack and protect software assets. These assets are the most valuable and sensitive parts of programs that need to remain confidential in the face of reverse engineering and of which the integrity needs to be maintained in the face of software tampering attempts.

We study and practice a wide range of software analysis and tampering techniques, including side channel and faultinjection techniques that rely on physical implementation features, as well as protections such as obfuscations that aim to mitigate unauthorized uses of those techniques to protect assets and their security requirements.

We study and practice the complex tasks of deploying good combinations of protections and of evaluating the achieved levels of protection, including the design and execution of empirical experiments in the domain of reverse engineering and software protection.

Furthermore, we study and practice how to use a range of software analyses and forensic techniques to analyze and detect malware, as well the techniques that malware authors deploy in an attempt to evade detection.

The attacks and mitigations in the scope of this course are commonly called man-at-the-end techniques, as they are deployed by parties that have (almost) complete control over the end devices on which they attack and analyze the software, such that they don’t need to rely on exploitable vulnerabilities for their attacks and analyses.

• Context: man-at-the-end attack model, software assets, security requirements, security economics
• Static reverse engineering tools and techniques: interactive disassemblers, decompilers, pattern matching, etc.
• Dynamic reverse engineering tools and techniques: debuggers, hooking, emulation, tracing, statistical analysis, symbolic and concolic execution, taint analysis, delta analyses, fuzzing, etc.
• Software tampering techniques, static and dynamic
• Software obfuscation techniques (layout, design, code, data)
• Preventive software protection techniques: anti-debugging, anti-tampering, anti-emulation
• Software protection evaluation and validation methodologies, incl. the design, execution, and analysis of empirical experiments with human subjects
• Software asset risk management approaches, decision support for software protection, impact on software development life cycle
• Physical attacks based on side channels and fault injection
• Malware analysis, detection, and classification techniques
• Domain-specific attacks and defense on assets embedded in software: cryptographic assets, machine learning models, and possibly others
• Hardware support for code and data confidentiality and integrit

Final competences

1. Understanding the man-at-the-end attack model, including its relevant constructs, models, and methods.
2. Knowing how to devise appropriate man-at-the-end attack strategies based on knowledge about the targeted assets.
3. Knowledge of, a deeper understanding of, and experience with the tools and techniques commonly deployed in man-at-the-end software attacks.
4. Capacity to execute a range of man-at-the-end attack steps on software (reverse engineering, software tampering) as part of such strategies.
5. Extended knowledge of the techniques used for software protection against man-at-the-end attacks, understanding their limitations and the need to combine and layer multiple techniques to obtain useful protection.
6. Basic experience in using software protection tools for deploying those techniques.
7. Understanding strategies for and the complexities of deploying, evaluating, and validating software protections, including applicable risk management approaches.
8. Knowing how to design, execute, and analyze empirical experiments involving human subjects for expanding the knowledge in software protection and man-at-the-end attack models.
9. Knowing the different types of malware and their core features.
10. Understanding the most used malware detection, classification, and analysis methods, and how malware tries to evade those.
11. Being able to deploy forensic malware analysis techniques and tools.
12. Communicating and presenting domain-specific knowledge in a correct and clear manner, with the appropriate language skills, incl. the use of correct terminology.