ESA

Credits: 5 EC

Prerequisites: Security and Cryptography (Crp). Students need to have previous experience working with Python and Java.

Motivation: Dan Geer once famously said: “Any security technology whose effectiveness can’t be empirically determined is indistinguishable from blind luck.” Or indeed, we may add, from placebo. The foundation of sound engineering is a deep understanding of the problem space, the technological state of the art, and the human element in both. In the past two decades, the security community has learned many important and sometimes painful lessons about what it means to design secure systems. Very frequently, these lessons were the result of large-scale study of a particular technology involving empirical methods. On too many occasions, the results that the analysts obtained proved previous assumptions wrong, and sometimes dangerously so. Successful security engineers need to know how they can approach and solve a new security problem by using empirical methods that will yield reliable results. This course will teach students the path from understanding a problem via measurement and analysis to deriving a successful design that achieves the appropriate level of security.

Synopsis: This course will present a selection of the most important lessons that the security community learned through the application of empirical methods of measurement and analysis. We introduce results from recent research and from case studies of practice to bring students the skills to assess and improve the security of deployed systems. A particular focus is on data-driven approaches to collect operational data about a system's security. We explore deployment issues at local and global scale and also take human factors explicitly into account. Examples are network security, Web security, mobile security and privacy, and the application of machine learning in security. As a result, students will learn to put building blocks of security together in a sound way, to arrive at engineering solutions that are empirically verifiable, functional, and secure against realistic threats.

Learning outcomes: After completion of the course, the students will be able to:

• Understand and explain relevant methods (measurement and analysis) to determine the security of an Internet or mobile technology empirically and at scale
• Understand recent literature on empirical security analysis and apply the findings in the context of similar systems and technologies
• Analyze the security and principal building blocks of an Internet technology using appropriate tools for the task
• Compare, evaluate, and apply the principles that the security community learned, and continues to learn, in empirical studies to the design and engineering process of a system
• Create and design a measurement framework to empirically determine the deployment and use of a security system or technology on the Internet

Lecturers: Dr Ralph Holz (UT) and Dr Andrea Continella (UT)

Examination: Closed book written exam, 50%. Two assignments, each 25%.

Contents:

• Introduction to measurement of security: techniques for data collection and analysis methods
• Each of the following topics is covered in four steps: (1) introduction to the problem, (2) techniques and examples of real-world empirical analysis, (3) key results, (4) lessons learned for future engineering.
  • Real-world cryptography: when “secure” algorithms break in practice and the problems of the deployment chain
  • Usability in security: the importance of user-focused design
  • Network security: TLS and X.509 as an example of an Internet-scale problem
  • Transparency mechanisms: Certificate Transparency and the applications of transparency for public auditing at Internet scale
  • Machine Learning in security: the promise and the pitfalls
  • Web security
  • Security in the mobile ecosystem: the Android use case
  • Privacy in the mobile ecosystem: identifying and analyzing data flows and data leaks

Core study material: 

• Dossier of academic publications and industry whitepapers (required): will be provided during the course
• Book chapters on security and usability (required): Chapters 1-3 in Security and Usability, L. F. Cranor, S. Garfinkel, O’Reilly Media, 2005. ISBN 978-0596008277
• Chapters on Android (reverse) engineering (required): Chapters 1-5 in Android hacker’s handbook, J. J. Drake, Z. Lanier, C. Mulliner, P. O. Fora, S. A. Ridley, G. Wichersky, Wiley, 2014. ISBN 978-1118608647
• Book on security engineering (recommended): Security Engineering, R. Anderson, 3^rd ed, Wiley, 2020. ISBN 978-1119642787
• Collection of engineering lessons (recommended): Engineering Security, P. Gutmann, 2014. Available free online, URL on demand