Machine Learning Security

Machine Learning SecurityMLSECCYCydrillCY-MLSEC1.0<ul> <li>Getting familiar with essential cyber security concepts</li><li>Learning about various aspects of machine learning security</li><li>Attacks and defense techniques in adversarial machine learning</li><li>Identify vulnerabilities and their consequences</li><li>Learn the security best practices in Python</li><li>Input validation approaches and principles</li><li>Managing vulnerabilities in third party components</li><li>Understanding how cryptography can support appplication security</li><li>Learning how to use cryptographic APIs correctly in Python</li><li>Understanding security testing methodology and approaches</li><li>Getting familiar with common security testing techniques and tools</li></ul>General machine learning and Python developmentPython developers working on machine learning systems<ul> <li>Cyber security basics</li><li>Machine learning security</li><li>Input validation</li><li>Security features</li><li>Time and state</li><li>Errors</li><li>Using vulnerable components</li><li>Cryptography for developers</li><li>Security testing</li><li>Wrap up</li></ul>DAY 1 Cyber security basics <ul> <li>What is security?</li><li>Threat and risk</li><li>Cyber security threat types</li><li>Consequences of insecure software <ul> <li>Constraints and the market</li><li>The dark side</li></ul></li><li>Categorization of bugs <ul> <li>The Seven Pernicious Kingdoms</li><li>Common Weakness Enumeration (CWE)</li><li>CWE Top 25 Most Dangerous Software Errors</li><li>Vulnerabilities in the environment and dependencies</li></ul></li></ul>Machine learning security <ul> <li>Cyber security in machine learning <ul> <li>ML-specific cyber security considerations</li><li>What makes machine learning a valuable target?</li><li>Possible consequences</li><li>Inadvertent AI failures</li><li>Some real-world abuse examples</li><li>ML threat model <ul> <li>Creating a threat model for machine learning</li><li>Machine learning assets</li><li>Security requirements</li><li>Attack surface</li><li>Attacker model – resources, capabilities, goals</li><li>Confidentiality threats</li><li>Integrity threats (model)</li><li>Integrity threats (data, software)</li><li>Availability threats</li><li>Dealing with AI/ML threats in software security</li><li>Lab – Compromising ML via model editing</li></ul></li><li>Using ML in cybersecurity <ul> <li>Static code analysis and ML</li><li>ML in fuzz testing</li><li>ML in anomaly detection and network security</li><li>Limitations of ML in security</li></ul></li><li>Malicious use of AI and ML <ul> <li>Social engineering attacks and media manipulation</li><li>Vulnerability exploitation</li><li>Malware automation</li><li>Endpoint security evasion</li></ul></li></ul></li><li>Adversarial machine learning <ul> <li>Threats against machine learning</li><li>Attacks against machine learning integrity <ul> <li>Poisoning attacks</li><li>Poisoning attacks against supervised learning</li><li>Poisoning attacks against unsupervised and reinforcement learning</li><li>Lab – ML poisoning attack</li><li>Case study – ML poisoning against Warfarin dosage calculations</li><li>Evasion attacks</li><li>Common white-box evasion attack algorithms</li><li>Common black-box evasion attack algorithms</li><li>Lab – ML evasion attack</li><li>Case study – Classification evasion via 3D printing</li><li>Transferability of poisoning and evasion attacks</li><li>Lab – Transferability of adversarial examples</li></ul></li><li>Some defense techniques against adversarial samples <ul> <li>Adversarial training</li><li>Defensive distillation</li><li>Gradient masking</li><li>Feature squeezing</li><li>Using reformers on adversarial data</li><li>Lab – Adversarial training</li><li>Caveats about the efficacy of current adversarial defenses</li><li>Simple practical defenses</li></ul></li><li>Attacks against machine learning confidentiality <ul> <li>Model extraction attacks</li><li>Defending against model extraction attacks</li><li>Lab – Model extraction</li><li>Model inversion attacks</li><li>Defending against model inversion attacks</li><li>Lab – Model inversion</li></ul></li></ul></li><li>Denial of service <ul> <li>Denial of Service</li><li>Resource exhaustion</li><li>Cash overflow</li><li>Flooding</li><li>Algorithm complexity issues</li><li>Denial of service in ML <ul> <li>Accuracy reduction attacks</li><li>Denial-of-information attacks</li><li>Catastrophic forgetting in neural networks</li><li>Resource exhaustion attacks against ML</li><li>Best practices for protecting availability in ML systems</li></ul></li></ul></li></ul>DAY 2 Input validation <ul> <li>Input validation principles <ul> <li>Blacklists and whitelists</li><li>Data validation techniques</li><li>Lab – Input validation</li><li>What to validate – the attack surface</li><li>Where to validate – defense in depth</li><li>How to validate – validation vs transformations</li><li>Output sanitization</li><li>Encoding challenges</li><li>Lab – Encoding challenges</li><li>Validation with regex</li><li>Regular expression denial of service (ReDoS)</li><li>Lab – Regular expression denial of service (ReDoS)</li><li>Dealing with ReDoS</li></ul></li><li>Injection <ul> <li>Injection principles</li><li>Injection attacks</li><li>SQL injection <ul> <li>SQL injection basics</li><li>Lab – SQL injection</li><li>Attack techniques</li><li>Content-based blind SQL injection</li><li>Time-based blind SQL injection</li></ul></li><li>SQL injection best practices <ul> <li>Input validation</li><li>Parameterized queries</li><li>Additional considerations</li><li>Lab – SQL injection best practices</li><li>Case study – Hacking Fortnite accounts</li><li>SQL injection and ORM</li></ul></li><li>Code injection <ul> <li>Code injection via input()</li><li>OS command injection <ul> <li>Lab – Command injection in Python</li><li>OS command injection best practices</li><li>Avoiding command injection with the right APIs in Python</li><li>Lab – Command injection best practices in Python</li><li>Case study – Shellshock</li><li>Lab – Shellshock</li><li>Case study – Command injection via ping</li><li>Python module hijacking</li><li>Lab – Module hijacking</li></ul></li></ul></li><li>General protection best practices</li></ul></li><li>Integer handling problems <ul> <li>Representing signed numbers</li><li>Integer visualization</li><li>Integers in Python</li><li>Integer overflow</li><li>Integer overflow with ctypes and numpy</li><li>Lab – Integer problems in Python</li><li>Other numeric problems <ul> <li>Division by zero</li><li>Other numeric problems in Python</li><li>Working with floating-point numbers</li></ul></li></ul></li><li>Files and streams <ul> <li>Path traversal</li><li>Path traversal-related examples</li><li>Lab – Path traversal</li><li>Additional challenges in Windows</li><li>Virtual resources</li><li>Path traversal best practices</li><li>Format string issues</li></ul></li><li>Unsafe native code <ul> <li>Native code dependence</li><li>Lab – Unsafe native code</li><li>Best practices for dealing with native code</li></ul></li><li>Input validation in machine learning <ul> <li>Misleading the machine learning mechanism</li><li>Sanitizing data against poisoning and RONI</li><li>Code vulnerabilities causing evasion, misprediction, or misclustering</li><li>Typical ML input formats and their security</li></ul></li></ul>DAY 3 Security features <ul> <li>Authentication <ul> <li>Authentication basics</li><li>Multi-factor authentication</li><li>Authentication weaknesses – spoofing</li><li>Case study – PayPal 2FA bypass</li><li>Password management <ul> <li>Inbound password management <ul> <li>Storing account passwords</li><li>Password in transit</li><li>Lab – Is just hashing passwords enough?</li><li>Dictionary attacks and brute forcing</li><li>Salting</li><li>Adaptive hash functions for password storage</li><li>Password policy <ul> <li>NIST authenticator requirements for memorized secrets</li><li>Password length</li><li>Password hardening</li><li>Using passphrases</li><li>Password change</li><li>Forgotten passwords</li><li>Lab – Password reset weakness</li></ul></li><li>Case study – The Ashley Madison data breach <ul> <li>The dictionary attack</li><li>The ultimate crack</li><li>Exploitation and the lessons learned</li></ul></li><li>Password database migration</li></ul></li><li>Outbound password management <ul> <li>Hard coded passwords</li><li>Best practices</li><li>Lab – Hardcoded password</li><li>Protecting sensitive information in memory <ul> <li>Challenges in protecting memory</li></ul></li></ul></li></ul></li></ul></li><li>Information exposure <ul> <li>Exposure through extracted data and aggregation</li><li>Case study – Strava data exposure</li><li>Privacy violation <ul> <li>Privacy essentials</li><li>Related standards, regulations and laws in brief</li><li>Privacy violation and best practices</li><li>Privacy in machine learning <ul> <li>Privacy challenges in classification algorithms</li><li>Machine unlearning and its challenges</li></ul></li></ul></li><li>System information leakage <ul> <li>Leaking system information</li></ul></li><li>Information exposure best practices</li></ul></li></ul>Time and state <ul> <li>Race conditions <ul> <li>File race condition <ul> <li>Time of check to time of usage – TOCTTOU</li><li>Insecure temporary file</li></ul></li><li>Avoiding race conditions in Python <ul> <li>Thread safety and the Global Interpreter Lock (GIL)</li><li>Case study: TOCTTOU in Calamares</li></ul></li></ul></li><li>Mutual exclusion and locking <ul> <li>Deadlocks</li></ul></li><li>Synchronization and thread safety</li></ul>Errors <ul> <li>Error and exception handling principles</li><li>Error handling <ul> <li>Returning a misleading status code</li><li>Information exposure through error reporting</li></ul></li><li>Exception handling <ul> <li>In the except,catch block. And now what?</li><li>Empty catch block</li><li>The danger of assert statements</li><li>Lab – Exception handling mess</li></ul></li></ul>Using vulnerable components <ul> <li>Assessing the environment</li><li>Hardening</li><li>Malicious packages in Python</li><li>Vulnerability management <ul> <li>Patch management</li><li>Bug bounty programs</li><li>Vulnerability databases</li><li>Vulnerability rating – CVSS</li><li>DevOps, the build process and CI / CD</li><li>Dependency checking in Python</li><li>Lab – Detecting vulnerable components</li></ul></li><li>ML supply chain risks <ul> <li>Common ML system architectures</li><li>ML system architecture and the attack surface</li><li>Case study – BadNets</li><li>Protecting data in transit – transport layer security</li><li>Protecting data in use – homomorphic encryption</li><li>Protecting data in use – differential privacy</li><li>Protecting data in use – multi-party computation</li></ul></li><li>ML frameworks and security <ul> <li>General security concerns about ML platforms</li><li>TensorFlow security issues and vulnerabilities</li><li>Case study – TensorFlow vulnerability in parsing BMP files (CVE-2018-21233)</li></ul></li></ul>DAY 4 Cryptography for developers <ul> <li>Cryptography basics</li><li>Cryptography in Python</li><li>Elementary algorithms <ul> <li>Random number generation <ul> <li>Pseudo random number generators (PRNGs)</li><li>Cryptographically strong PRNGs</li><li>Seeding</li><li>Using virtual random streams</li><li>Weak and strong PRNGs in Python</li><li>Using random numbers in Python</li><li>Case study – Equifax credit account freeze</li><li>True random number generators (TRNG)</li><li>Assessing PRNG strength</li><li>Lab – Using random numbers in Python</li></ul></li><li>Hashing <ul> <li>Hashing basics</li><li>Common hashing mistakes</li><li>Hashing in Python</li><li>Lab – Hashing in Python</li></ul></li></ul></li><li>Confidentiality protection <ul> <li>Symmetric encryption <ul> <li>Block ciphers</li><li>Modes of operation</li><li>Modes of operation and IV – best practices</li><li>Symmetric encryption in Python</li><li>Lab – Symmetric encryption in Python</li><li>Asymmetric encryption <ul> <li>The RSA algorithm <ul> <li>Using RSA – best practices</li><li>RSA in Python</li></ul></li><li>Elliptic Curve Cryptography <ul> <li>The ECC algorithm</li><li>Using ECC – best practices</li><li>ECC in Python</li></ul></li><li>Combining symmetric and asymmetric algorithms</li></ul></li></ul></li><li>Homomorphic encryption <ul> <li>Basics of homomorphic encryption</li><li>Types of homomorphic encryption</li><li>FHE in machine learning</li></ul></li></ul></li><li>Integrity protection <ul> <li>Message Authentication Code (MAC) <ul> <li>MAC in Python</li><li>Lab – Calculating MAC in Python</li></ul></li><li>Digital signature <ul> <li>Digital signature with RSA</li><li>Digital signature with ECC</li><li>Digital signature in Python</li></ul></li></ul></li><li>Public Key Infrastructure (PKI) <ul> <li>Some further key management challenges</li><li>Certificates <ul> <li>Chain of trust</li><li>Certificate management – best practices</li></ul></li></ul></li></ul>Security testing <ul> <li>Security testing methodology <ul> <li>Security testing – goals and methodologies</li><li>Overview of security testing processes</li><li>Threat modeling <ul> <li>SDL threat modeling</li><li>Mapping STRIDE to DFD</li><li>DFD example</li><li>Attack trees</li><li>Attack tree example</li><li>Misuse cases</li><li>Misuse case examples</li><li>Risk analysis</li></ul></li></ul></li><li>Security testing techniques and tools <ul> <li>Code analysis <ul> <li>Security aspects of code review</li><li>Static Application Security Testing (SAST)</li><li>Lab – Using static analysis tools</li><li>Lab – Finding vulnerabilities via ML</li></ul></li><li>Dynamic analysis <ul> <li>Security testing at runtime</li><li>Penetration testing</li><li>Stress testing</li><li>Dynamic analysis tools <ul> <li>Dynamic Application Security Testing (DAST)</li></ul></li><li>Fuzzing <ul> <li>Fuzzing techniques</li><li>Fuzzing – Observing the process</li><li>ML fuzzing</li></ul></li></ul></li></ul> Wrap up <ul> <li>Secure coding principles <ul> <li>Principles of robust programming by Matt Bishop</li><li>Secure design principles of Saltzer and Schröder</li></ul></li><li>And now what? <ul> <li>Software security sources and further reading</li><li>Python resources</li><li>Machine learning security resources</li></ul></li></ul></li></ul>- Getting familiar with essential cyber security concepts - Learning about various aspects of machine learning security - Attacks and defense techniques in adversarial machine learning - Identify vulnerabilities and their consequences - Learn the security best practices in Python - Input validation approaches and principles - Managing vulnerabilities in third party components - Understanding how cryptography can support appplication security - Learning how to use cryptographic APIs correctly in Python - Understanding security testing methodology and approaches - Getting familiar with common security testing techniques and toolsGeneral machine learning and Python developmentPython developers working on machine learning systems- Cyber security basics - Machine learning security - Input validation - Security features - Time and state - Errors - Using vulnerable components - Cryptography for developers - Security testing - Wrap upDAY 1 Cyber security basics - What is security? - Threat and risk - Cyber security threat types - Consequences of insecure software - Constraints and the market - The dark side - Categorization of bugs - The Seven Pernicious Kingdoms - Common Weakness Enumeration (CWE) - CWE Top 25 Most Dangerous Software Errors - Vulnerabilities in the environment and dependencies Machine learning security - Cyber security in machine learning - ML-specific cyber security considerations - What makes machine learning a valuable target? - Possible consequences - Inadvertent AI failures - Some real-world abuse examples - ML threat model - Creating a threat model for machine learning - Machine learning assets - Security requirements - Attack surface - Attacker model – resources, capabilities, goals - Confidentiality threats - Integrity threats (model) - Integrity threats (data, software) - Availability threats - Dealing with AI/ML threats in software security - Lab – Compromising ML via model editing - Using ML in cybersecurity - Static code analysis and ML - ML in fuzz testing - ML in anomaly detection and network security - Limitations of ML in security - Malicious use of AI and ML - Social engineering attacks and media manipulation - Vulnerability exploitation - Malware automation - Endpoint security evasion - Adversarial machine learning - Threats against machine learning - Attacks against machine learning integrity - Poisoning attacks - Poisoning attacks against supervised learning - Poisoning attacks against unsupervised and reinforcement learning - Lab – ML poisoning attack - Case study – ML poisoning against Warfarin dosage calculations - Evasion attacks - Common white-box evasion attack algorithms - Common black-box evasion attack algorithms - Lab – ML evasion attack - Case study – Classification evasion via 3D printing - Transferability of poisoning and evasion attacks - Lab – Transferability of adversarial examples - Some defense techniques against adversarial samples - Adversarial training - Defensive distillation - Gradient masking - Feature squeezing - Using reformers on adversarial data - Lab – Adversarial training - Caveats about the efficacy of current adversarial defenses - Simple practical defenses - Attacks against machine learning confidentiality - Model extraction attacks - Defending against model extraction attacks - Lab – Model extraction - Model inversion attacks - Defending against model inversion attacks - Lab – Model inversion - Denial of service - Denial of Service - Resource exhaustion - Cash overflow - Flooding - Algorithm complexity issues - Denial of service in ML - Accuracy reduction attacks - Denial-of-information attacks - Catastrophic forgetting in neural networks - Resource exhaustion attacks against ML - Best practices for protecting availability in ML systems DAY 2 Input validation - Input validation principles - Blacklists and whitelists - Data validation techniques - Lab – Input validation - What to validate – the attack surface - Where to validate – defense in depth - How to validate – validation vs transformations - Output sanitization - Encoding challenges - Lab – Encoding challenges - Validation with regex - Regular expression denial of service (ReDoS) - Lab – Regular expression denial of service (ReDoS) - Dealing with ReDoS - Injection - Injection principles - Injection attacks - SQL injection - SQL injection basics - Lab – SQL injection - Attack techniques - Content-based blind SQL injection - Time-based blind SQL injection - SQL injection best practices - Input validation - Parameterized queries - Additional considerations - Lab – SQL injection best practices - Case study – Hacking Fortnite accounts - SQL injection and ORM - Code injection - Code injection via input() - OS command injection - Lab – Command injection in Python - OS command injection best practices - Avoiding command injection with the right APIs in Python - Lab – Command injection best practices in Python - Case study – Shellshock - Lab – Shellshock - Case study – Command injection via ping - Python module hijacking - Lab – Module hijacking - General protection best practices - Integer handling problems - Representing signed numbers - Integer visualization - Integers in Python - Integer overflow - Integer overflow with ctypes and numpy - Lab – Integer problems in Python - Other numeric problems - Division by zero - Other numeric problems in Python - Working with floating-point numbers - Files and streams - Path traversal - Path traversal-related examples - Lab – Path traversal - Additional challenges in Windows - Virtual resources - Path traversal best practices - Format string issues - Unsafe native code - Native code dependence - Lab – Unsafe native code - Best practices for dealing with native code - Input validation in machine learning - Misleading the machine learning mechanism - Sanitizing data against poisoning and RONI - Code vulnerabilities causing evasion, misprediction, or misclustering - Typical ML input formats and their security DAY 3 Security features - Authentication - Authentication basics - Multi-factor authentication - Authentication weaknesses – spoofing - Case study – PayPal 2FA bypass - Password management - Inbound password management - Storing account passwords - Password in transit - Lab – Is just hashing passwords enough? - Dictionary attacks and brute forcing - Salting - Adaptive hash functions for password storage - Password policy - NIST authenticator requirements for memorized secrets - Password length - Password hardening - Using passphrases - Password change - Forgotten passwords - Lab – Password reset weakness - Case study – The Ashley Madison data breach - The dictionary attack - The ultimate crack - Exploitation and the lessons learned - Password database migration - Outbound password management - Hard coded passwords - Best practices - Lab – Hardcoded password - Protecting sensitive information in memory - Challenges in protecting memory - Information exposure - Exposure through extracted data and aggregation - Case study – Strava data exposure - Privacy violation - Privacy essentials - Related standards, regulations and laws in brief - Privacy violation and best practices - Privacy in machine learning - Privacy challenges in classification algorithms - Machine unlearning and its challenges - System information leakage - Leaking system information - Information exposure best practices Time and state - Race conditions - File race condition - Time of check to time of usage – TOCTTOU - Insecure temporary file - Avoiding race conditions in Python - Thread safety and the Global Interpreter Lock (GIL) - Case study: TOCTTOU in Calamares - Mutual exclusion and locking - Deadlocks - Synchronization and thread safety Errors - Error and exception handling principles - Error handling - Returning a misleading status code - Information exposure through error reporting - Exception handling - In the except,catch block. And now what? - Empty catch block - The danger of assert statements - Lab – Exception handling mess Using vulnerable components - Assessing the environment - Hardening - Malicious packages in Python - Vulnerability management - Patch management - Bug bounty programs - Vulnerability databases - Vulnerability rating – CVSS - DevOps, the build process and CI / CD - Dependency checking in Python - Lab – Detecting vulnerable components - ML supply chain risks - Common ML system architectures - ML system architecture and the attack surface - Case study – BadNets - Protecting data in transit – transport layer security - Protecting data in use – homomorphic encryption - Protecting data in use – differential privacy - Protecting data in use – multi-party computation - ML frameworks and security - General security concerns about ML platforms - TensorFlow security issues and vulnerabilities - Case study – TensorFlow vulnerability in parsing BMP files (CVE-2018-21233) DAY 4 Cryptography for developers - Cryptography basics - Cryptography in Python - Elementary algorithms - Random number generation - Pseudo random number generators (PRNGs) - Cryptographically strong PRNGs - Seeding - Using virtual random streams - Weak and strong PRNGs in Python - Using random numbers in Python - Case study – Equifax credit account freeze - True random number generators (TRNG) - Assessing PRNG strength - Lab – Using random numbers in Python - Hashing - Hashing basics - Common hashing mistakes - Hashing in Python - Lab – Hashing in Python - Confidentiality protection - Symmetric encryption - Block ciphers - Modes of operation - Modes of operation and IV – best practices - Symmetric encryption in Python - Lab – Symmetric encryption in Python - Asymmetric encryption - The RSA algorithm - Using RSA – best practices - RSA in Python - Elliptic Curve Cryptography - The ECC algorithm - Using ECC – best practices - ECC in Python - Combining symmetric and asymmetric algorithms - Homomorphic encryption - Basics of homomorphic encryption - Types of homomorphic encryption - FHE in machine learning - Integrity protection - Message Authentication Code (MAC) - MAC in Python - Lab – Calculating MAC in Python - Digital signature - Digital signature with RSA - Digital signature with ECC - Digital signature in Python - Public Key Infrastructure (PKI) - Some further key management challenges - Certificates - Chain of trust - Certificate management – best practices Security testing - Security testing methodology - Security testing – goals and methodologies - Overview of security testing processes - Threat modeling - SDL threat modeling - Mapping STRIDE to DFD - DFD example - Attack trees - Attack tree example - Misuse cases - Misuse case examples - Risk analysis - Security testing techniques and tools - Code analysis - Security aspects of code review - Static Application Security Testing (SAST) - Lab – Using static analysis tools - Lab – Finding vulnerabilities via ML - Dynamic analysis - Security testing at runtime - Penetration testing - Stress testing - Dynamic analysis tools - Dynamic Application Security Testing (DAST) - Fuzzing - Fuzzing techniques - Fuzzing – Observing the process - ML fuzzing Wrap up - Secure coding principles - Principles of robust programming by Matt Bishop - Secure design principles of Saltzer and Schröder - And now what? - Software security sources and further reading - Python resources - Machine learning security resources4 jours3000.003000.003000.003000.003000.003000.003000.003000.003000.003000.003000.00