Deep Learning-Based Anomaly  Detection in TLS Encrypted Traffic

 The growing trend of encrypted network traffic is changing the cybersecurity threat scene. Most critical infrastructures and organizations enhance service delivery by embracing digital platforms and applications that use encryption to ensure that data and Information are moved across networks in an encrypted form to improve security. While this protects data confidentiality, hackers are also taking advantage of encrypted network traffic to hide malicious software known as malware that will easily bypass the conventional detection mechanisms on the system because the traffic is not transparent for the monitoring mechanism on the system to analyze. Cybercriminals leverage encryption using cryptographic protocols such as SSL/TLS to launch malicious attacks. This hidden threat exists because of the SSL encryption of benign traffic. Hence, there is a need for visibility in encrypted traffic. This research was conducted to detect malware in encrypted network traffic without decryption. The existing solution involves bulk decryption, analysis, and re-encryption. However, this method is prone to privacy issues, is not cost-efficient, and is time-consuming, creating huge overhead on the network. In addition, limited research exists on detecting malware in encrypted traffic without decryption. There is a need to strike a balance between security and privacy by building an intelligent framework that can detect malicious activity in encrypted network traffic without decrypting the traffic prior to inspection. With the payload still encrypted, the study focuses on extracting metadata from flow features to train the machine-learning model. It further deployed this set of features as input to an autoencoder, leveraging the construction error of the autoencoder for anomaly detection.