Cyber attackers constantly craft new attacks previously unknown to the security community. There are two approaches for detecting such attacks: (1) employing human analysts who can observe the data and identify anomalies that correspond to malicious intent; and (2) utilizing unsupervised automated techniques, such as clustering, that do not rely on ground truth. We conduct a security analysis of the two approaches, utilizing attacks against a real-world website. Through two experiments—a user study with 65 security analysts and an experimental analysis of attack discovery using DBSCAN clustering—we compare the strategies and features employed by human analysts and clustering system for detecting attacks. Building on these observations, we propose threat models for the human analysis process and for the unsupervised techniques when operating in adversarial settings. Based on our analysis, we propose and evaluate two attacks against the DBSCAN clustering algorithm and a defense. Finally, we discuss the implications of our insights for hybrid systems that utilize the strengths of automation and of human analysis to complement their respective weaknesses.