Unmanned Aerial Vehicles (UAVs) have been playing an increasingly active role in supporting search and rescue (SAR) operations in recent years. The benefits are multiple such as enhanced situational awareness, status assessment, or mapping of the operational area through aerial imagery. Most of these application scenarios require the UAVs to cover a certain area. If the objective is to detect people or other objects, or analyze in detail the area, then there is a trade-off between speed (higher altitude coverage) and perception accuracy (lower altitude). An optimal point in between requires active perception on-board the UAV to dynamically adjust the flight altitude and path planning. As an initial step towards active vision in UAV search in maritime SAR scenarios, in this paper we focus on analyzing how the flight altitude affects the accuracy of object detection algorithms. In particular, we quantify what are the probabilities for false negatives and false positives in human detection at different altitudes. Our results define the correlation between the altitude and the ability of UAVs to effectively detect people in the water.