The quantitative performance indicators of Physical Unclonable Functions (PUFs)-Randomness, Steadiness, Correctness, Diffuseness and Uniqueness-are strictly defined and applied to the evaluation of 45 arbiter PUFs on Virtex-5 FPGAs. The indicators effectively reflect the characteristics of PUFs ranging from 0 to 1 with 1 being the highest performance. The indicators enable the easy measurement and intuitive understanding of PUF performances. The experimental results shows that the arbiter PUFs have excellent overall intra-device performances though a slight bit bias is indicated. The inter-device performance is moderate and will suffice for the practical use of PUFs for device authentication and so on. Additionally, the reliability of the obtained PUF performances is statistically discussed in terms of the Confidence Interval and the number of devices. This paper presents in detail the definitions of the performance indicators and the quantitative and statistical evaluation results of the arbiter PUFs.