The codebook-based techniques are extensively utilized in analog beamforming and combining to overcome high path-loss in millimeter wave (mmWave) massive multiple-input multiple-output (MIMO) communications. However, to find the best analog precoder and combiner, a complex search based on predefined codebook is required in conventional schemes, which leads to large time cost. For the purpose of reducing complexity, we propose a new integer coded quantified angles-based particle swarm optimization (IC-PSO) beamforming algorithm. We firstly propose a joint search scheme based on integer coded (IC) quantified angles which transforms the search space into the integer field to simplify the search space. To converge to the optimal solution quickly, an improved particle swarm optimization (PSO) algorithm is further proposed. In this way, the best precoder and combiner can be found with lower complexity. Furthermore, we optimize the inertia weight and acceleration coefficients and process the out-of-bounds particles, which can improve the search ability of the PSO. Theoretical analysis indicates that the proposed IC-PSO beamforming has the lower complexity than some existing methods. Simulation results show that the algorithm has a satisfactory achievable rate which can achieve almost 98% performance of the full-search beamforming.