The authors consider polystable Higgs bundles twisted by a holomorphic line bundle \(L\) over a compact Kähler manifold \(X\), with vanishing first and second Chern classes of the bundle. The case \(\deg L=0\) and \(L^n \cong\) trivial bundle for some \(n\in\mathbb{N}\) is treated. Then \(L\) corresponds to a unitary character \(\pi_1(X)\to U(1)\) of the fundamental group of \(X\). This is used to identify the corresponding Tannaka group in terms of the pro-reductive completion of \(\pi_1(X)\).