We achieve characterizations of those ultradistribution \(\mu\in{\mathcal E}_{(\omega)}'(\mathbb{R}^n)\) (resp. \({\mathcal E}_{\{\omega\}}'(\mathbb{R}^n)\)) with compact support such that for each ultradifferentiable function \(f\) in \({\mathcal E}_{(\omega)}(\mathbb{R}^n)\) (resp. in \({\mathcal E}_{\{\omega\}}(\mathbb{R}^n)\)) each solution \(\nu\in{\mathcal D}_{(\omega)}'(\mathbb{R}^n)\) (resp. in \({\mathcal D}_{\{\omega\}}'(\mathbb{R}^n)\)) of the convolution equation \(\mu*\nu= f\) belongs to the same class as the function \(f\). The characterizations are in terms of an appropriate slowly decreasing condition and the location of the zeros in \(\mathbb{C}^n\) of the Fourier-Laplace transform of \(\mu\) or in terms of the behaviour of the singular support of an elementary solution of the convolution equation. These characterizations extend classical results of Ehrenpreis and Hörmander for distributions and of Björck and Chou for ultradistributions.