This paper shows the correlation between chemical order, lattice strains, and magnetic properties of Heusler Co$_2$FeAl films epitaxially grown on MgO(001). A detailed magnetic characterization is performed using vector-field magnetometery combined with a numerical Stoner-Wohlfarth analysis. We demonstrate the presence of three types of in-plane anisotropies: one biaxial, as expected for the cubic symmetry, and two uniaxial. The three anisotropies show different behavior with the annealing temperature. The biaxial anisotropy shows a monotonic increase. The uniaxial anisotropy that is parallel to the hard biaxial axes (related to chemical homogeneity) decreases, while the anisotropy that is supposed to have a magnetostatic origin remains constant.