The effects of a para substituent, as the electron-donating -OCH3 and -OtBu groups and the electron-withdrawing -Br and -F atoms, on azobenzene isomerization have been investigated in a series of imidazolium ionic liquids (BMIM PF6, BMIM BF4, BMIM Tf2N, EMIM Tf2N, BM2IM Tf2N, and HMIM Tf2N). The thermal cis-trans conversion tends to be improved in the presence of the substituent, as pointed out by the first-order rate constants measured at 25 °C. Both the rotation and the inversion mechanisms occur in BMIM Tf2N, EMIM Tf2N, and HMIM Tf2N, as highlighted by typical V-shape Hammett plots, but only rotation takes place in BMIM PF6, BMIM BF4, and BM2IM Tf2N. The possible interactions between the cation and the anion of the solvent and both the isomers of the azobenzene derivatives have been studied by small-wide-angle X-ray scattering (SWAXS). The calculated cis population in the photostationary state and the hardness parameter η of the trans isomer show that azobenzene and F-azobenzene are the less reactive molecules for the trans-cis conversion in all the investigated ionic liquids.