Several metal-organic framework compounds (MOF-5, MIL-68(Ga), MIL-68(In), MIL-53(Al)) were loaded with azobenzene (AZB), as confirmed by XRPD measurements and elemental analysis. By IR spectroscopy, it was shown that the light-induced trans/cis isomerization of AZB in these hybrid host-guest compounds is improved compared to that of solid AZB. A population of the excited cis state up to 30% has been obtained for AZB0.66@MIL-68(In). However, no light-induced trans/cis isomerization was observed for AZB0.5@MIL-53(Al). Structural models obtained from high-resolution synchrotron powder diffraction data show that AZB molecules are densely packed within the channels of MIL-53(Al) so that no trans/cis isomerization can occur. A different situation was observed for AZB in the larger channels of MIL-68(Ga). Thus, this investigation shows the influence of the host material on the switching behavior of the embedded AZB molecules.