The use of biochars (BCs) and activated carbons as a way of sequestering soil-bound pollutants such as chlordecone (CLD) is increasingly being studied. This study aims at assessing the impact of Sargassum BC/AC particle size and Sargassum BC amendment rate on CLD adsorption in Nitisol and in Andosol. Four different types of carbonaceous matrices were tested: Sargasso carbon activated by phosphoric acid (SargH3PO4), Sargasso carbon activated by steam (SargH2O), biochar of Sargasso (Ch Sarg700), and a commercial activated carbon (ORBO™). In a first experiment, CLD contaminated Andosol and Nitisol were amended with 2% of each carbonaceous matrix divided into four particles size classes (< 50 µm, 50–150 µm, 150–200 µm, and > 200 µm). In a second experiment, the contaminated soils were amended with the biochar of Sargasso at five application rates (0, 0.25, 0.5, 1, and 2% (w/w)). After a 4-month aging, environmental availability tests were carried out on the soils of both experiments. The results of the first experiment showed that the best reductions of CLD environmental availability were obtained in both soils with the biochar of Sargasso and the ORBO™. More specifically, in nitisol, particle size under 50 µm of biochar of Sargasso and AC ORBO™ showed a CLD environmental availability reduction up to 72 ± 2.6% and 79 ± 2.6%. In Andosol, there was no significant difference between the three particle sizes (< 50 µm, 50–150 µm, and 150–200 µm) of the biochar of Sargasso on the reduction of environmental availability (average reduction of 43 ± 2.5%). The results of the second experiment showed that an amendment rate increase improves the immobilization of CLD. When the amendment rate was increased from 0.25 to 2%, the environmental availability was reduced by 43% in Nitisol and 50% in Andosol.