Gellable composite materials (GCM) were prepared from a smelting slag of copper and nickel deposits and cement, and activated using gypsum and chemical activators. The effects of material ratio, dosage of chemical activators, and gypsum on the mechanical properties of GCM were studied. Our results showed that the chemical activators of Na2SO4, Na2SiO3, NaOH, and Na2CO3 could improve the compressive strength of the GCM. Considering the market cost and ease operation, the compressive strength of the GCM could be significantly improved with 2% Na2SO4. The experiment results also showed that the compound chemical activator could improve the compressive strength of gelled material. The strength of GCM reaches 41.6 MPa when 2% gypsum and 80% of smelting slags of copper and nickel deposits were used, which met the national standards requirements of GCM. As such, it is expected that a large amount of copper and nickel mining smelting slag could be utilized for the production of cementitious materials.