Red-bed soft rock is highly susceptible to softening, swelling, and disintegration upon water exposure and exhibits pronounced rheological behavior, which often leads to structural deformation and failure in engineering constructions. The rock has attracted widespread attention in geotechnical research. The mechanical properties of rocks are significantly influenced by sample size and geometry, yet existing size effect models are typically tailored to specific rock types. Therefore, establishing a unified size effect model and understanding the influence of size on the mechanical behavior of red-bed soft rocks are of considerable significance.

A series of unconfined uniaxial compression tests was conducted on red-bed soft rock specimens with varying height-to-diameter ratios (H/d) to investigate their peak stress (σ_P), peak axial strain (ε_P), and average elastic modulus (E_av). The dispersion of mechanical parameters under different H/d ratios was analyzed using a dispersion parameter defined as the mean coefficient of variation across these key mechanical properties. Subsequently, five machine learning algorithms, decision tree regression (DTR), support vector regression (SVR), multilayer perceptron (MLP), random forest regression (RFR), and extreme gradient boosting regression (XGR), were employed to model the size effect of red-bed soft rocks. The original dataset was expanded through linear interpolation, and the influence of Gaussian noise of varying intensities was investigated. The hyperparameters of DTR, SVR, RFR, and XGR were determined through grid search, and the network structure of MLP was determined through manual tuning, while model robustness was evaluated using 10-fold cross-validation. Performance metrics on both training and test datasets were reported. In addition, a global interpretability analysis of the optimal XGR model was conducted using Shapley additive explanations (SHAP) to rank feature importance and perform local interpretations on three representative actual test samples (H/d = 0.4, 1.0, and 2.0). To assess model generalizability, the trained models were used to predict the uniaxial compressive strength of rock types with non-standard sizes.

As the H/d ratio decreased, both uniaxial compressive strength and peak axial strain increased. The uniaxial compressive strength of the 0.4 H/d ratio group specimen was 1.6–2.1 times that of the 2.0 H/d ratio group specimen, and the peak axial strain was 2.3–3.3 times that of the latter. The stress–strain curves exhibited a transition from brittle to ductile failure modes. The failure mode of the specimen gradually changed from shear failure to splitting failure, and then to complex failure modes. Existing empirical models failed to accurately fit the size effect on uniaxial compressive strength of red-bed soft rock from the Huma Ridge area, with R² values below 0.6. Among the mechanical parameters, the dispersion order from highest to lowest was E_av, σ_P, and ε_P. The dispersion parameter of mechanical characteristics increased and then decreased with H/d ratio, peaking between H/d = 1.2–1.4. Among the five machine learning algorithms, DTR was the most sensitive to noise and exhibited the lowest stability and predictive performance (minimum R² = 0.175). In contrast, XGR, RFR, MLP, and SVR effectively captured the complex nonlinear relationships between input features and σ_P, with test R² values of 0.989, 0.972, 0.967, and 0.965, respectively. SHAP-based analysis of the XGR model revealed that H/d was the most influential feature, followed by E_av and ε_P. The average absolute SHAP values are 3.98, 1.09, and 0.56. Higher H/d values had a negative impact on model predictions, while E_av and ε_P showed positive contributions. Local interpretation indicated that as H/d increased from 0.4 to 2.0, its contribution weight declined and shifted from positive to negative. For sample C5-02 (H/d = 1), E_av exhibited a strong negative contribution to the predicted σ_P. When applied to predict σ_P of other rock types, the MLP model demonstrated relatively better generalizability, accurately estimating σ_P for coal rock, lean ore, and marble. However, it tended to overestimate σ_P for gypsum and underestimate it for sandstone. This discrepancy was attributed not only to differences in microstructural characteristics but also to variation in feature contributions. Specifically, SHAP analysis showed that E_av had the highest contribution weight in gypsum, lean ore, and marble predictions, whereas H/d dominated in sandstone predictions, resulting in prediction bias.

Changes in various mechanical properties are obtained as the H/d ratio decreases through uniaxial compression tests, and the discreteness of these indicators is analyzed. Among all models tested, XGR achieves the highest accuracy in predicting σ_P of red-bed soft rocks. In terms of model consistency, the MLP model trained on the red-bed soft rock dataset can predict the σ_P of other non-standard sized rocks with an error of less than 20%. While the MLP model shows promising results in predicting σ_P of other rock types, its accuracy remains limited due to insufficient training diversity. To improve model generalizability and applicability, future work should incorporate additional rock types and features such as rock density and fracture parameters. This result provides a preliminary exploration and reference for the construction of more universal rock strength prediction models in the future.