Table 3 Diagnostic performance of six different machine learning models for ALK mutation status in the training and validation sets 1–3

LR

Training set

0.872

(0.810–0.920)

82.86 (29/35)

(70.37–95.34)

80.99 (98/121)

(74.00-87.98)

81.41 (127/156)

(75.31–87.52)

55.77 (29/52)

(42.27–69.27)

94.23 (98/104)

(89.75–98.71)

Validation set 1

0.779

(0.681–0.858)

78.26 (18/23)

(61.40-95.21)

68.57 (48/70)

(57.70-79.45)

70.97 (66/93)

(61.74–80.19)

45.00 (18/40)

(29.58–60.42)

90.57 (48/53)

(82.70-98.44)

Validation set 2

0.803

(0.727-865)

76.00 (19/25)

(59.26–92.74)

75.44 (86/114)

(67.54–83.34)

75.54 (105/139)

(68.39–82.69)

40.43 (19/47)

(26.40-54.46)

93.48 (86/92)

(88.43–98.52)

Validation set 3

0.752

(0.664–0.827)

75.00 (18/24)

(57.68–92.32)

68.82 (64/93)

(59.40-78.23)

70.09 (82/117)

(61.79–78.38)

38.30 (18/47)

(24.40–52.20)

91.43 (64/70)

(84.87–97.99)

RF

Training set

0.933

(0.881–0.967)

77.14 (27/35)

(63.23–91.05)

93.39 (113/121)

(88.96–97.82)

89.74 (140/156)

(84.98–94.50)

77.14 (27/35)

(63.23–91.05)

93.39 (113/121)

(88.96–97.82)

Validation set 1

0.673

(0.568–0.767)

30.44 (7/23)

(11.63–49.24)

94.29 (66/70)

(88.85–99.72)

78.50 (73/93)

(70.14–86.85)

63.64 (7/11)

(35.21–92.06)

80.49 (66/82)

(71.91–89.07)

Validation set 2

0.626

(0.540–0.706)

56.00 (14/25)

(36.54–75.46)

64.91 (74/114)

(56.15–73.67)

63.31 (88/139)

(55.30-71.32)

25.93 (14/54)

(14.24–37.61)

87.06 (74/85)

(79.92–94.19)

Validation set 3

0.672

(0.580–0.756)

62.50 (15/24)

(43.13–81.87)

73.12 (68/93)

(64.11–82.31)

70.94 (83/117)

(62.71–79.17)

37.50 (15/40)

(22.50–52.50)

88.31 (68/77)

(81.14–95.49)

SVM

Training set

0.912

(0.855–0.951)

85.71 (30/35)

(74.12–97.31)

85.12 (103/121)

(78.78–91.46)

85.26 (133/156)

(79.69–90.82)

62.50 (30/48)

(48.80–76.20)

95.37 (103/108)

(48.80–76.20)

Validation set 1

0.822

(0.729–0.894)

86.96 (20/23)

(67.87–95.46)

71.43 (50/70)

(60.85–82.01)

75.27 (70/93)

(66.50-84.04)

50.00 (20/40)

(34.50–65.50)

94.34 (50/53)

(84.63–98.06)

Validation set 2

0.841

(0.769–0.897)

76.00 (19/25)

(59.26–92.74)

84.21 (96/114)

(77.52–90.90)

82.73 (115/139)

(76.45–89.20)

51.35 (19/37)

(35.25–67.46)

94.12 (96/102)

(89.55–98.68)

Validation set 3

0.771

(0.685–0.844)

75.00 (18/24)

(57.68–92.32)

73.12 (68/93)

(64.11–82.31)

73.50 (86/117)

(65.51–81.50)

41.86 (18/43)

(27.11–56.61)

91.89 (68/74)

(85.67–98.11)

KNN

Training set

0.806

(0.735–0.864)

80.00 (28/35)

(66.75–93.25)

71.07 (86/121)

(62.99–79.15)

73.08 (114/156)

(66.12–80.04)

44.44 (28/63)

(32.17–56.71)

92.47 (86/93)

(87.11–97.84)

Validation set 1

0.752

(0.652–0.836)

86.96 (20/23)

(67.87–95.46)

55.71 (39/70)

(53.65–73.23)

63.44 (59/93)

(55.29–71.32)

39.22 (20/51)

(25.82–52.62)

92.86 (39/42)

(80.99–97.54)

Validation set 2

0.675

(0.590–0.752)

76.00 (19/25)

(56.57–88.50)

61.40 (70/114)

(52.47–70.34)

64.03 (89/139)

(56.05–72.01)

30.16 (19/63)

(18.83–41.49)

92.11 (70/76)

(86.04–98.17)

Validation set 3

0.728

(0.638–0.806)

75.00 (18/24)

(57.68–92.32)

60.22 (56/93)

(50.27–70.16)

63.25 (74/117)

(54.51–71.98)

32.73 (18/55)

(20.33–45.13)

90.32 (56/62)

(82.96–97.68)

LDA

Training set

0.746

(0.670–0.812)

80.00 (28/35)

(66.75–93.25)

76.86 (93/121)

(69.35–84.37)

77.56 (121/156)

(71.02–84.11)

50.00 (28/56)

(34.50–65.50)

93.00 (93/100)

(88.00–98.00)

Validation set 1

0.640

(0.534–0.737)

52.17 (12/23)

(32.96–70.76)

77.14 (54/70)

(67.31–86.98)

70.97 (66/93)

(61.74–80.19)

42.86 (12/28)

(26.51–60.93)

83.08 (54/65)

(73.96–92.19)

Validation set 2

0.714

(0.632–0.788)

68.00 (17/25)

(48.41–82.79)

71.93 (82/114)

(63.68–80.18)

71.22 (99/139)

(63.70-78.75)

34.69 (17/49)

(21.37–48.02)

91.11 (82/90)

(85.23–96.99)

Validation set 3

0.655

(0.562–0.740)

70.83 (17/24)

(50.83–85.09)

62.37 (58/93)

(52.52–72.21)

64.10 (75/117)

(55.41–72.80)

32.69 (17/52)

(19.94–45.44)

89.23 (58/65)

(81.69–96.77)

XGBoost

Training set

0.861

(0.797–0.911)

80.00 (28/35)

(66.75–93.25)

75.21 (91/121)

(67.51–82.90)

76.28 (119/156)

(69.61–82.96)

48.28 (28/58)

(35.42–61.14)

92.86 (91/98)

(87.76–97.96)

Validation set 1

0.701

(0.597–0.792)

65.22 (15/23)

(44.89–81.19)

68.57 (48/70)

(57.70-79.45)

67.74 (63/93)

(58.24–77.24)

40.54 (15/37)

(24.72–56.36)

85.71 (48/56)

(76.55–94.88)

Validation set 2

0.746

(0.665–0.816)

72.00 (18/25)

(52.42–85.72)

71.93 (82/114)

(63.68–80.18)

71.94 (100/139)

(64.47–79.41)

36.00 (18/50)

(22.70–49.30)

92.14 (82/89)

(86.54–97.73)

Validation set 3

0.729

(0.639–0.807)

75.00 (18/24)

(57.68–92.32)

64.52 (60/93)

(54.79–74.24)

66.67 (78/117)

(58.12–75.21)

35.29 (18/51)

(22.18–48.41)

90.91 (60/66)

(83.97–97.84)