You didn't include them for consideration. That said, Worthy was a really, really good player, but he wasn't a great one. He was a very, very good one who complimented great players, which Magic and Kareem were.
The Lakers had a talent for finding excellent complimentary pieces, like Robert Horry with the Kobe Lakers, but if Worthy isn't with the Lakers he's not a HOF. His numbers won't put him there without the championship glow and a few great performances on that stage. In a twelve year career he managed better than 20 pts in a season average four times, as you note, while seemingly confused about the role of a point guard, like Magic. I'll get to the numbers directly.
Magic's first two rings came without Worthy even on the team. Heck, Wilkes was arguably as talented as Worthy.
Magic avg: 19.5 pts shooting .533%, 7.2 rbs, 11.2 ast, and about 2 steals a game.
So Magic scored, on average, right at twenty for his career while being integral in another twenty two or so points a game as a facilitator while collecting more rbs, as a point guard, than Worthy.
lain:
Kareem: 24.6 pts, shooting .559%, 11.2 rbs, 3.6 ast, and about 1 stl a game, nearly 3 blks a game.
Linchpin. Great numbers over a long career. Backbone of the franchise for a long, long time.
Worthy: 17.6 pts, shooting .532%, 5.1 rbs, 3 ast, 1 stl. Just not on equal footing with the big two.
Wilkes: 18.4 pts, shooting .516%, 5.4 rbs, 2.6 ast, 1.2 stl. in his seven years as a Laker.
He doesn't deserve to be put on the same level as Magic or Kareem. Neither does Worthy. He's a lot like Horry.