In order of appearance:
1978 - 375 Castle: This held the title for a short while, partially because it was the ONLY castle set at the time! But I also liked it less than the other castle sets of the day because the yellow scheme for the building always seemed silly to me. Not terrible, but kinda odd.
1987 - 6016 Knight's Aresenal: This one took the title away from 375 because you only got 1 figure, and because I never understood the rubber tubing as a kid. I understand today what they were going for, but it always looked to me like some sort of space-age cross-theme bizarrity.
1993 - 6043 Dragon Defender: Ok, Dragon Masters. "Hated it!" I recall looking at Dragon Masters back in the 1996 timeframe and thinking they were the lamest faction ever. Their castle was a contender for the "worst set" except it did have some good qualities. The Dragon Defender, however, did not. The catapult mechanism didn't work very well, forced you to try and shoot both at once, it was brightly colored, and didn't do a good job resembling an evil dragon head. If you showed up to battle with that monstrosity, you'd win only because your opponents would be laughing too hard to fight back.
1997 - 6037 Witch's Windship: Similar to Dragon Masters, Fright Knights took the cake at the time for "worst subtheme" until KK2. Poor sets abounded, the worst of which I think were Witch's Windship and the Fright Knights Flying Machine. But Witch's Windship usurped the "worst set" crown because it's just too wacky of an idea for me. On the basis of part selection it might be better, but not by enough to offset the all-too-strange idea of Willa needing a big basket to fly in, that a dragon would bother carrying it, and that it's basically trying to rip off the idea of an 1800's hot-air-balloon.
2004 - 8779 The Grand Tournament: And finally, the current record holder in my book. There are a few other poorer sets like the large-scale constructable figures, or possibly some DUPLO or Belville castles-- but I'm not really counting them. The Grand Tournament set a new low for strangeness. KK2 was weird enough with its futuristic concepts like "magical armor that prevents knights from getting hurt", but it also had a color scheme for figures that was atrocious for all but the youngest of kids. Further, there were no soldiers to speak of, except for the Shadow Knights (who were the only real redeeming factor of the lineup). To top it off, they debuted in 2004 with the color change, which forever puts a black mark on them in my memory.
This set in particular was nothing more than a play activity for kids rather than anything even closely resembling something medieval. Sure, the contestants rode "horses" (if you want to call them that), but they had enormous wheels, and were clearly intended to smash into each other rather than knock each other off their mounts. Further, the set even deviated from the plot line of the story, since as I recall, they were expected to fight with swords, and it came down to Jayko using the uber-shield in order to win, which wasn't even in the set. Also, although my memory is hazy on the subject, didn't Jayko have to defeat Vladek before the King was freed from custody? Anyway, the lack of useful pieces, useful figures, and purely activity-based rather than building or figure-playing based theme to the set wins it the "Worst Set Ever" award from me.