This sounds fun! Someday when I have a dedicated Lego room, one of my big goals is to start a modular semi-permanent layout that I can continue to add to over time. It would have a big castle, town, outlaying farms and quarries, etc. Somewhere in the neighborhood of 10 x 4 64 stud plates in size.
That's all a long-ways off though
I hear ya
My approach right now is that I want to gather up the necessary figs, build them standalone buildings/shops and also finish building my keep one of these days. Now while I don't have the room to display all of it at once, I could work on getting them modularized so that I could put them together at some point.
So far I have
lord, lady, daughter, (maybe) son, stewart, 1 female servant, 1 male servant, captain of the guard, household guard (all 13 of them), soldiers (all 15 of them), blacksmith, apothecary/healer, peasant.
Now this might sounds like a short list and one that could be easily filled, but it's harder to do it with fleshies as quite a few of female torsos that I have are not usable.
Also I already see a problem: my guards and knights total 40 and I think my keep that is about 54 x 40 at its base is not going to be big enough to house them :/