The models lego has used, not just for castle, but all genres have always been gross approximations or representational. This is why Castle walls are short and thinner than those in my condo. This is also why town's cars seat 1 minifig abreast.
But if you're going for a "realistic" castle just use real castle dimensions. Or go for something inbetween. You don't need to spend a fortune on bricks and be a slave to prototype by making massive walls. Go for something between, like 4 studs.
In the end, all that matters is that you think it looks good, and that it's difficult for pets or small children to destroy.
