As players level up in an RPG, they expect increased challenge. They want bigger monsters that are harder to kill and that threaten to kill them first, faster.
The formula seems like it would be simple: As player characters level-up and gain hit points, you make the monsters deal more damage. In other words:
You may see the problem already. The player characters have ostensibly increased their power by leveling up, so they're ready to face greater threats. But how is a level 17 character more powerful than a level 1 character if both can be killed in the same number of hits? Flavour goes a long way, but when the dynamics of combat doesn't change, people eventually start to figure out that this "bigger" monster is actually the same level 1 monster but with bigger numbers.
What if you change the monster's damage from 50 to, say, 25? Now it would take the monster 5 hits to kill the level 17 character. But wait, that means a level 1 monster is actually more efficient than the level 17 monster. The scarier monster has to hit twice as much to kill a character, so actually the level 1 monster seems like it was the greater threat.
It's counterintuitive, but in game design levels only go so high. Not literally, of course numbers are indeed infinite, but there's a "ceiling," after which all high numbers are essentially the same.
Leveling up is more than handing out more numbers to players and monster stats. In fact, that's why the Challenge Rating (CR) system of D&D is so imprecise. If it were pure numbers, it would be relatively easy to design, as long as you allow for the randomness of dice.
There are a lot more to monsters than just damage, but I wonder whether it really comes across to players.
D&D has a tier structure, with player levels being divided into 4 tiers of 5. I think the key to creating a believable threat to resilience ratio isn't about the kind of damage a monster does, but instead everything else a montster can do. And I think monsters ought to be designed in a way that monster abilities are regimented, and increase according to player tier levels.
As a caveat: I haven't playtested or even designed any of this. These are just thoughts, half reverse-engineering what D&D already does, and half searching for structure behind how I plan adventures for my group.
Here's my idea for monster design over the course of a player character's career:
As players progress between tiers, they'll start to notice that getting hit by a monster is no longer the only thing they need to fear.
I like deadly games, and I think D&D would do well to get hit points under control. Hit points increase as you level up. In earlier editions, there actually was a limit to how much specific races would continue to increase in levels (and therefore hit points} but that's no longer the case. And it doesn't matter, because I think hit points are a false indicator of power. I don't think players care about hit points, especially after they've got over 5 or 10 hit die to spend on recovery. Players care about abilities and extra attacks and loot. Hit points grow back, so it's silly to put an emphasis on them.
I think hit points ought to be limited to 20. You start with the number on your initial hit die (d4 for wizards, d10 for fighters, d8 for monks, and so on) and you get 1 hit point with each level, until you have 20 hit points. That's it.
Here's how monsters get adjusted to account for that:
A GM can have monsters roll for damage, or just deal an average amount, but by the time a player is in tier 4 play, a monster could potentially kill with one hit. Add to that the special attack ability that a monster in that tier has, and monsters are a serious threat.
Then again, so are the players. But at high level play, combat isn't usually about standing around waiting for your opponent to hit you with a sword. By tiers 3 and 4, players have gained feats and abilities that can banish, freeze, explode, and melt monsters. Sometimes they're going to encounter a big creature and end the combat in 2 turns thanks to a lucky hit and a lucky spell.
Other times, though, they're going to encounter a creature that succeeds its saving throw, gets a good hit in, rolls serious damage, and potentially kills a PC.
But also empowering. And that's what we're here for. High highs and deadly lows.
I don't have plans to actually implement this design idea, and I know I'm not the only person to have thought of these tricks. I'm posting it here to publically express, I guess, my vague hope that a D&D version in the future goes in this direction. There's almost certainly an OSR game out there that already does.
Anyway, they're concepts to think about, and maybe keep in mind, and possibly adapt in some small way, as you prep for your next campaign.