I've struggled with this my entire guitar-playing life.
I've never been able to kick this habit (probably because I also struggle with little things like "practicing regularly" and "doing drills and exercises instead of just playing songs"), but a few things I've come across that should help if you stick with them:
1) Play scales and various simple exercises to a metronome, VERY, VERY
slowly. Focus on keeping the pinky in position, and gradually up the tempo, stopping and slowing back down any time the pinky starts straying out again. This is also a good way to practice keeping your left hand loose and not playing with more pressure than necessary.
2) An exercise my guitar teacher gave me when I first started playing: Play any four consecutive semi-tones, one finger per note, four notes per string. So like 5-6-7-8 on the E, then the same on the A, etc across all six (or seven, eight, or god forbid, 9) strings. When you get to the top string, do the reverse and play 8-7-6-5 going back down.
Here's the trick though - take it slow, play to a metronome, and concentrate on not moving any of your fingers until you need them to play the next note. In other words, you play the 5th fret on the E with your index finger, and you keep that there even after you put the middle finger down on the 6th fret. Then you keep both in position when you fret the 7th with your ring finger, etc. Same thing when you move between strings. Keep your middle, ring and little fingers down on their respective frets on the low E when you move your index to the 5th fret on the A string.
If you want, you can start on the 1st fret of the low E and then shift up a half step every time you get to the top or bottom string, and just play it that way up and down the neck until you and anyone else in ear shot goes absolutely fucking mental from listening to slow-ass chromatic scales.
Edit: Son of a bitch! Yeah jaxadam's explanation is way clearer and more concise