Yes, putting in a router will take care of this problem... a router allows more than one person at a time to use your internet connection without your having to deal with this nonsense. A basic router, the kind typically used for the home, are pretty inexpensive and not usually difficult to set up.

OTOH, if, as you say, it's fairly rare for you to need to allow more than one person to connect at a time, you could also just do the thing where you change from one computer to the other one and cycle the cable modem... that's annoying, but it works fine, and it will save you the expense and possible hassle of buying and installing a piece of equipment you'll have little need for. Up to you, obviously.