Try using theadding a wheel group rather than using sudo. sudo allows a user to execute as though they are root which means it isn't any different than running:
su -
su -
to become root. The root uid=0; the wheel uid=10. People use the names but the OS uses the uid. Certain commands cannot be run by a member of the wheel group. (If you use wheel don't make the mistake of adding root as a wheel group member). Take a look at Red Hat documentation if you don't know what wheel is.
Another option is to use an ssh key rather than a password. Assuming you can be sure the folks using sudo don't add their public keys to the ~/.ssh/authorizedkeys file this gets around any concern of the root password changing because root password is effectively ignored.
If you'd rather get around extending trust to these users (to not add their own public key) try using extended file attributes and using the Immutable bit. After creating your ~/.ssh/authorizedkeys file, and after configuring /etc/ssh/sshd_config and /etc/ssh/ssh_config as you like, set the Immutable bit. Sure, there're ways to screw with that too -- nothing is perfect.
In any system, insiders are the highest risk. The person who runs the show is at the top of the pile. A related thought pertains to contracts. Lawyers will tell you, "Never do business with someone you need a contract to do business with". Common sense tells us, "Never do business without a contract". When you find someone you trust enough to do business with, write the contract based on each other's needs.
-JohnC All the submitted answers, MPHincluding mine, BSEEfail to address physical access. Whomever has it, CISSPhas control. If it isn't you then there's likely a way for you to get the person who does to physically access the machine in the event someone finds a way to prevent you from logging in, or if they find a way to log in even after you've attempted to prevent that from happening. Of course, if you've got physical access than there may not be a need for any of these things although implementing a couple should be considered anyway, if you're interested in NOT being inconvenienced.
-John Crout