When should I extrapolate and when should I interpolate?

You interpolate when you know the 'before' and 'after' values.

For example: in a point-and-click game, player is currently at position X, and from his interface, he clicks on spot Y. You must interpolate the displacement between X and Y because you know the two values.

You extrapolate when you guess what's going to be future value, based on what you already know.

For example: in a first-person-shooter game, player is at position X and he's been depressing the UP arrow on his keyboard for the last second, you can suppose that he'll be doing this again for the next second because that's basically the most frequent behaviour in your game; so you extrapolate the position that he'll be next frame. The server sends it to the other game clients, this accounts for the delay in the transmission from the server to the players.

Extrapolation is used mainly for movement prediction. It is not needed by the game server, but the game clients need it to display a somewhat realistic and current vision of the state of the game in order to give a smooth visual experience to the players.

Interpolation is done when you have both a start and end value, and you want to estimate what happens between this start and end value. An example would be to move a player from Position A to Position B in a fluid motion.

Extrapolation, is done when you have a start value, but do not yet have data for the end. You can then extrapolate based on what data you have. For example, based on a player's previous movements, you can determine where he is probably going to be in the next frame.

Always interpolate when you can.

When you don't have enough information to interpolate then you need to extrapolate.

It really is that simple, don't over-think it :)

To explain a bit more:

In general interpolation is better because interpolation is always right. To extrapolate you have to guess. Then you have to deal with what happens when you guess wrong, which leads to rubber banding or popping and all sorts of systems to deal with handling all of that and disguising it.

What happens if you extrapolated a bat position and showed it going to the right place and bouncing the ball, then realize afterwards that you were wrong and it didn't bounce the ball? There is no good way to handle that scenario.

You interpolate to find states between known values, and you extrapolate to find future states.

Think of the problem in terms of state variables, like positions and velocities. In the best of all scenarios, every computer which needs to work with state has access to the state data for the time they want to work on. For example, a collision algorithm to see whether laser-rifle shot X interesets player A's head, the best of all cases is when the algorithm knows the exact position of every object at the time the laser was fired.

In the real world, we are not always so lucky. Sometimes the truth information we receive is more sparse. For example, if player A is a remote player on another computer, you may not know exactly where they're going when you fire the laser and need to calculate the shot. In this case, you need to create an estimator for A's position, typically with interpolation or extrapolation.

The difference between the two is whether you have data that is bounded on both sides, or only one side. Let's say that Player A has already announced their truth position for t=0 and t=1. Player B shoots a laser at t =0.5. In many situations Player A's announcement of their position at t=1 can occur before Player B pulls the trigger. Why? In many games, the responsiveness of the controls is less than perfectly instantaneous. In a racing simulation, much of the player's position is bounded by the physics of a moving vehicle. You may choose to announce a "future position" because you know you really can't steer all that much in a short period. If you have information in the future, you can interpolate between the two values.

What if you aren't lucky enough to have a t=1 value? What if Player A wasn't in a position to announce their future location, and you're stuck deciding whether you hit or missed with only the information from t=0? In this case you have to extrapolate. In extrapolation, you use what you know about the motion to extend beyond any data you have. You might know that Player A has a certain velocity, so presume that if you multiply that by time, you can get a position at each time.

The difference is in the behaviors. Interpolation requires you to have an upper and a lower bound, which you do not always have. However, in nearly all situations it has vastly better results than extrapolation. Extrapolation can easily lead to unrealistic movements. Consider the case of a player who is sidestepping left and right to avoid being shot while advancing. At any given point, their velocity is along a diagonal, so if you extrapolate, the player may appear to run off to the side when, in fact, they never do. If you only do interpolation, the values tend not to stray outside realistic values.

Interpolation and extrapolation are two extremes in the world of filtering. There are many many many many many filters out there for handling data like this which mix and match properties between interpolation and extrapolation. Accordingly, don't be surprised if you see algorithms that are not clearly interpolation or clearly extrapolation. Those two are just the tip of the ice berg.