There are several things you can do.
You can centralize all the physics objects on the server and synch
coordinates to the players objects on all clients. This is the
easiest and works without many flaws, however it uses a lot of
resources and requires lots of bandwidth. You can optimize bandwidth
usage by only sending values to the player of other players that are
within a certain radius.
You can do as Neenster mentioned and have the server and clients
simulate physics, every so often the server will correct the
clients. This means all clients calculate there own physics for
every player, and you would sync keypress events over the server
giving the trajectory of each player on every client. Every, lets
say, 5 seconds the server broadcasts it's physics simulation and all
clients accept the change. This may create slight offsets that are
unnoticeable most times, but during network lag and packet
loss(inevitable with high traffic UDP) you'll notice your player
and/or other players glitching around the screen and changing
position rapidly and choppily(is that a word?).
You can have each client calculate its own physics and
sync its coordinates. This makes it difficult to simulate physics on
objects shared between clients. Its a pretty complex concept to
implement if you want to do anything snazzy, because certain object don't
necessarily belong to any client.
The first is probably the easiest and should allow you to have about 4-5 players with little lag. It would require each match to have it's own server. If you're doing LAN matches this is hands down the way to go.
The second is probably the most practical, however it can be difficult to implement. It also is pretty resourceful to run physics simulations on the server. If you have centralized servers you'd probably need to load balance to several machines, maybe allow 10 matches a server, load new matches to the server with the least matches.
The third is definitely the least stressful on the server, and is probably the best solution if you're doing a peer to peer network scheme. As I mentioned it can be hard to sync objects other than your player object because those objects are alterable by other clients also.
I can't tell you which one to use because I don't know how your game works. All I can do is give you the facts. If you have any further questions feel free to comment.