I know about XNA and physics engines such as farseer but, at the moment, I'd like to get started without attaching myself to a particular framework or set of classes.
You'll need to use some API for rendering, otherwise you won't be seeing the effects of your physics simulation. You say you are proficient in C#, but do you have any experience with graphics in C#? It could be for instance, just plain old GDI in WinForms, a DirectX wrapper like SlimDX or SharpDX, an OpenGL wrapper like OpenTK, or a game-oriented framework like XNA. Doesn't really matter which, but you'll need to learn at least one graphics API, otherwise how do you expect to see your physics running?
I guess I'm trying to avoid using too large a framework that will do too much for me.
Then the suggestion to use XNA is quite valid. Take a look at this. This is the amount of code you need to add to an empty XNA project to get it to render a sprite. And that's pretty much the extent of what you'd need to learn from the framework in order to to play around with physics. In comparison, it's a lot less work than any of the alternatives I mentioned.
So if your intent is just to get your hands dirty with physics, I'd recommend learning the bare minimum required to do some rendering in XNA; and then try and play around with Farseer Physics for instance. Shouldn't be hard to find some tutorials by googling.
