I'm looking to be able to customise the part of the os which the user interacts with (the application?), for instance the task bar (on windows), start button, desktop etc. as I have no interest in changing how to load files and process applications, but where do I start looking?
I'm very particular about my desktop too. GNU/Linux has the potential to be much more flexible this way than either Windows or OSX due to the modular design, but to get the most out of it you need to understand how it fits together.
Beyond the kernel, there are three layers involved in a normal linux GUI:
The Xorg server; this is universal -- AFAIK, there isn't any alternative to it (Wayland not-with-standing, see Thomas Nyman's answer). It's called a server because it is; GUI applications are clients of the server. Xorg (formerly X11, and aka. the "X Windowing System") dates back to about the same time as the first MS and Apple GUIs.1 While you can use it all on its own, what it provides is very rudimentary from a user perspective. However, it is important to know about and understand that it is a distinct entity, kind of like the engine in a car. All on its own, an engine is not much use to most people, but of course, neither is a car without an engine.
The window manager, abbreviated WM. While X provides the basic facilities for making windows, the WM adds things like titlebars, frames, and free standing menus. Some window managers are designed exclusively for integration with #3 below, however, many of them can also be used alone. Since the WM is responsible for window behavior and appearance, a good standalone WM can provide you with everything you need -- keeping in mind that things like taskbars are actually special forms of "window", and some WM's can configure and instantiate them. Also, taskbars and docks are often independent applications that can be run in a WM without needing #3. I totally recommend trying this at some point, but I think it may be a little confusing for new users.
The desktop environment, abbreviated DE. DE's are built on top of WM's and often allow you to choose which WM to use, although initially you should stick with the default. In many ways, a DE is actually a GUI front-end to a WM; WM's generally are configured with text files, and the DE provides a point and click interface. They usually also include an integrated application suite, the centerpiece of which is a file browser. However, those applications often don't need the DE (e.g., my fav terminal app is XFCE's Terminal, but I don't use XFCE). Taskbars, as already mentioned, can be an example of this too. So there is a lot of potential for mixing and matching here.
I'm not an authority on the state of the major DE's, so I can't say exactly how flexible they all are WRT configuration. Traditionally, flexibility is more a characteristic of the WM -- the DE ends up limiting this to fit its particular vision. However (you were warned!) a truly dynamic standalone WM usually involves a lot of work with text files about on par with the complexity of HTML/CSS -- it's not hard to do something basic, but getting exactly what you want easily takes some experience.
If you aren't adverse to burning some CD's, I'd cut a few live CD's that use different DE's -- Gnome, KDE, XFCE, etc. -- then see if you can figure out how to make them look the way you want them to look in a short period of time. That may not happen, but it should give you an idea of which one you prefer to work with. Ignore the themes and colors at that point and focus on function.
Currently I use KDE as my primary environment. As mentioned, I am very particular and it is very configurable. I have not looked at GNOME in a long time and don't know how it compares, configurability wise, but I would think those would be the two big candidates for flexibility in the DE category.
The final thing that should be mentioned is the widget set, which is sort of a layer like 1,2,3 above (but not really).2 The widget set is what determines the appearance of generic stuff inside the window frame (buttons, scrollbars, etc.), which is why there are TWO distinct, independent forms of "theme" in play with the GUI appearance: the theme of the widget set (inside the frame), and the theme of the WM (the frame, the titlebar). The most common widget set in linux is GTK, which was developed in conjunction with GNOME but is used by other DE's as well. A significant exception to this is KDE, which uses Qt, a cross-platform toolkit.
1 Yes, that's older than linux. X is used all across the "unix-like" world.
2 "Widget set" is a bit of a loose term since Qt and GTK (+glib) do significantly more than just provide widgets, but that's good enough for now.
