You don't actually have to purchase the Data Interoperability Extension to integrate basic WFS functionality into ArcGIS. The installer is on your ArcGIS DVD / .iso. The extension is an ESRI-wrapped basic version of FME and can read GML into ArcView without a license.
By default it starts by downloading all features in the WFS layer you connect it to and caches them for 24 hours. Once the features are cached a WFS-sourced layer functions just like any other layer which includes querying, attribute-based symbology etc.
If your layer has a unique numeric identifier field you can also use the WFS 'live' with no caching and requests going to GeoServer everytime the ArcGIS user moves the map, does an identify etc - this means you don't have to wait 24 hours for the cache to expire so you can see updated data but obviously slows things down (as far as I'm aware there is no easy way to manually dump the cache to force ArcView to rebuild it, but this may be possible through ArcObjects).
There is a bug / deficiency in the integration between GeoServer and ArcGIS around XML namespacing. I asked about it on the ESRI forum here and the brief response suggests that the problem could be fixed or worked-around more easily in 10.1