Take a look at what OpenStreetMap does. There's a page describing the nature of tile server disk usage. If you go up to zoom level 18 worldwide, you're talking about 91,625,968,981 tiles, which would take around 54000GB of disk space, but would mostly never be viewed.
So I'm not sure if it would ever be a sensible approach, but having said that, I heard that MapBox do pre-generate all their tiles when hosting a tile set. I think they go up to higher zoom levels just in the cities or something like this.
The approach OpenStreetMap tile servers use, is a combination of on-the-fly rendering and caching. The management of this is done with a specially written apache module called mod_tile
Either way, if you want to do things worldwide up to a high zoom level, you need something a little more complicated than a filesystem full of 256x256px PNG images. mod_tile stores files in cache as a 'meta-tiles'. MapBox uses a format called MBTiles to store all the tiles in a database file.
I mentioned MapBox a few times. Another interesting hosting option is tiledrawer, or there's various other providers of tile hosting/rendering and other map services
In general you'll find the OpenStreetMap tech community have a lot of experience with this kind of thing. You can contact them in various ways. There's even a question & answer site: https://help.openstreetmap.org
