The API is indirectly killing OS, but it's the DB access behind it the real reason. The API just is so massive the the site slashdots itself.
I was talking about this with OS just today. I realize reading this and comparing with my own assumption that we severely underestimate the size an traffic involved in OS.
Some stats that have been mentioned around:
-Half a million daily subtitle downloads.
-A third of a daily million pageviews.
-40 thousand subs downloaded from the web page (no massive download)
-Over 10GB of Subs in individually compressed files.
P2P has been mentioned before. I still believe it's a non-item and makes no sense. We're talking millions of individual files, with thousands more every day, each less than 50k in size.
What I've proposed is to have part of the API be file-dependant instead of DB-dependant, something you've touched upon here. Also: To put them all in Amazon S3 or a CDN.
byhash/moviehash-bytesize/iso-639/
--> list.xml
byimdibd/iso-639/
--> list.xml
But without knowing the plan for the rewrite all of this is just that, speculation
OpenSubtitles is not Open, so we can only give ideas and offer help where possible.
Incidentally: I have invested a ton of effort and time in *selling* OpenSubtitles as a subtitle repository but I won't hide the fact that, to me, the selling point is the API and hash-based searches. Without those OpenSubtitles is one of the biggest subtitle repositories, but not much difference outside of that. The differentiator is the API and as such all my suggestions are aimed at making the API feasible and cost-effective. If I offer suggestions for the web itself it's just because I understand *something* needs to pay for all this bandwidth and CPU and web ads is the only thing I'm willing to accept, rather than advertising in the API or, god forbid, in the subtitles themselves.