How to deal with duplicate subs

[ALLPlayer]

I'm glad that you are taking care of the problem. I can understand that getting ALLPlayer wider without settings right filtering at the back-end will cause problems with multisubs. Let me know if we could help from the client perspective to make it more efficient for other than Poles nations.

Andy

[oss]

tuszyn: I decided not to remove anything from the subtitles, which were uploaded here (I can, but I will not do that) - of course, we can vote about that, if it is good remove line like napiproject and so on, but I dont think so (I mean when I save subtitles, when comparing subtitls, I can make a small filter for that, so it will work even better)
They did own job, they add that line, so...anyway, if you have some string which is good to remove, just send them (again, they will be not removed in subtitles itself, only for comparing).

But I am thinking about that, and it is not good anyway to remove even for comparing, I have own reasons for that. If the filter you gave me will be final and it will not change in the time, it is ok, otherwise not (when changing filter, I have to refilter all subtitles...), my method is strong against this anyway.

I will work on that hard, it is not easy job, to refilter all those subtitles, it is also CPU consuming, I hope I will get really good results, because my method seems really promising and should be also quite fast.

ALLPlayer: you did perfect job, we are looking for new release even with improved Release Name autofilling and proper 2CD uploading and so on. Then I will write some article on blog about that.

[eduo]

Site is still paralyzed.
I suggest to FULLY disable anonymous uploads
til solution comes.
I'm fuckin not joking.

No, sir. You're not "fuckin" and neither are you joking. What you're doing is swearing. You're also out of place. I don't see a need for talking like this, especially when you can't even do it right.

I'm sorry, but this is completely out of place.

I'm unsubscribing from this thread. I have enough ridiculous swearing in my own life to have to read the same here.

[ensky]

Well, without any "swearing" that can be totally misunderstood by non-english folks - a simple question - when do you think this great site is going to be functional again?
It is lagging and erroring horrendously, as of now.

I am surprised it allowed me to create an account and login with few tries - only took an hour ...

Best regards

[eduo]

Site is still paralyzed.
I suggest to FULLY disable anonymous uploads
til solution comes.
I'm fuckin not joking.

No, sir. You're not "fuckin" and neither are you joking. What you're doing is swearing. You're also out of place. I don't see a need for talking like this, especially when you can't even do it right.

I'm sorry, but this is completely out of place.

I'm unsubscribing from this thread. I have enough ridiculous swearing in my own life to have to read the same here.

Would be easier if you spoke english.

Don't be confused. My english is probably one of the best around here but this has nothing to do with speaking english (something I clearly do better than you) but with being out of line and swearing without any need.

Is any of those poles going to read your swearing? No. Is os going to pay more attention to you because you're swearing (and swearing badly, at that. It should be "I'm not fucking joking", if anything)? No, he's not.

You may have a point, but it gets lost in the childish language.

Ensky: The swearing is not confusing to non-english folks. It's just out of line to everyone reading this. The fact that the english is badly written and fractured is secondary and not to be ashamed of, except when those perpetrating it believe it's the others who don't understand.

The servers have been faulty for a long time now. AllPlayer's inclusion of automatic upload capabilities just was the final straw to bring the server to its knees. Some of us have been offering hosting and mirroring for a while to avoid this.

I believe the first step would be to disable anonymous subtitle upload, to control the damage while other steps are taken. But I can understand why os (who should really pick a better nickname ) doesn't want to do that.

The next thing to do is to block uploads by AllPlayer. It's not worth it at this time. The functionality is broken and the AllPlayer dev should fix it. Since all the uploads are supposed to come with a user-agent then AllPlayer's should be blocked indefinitively (replaced by a different one when it's fixed).

If none of the above can be done, then no upload should be accepted for any polish video that already has 5 subtitles uploaded.

These simple and quick measures could help have the server work better for the rest.

[eduo]

eduo, I am upset because of my very favourite site is unusable, so I am sorry.
As I can see, we both have the very same aims, so there's no need to fight (which would be NICE in any other cases:DDD).

Agreed.

And, hell, I can tell who speaks better english, and yours' not better than mine, dont write comments in angry mood and everything will be OK:PPP

Sorry. Maybe I didn't mention it. I have been an english teacher. It's not my opinion, I actually know I have a better level of english. This is a fact, and nothing to be ashamed of. We can drop the subject or, if you want, I can put on my english teacher hat and correct your posts. I don't think that's necessary, though.

[oss]

Hi all,

today I finished programming of similar subtitles. Script is still running and disabling similar subtitles (so far disabled around 2000 subs). I will not define exactly what is similar subtitle and how exactly it works.

For uploading - I am checking similarity only if it is anonymous uploader. After some time, when it will work good I can change it for all users - I need your opinion, admins.

Also I changed behavior of disabling subtitles - that means, if you disable as admin subtitle, same (md5) subtitle cannot be uploaded.

For server response times - I see in mtop there are some queries that takes more than 30 seconds, sometimes more than 100...I can only optimize and optimize those queries (if it is still possible).

Also I have some more ideas how to improve response times I will work on that in these days.

For server mirroring - we can try static.opensubtitles.org as first for static files, it helps a little bit.

[ALLPlayer]

We have way to cut down on subs that differ from each other. Our ALL will set some footprint in subs so we should see progress in upload and get *8 less subs upload to the server than we have now.. keep you posted.

PS. more comes, even more subs we will cut down with setting the issue with 1CD, 2CD subs and that you will see *16 less subs than now

[oss]

ok, today I worked hard again here are some results:

* total subtitles removed using my algo: around 20.000
* optimized download for subtitles
* optimized some tables and indexes, site should work faster, we will see on peak

Omerta: I know, what you mean about CPU, but believe me, I programmed so it should use less CPU - now I am storing md5 of subtitles to another table, so when server receive same subtitle, which was already rejected it will not make comparison again. We will see, but I hope server will be more stable, but it seems I will need to setup slave db server anyway...db is always bottleneck

AllPlayer: please explain that with footprint, also please don't change subtitles which are uploaded to server in anyway, it is not good idea.

thanks.

[ALLPlayer]

OK, so soon we will get new version of ALLPlayer that should dramatically decrease the identical subs. We have added three algorithms that makes this happen.
1. Fixing the 1CD, 2CD issue
2. Usually as many sites that distribute subs, they add own footage, so the same sub can has many control sums, and Server sees this as different subs not as one.
3. We have 2 mln users, so if every 200 will do automatic upload we will get the right subs but OpenSub server will not get slammed

[oss]

please explain in technical language point 2 please.

Again, I think it is not good, if you will do anything with source subtitles, it is not system solution, everything should be done on server side, if possible.

[ALLPlayer]

The subs are made by the groups, its great credit to them that they do and upload subs for community. This is sth that we don't want to touch. But there are guys that promote own sites due competition is great and they would like to show that more subs are coming from. like napisy.info, napisy24 or hatak. They add extra language that changes the sum control of the sub. Our idea is to leave the signature of the authors but unite the subs that are the same.

[oss]

AllPlayer - I understand you - you want to help opensubtitles.org. But in general, I dont think it is good idea - I cannot rely on your system, because there will be more programs like yours - will they implement same mechanizm? I dont think so...

So, it is up to you, if you implement this, on opensubtitles server will be uploaded same amount (cca, but I will not go to details... of subtitles without other servers signatures