Forum rules
Under no circumstances is spamming or advertising of any kind allowed. Do not post any abusive, obscene, vulgar, slanderous, hateful, threatening, sexually-orientated or any other material that may violate others security. Profanity or any kind of insolent behavior to other members (regardless of rank) will not be tolerated. Remember, what you don’t find offensive can be offensive to other members. Please treat each other with the kind of reverence you’d expect from other members.
Failure to comply with any of the above will result in users being banned without notice. If any further details are needed, contact: “The team” using the link at the bottom of the forum page. Thank you.
User avatar
explosiveskull
Posts: 146
Joined: Mon Jul 17, 2017 4:33 pm

No point in ratings

Sun Apr 18, 2021 1:12 am

Hi. I have been monitoring the ratings feature on the site and since I have been uploading, I have seen literally no use for this on the site other than haters submitting low ratings, for which they don't provide any explanations and ignore requests to provide explanations.

Has this feature ever helped in some way before? If not, probably won't be a bad idea to slowly get rid of it. As an example, just take a look at my most recent uploads. You can see multiple bad ratings based on literally no reason. At some points, I have even noticed people registering just to submit a bad rate, which reminds me of Subscene.

User avatar
SmallBrother
Site Admin
Posts: 3287
Joined: Sun Mar 04, 2012 12:59 pm
Location: Somewhere on this globe

Re: No point in ratings

Sun Apr 18, 2021 8:56 pm

I understand. Yes, I have some doubts, too. And yes, I have seen similar shit. People rating 10/10 just because they are happy (before even seeing the movie/episode) and I have seen 1/10 because it's competition, or because there is one comma too much or a dot too little or the given release is wrong...

Ratings are subjective, a personal opinion, and therefore hard to 'justify'. But there are admins who could remove ridiculous ratings and bad marks. People tend to rate the lowest 1/10 if 'something' is wrong. Which doesn't mean they are aggressive or 'against' you, usually it only means they are just not very nuanced. In the Dutch section, I have written kind of a 'guide' how to rate subtitles.

It is NOT a good idea if an admin is evaluating every user's personal opinion. But I think a 1/10 and a 'bad' mark is only suitable for utterly useless subs. And therefore they can easily be removed, if it is just a 'political' or 'idiot' rating. For me, as admin, I have as guideline that I do NOT evaluate/remove ratings, UNLESS they are obviously nonsense. If so, I notice and remove by myself, or users can ask me to remove.

User avatar
explosiveskull
Posts: 146
Joined: Mon Jul 17, 2017 4:33 pm

Re: No point in ratings

Sun Apr 18, 2021 10:58 pm

The only thing I don't agree with you on, SB, is the "can't be justified" part. Everything can be justified. Like, "why do you think it's worth only 5/10" or "what do you think I should do to make it a 10/10 in your opinion". That way, I can think about it and see if it's reasonable and matches subtitling standards so I can decide whether to apply changes or keep them as they are. But when you just register your account in 2 minutes and submit a 5/10 out of nowhere with no comments, that can only mean you're a hater.

The idea I had in mind about ratings was to make it more like something that requires comments and without comments they won't go through. This is a good way to identify who's rating based on reason and who's just trying to show hate and troll.

User avatar
SmallBrother
Site Admin
Posts: 3287
Joined: Sun Mar 04, 2012 12:59 pm
Location: Somewhere on this globe

Re: No point in ratings

Sun Apr 18, 2021 11:11 pm

Yes, I agree, the rating system could use some improvement ;-)

What I meant with "can't be justified" is (for example) the difference between 5/10 and 7/10. How do you 'weigh' some mistakes, the severity of it, the number of it, the value of it... That's impossible to verify or even to discuss. The only thing which is possible is the extreme cases: I have seen Google translations getting a 10/10 (I am serious!) and I have seen a 1/10 because there were a couple of OCR errors.

For now, the only thing admins and users can do is ask the others to be a bit 'objective' in their subjectivity.
Obviously this doesn't help against haters and so.

User avatar
oss
Site Admin
Posts: 5148
Joined: Sat Feb 25, 2006 11:26 pm
Contact: Website

Re: No point in ratings

Mon Apr 19, 2021 11:47 pm

if you have some ideas how to make ratings better (in new site), let us know, so we can work on this. Maybe the user have to download subtitle, and be allowed to rate it after 1 hour minimum...? Must provide comment, if rating is lower than Max rating...?

User avatar
explosiveskull
Posts: 146
Joined: Mon Jul 17, 2017 4:33 pm

Re: No point in ratings

Tue Apr 20, 2021 7:49 am

oss wrote:if you have some ideas how to make ratings better (in new site), let us know, so we can work on this. Maybe the user have to download subtitle, and be allowed to rate it after 1 hour minimum...? Must provide comment, if rating is lower than Max rating...?

The last one seems to be reasonable in my opinion. If the rating is too low, we need to know "what's wrong" and if it's close to max but not max, "what could've been done better?" These are legit questions to ask when it comes to ratings and I think if the user provides answers to these, we won't need any other limitations. Just as simple as that.

os_dev
Posts: 80
Joined: Wed Oct 17, 2018 3:42 pm

Re: No point in ratings

Tue Apr 20, 2021 9:39 am

I've been giving a lot of efforts trying to make a better way to rate subtitles on the new site, could you give me your opinion on it ?

The basic idea is:

- if the subtitle has nothing wrong, then the user gives 5 stars, there's nothing more to add

Image

- if the subtitle has less than 4 stars, then checkboxes appear, user can specify what was the problem

Image




- if the user ticks "other", then a text field appears, allowing the user to fill in more details.

Image

User avatar
SmallBrother
Site Admin
Posts: 3287
Joined: Sun Mar 04, 2012 12:59 pm
Location: Somewhere on this globe

Re: No point in ratings

Tue Apr 20, 2021 9:51 am

oss wrote:if you have some ideas how to make ratings better (in new site), let us know, so we can work on this. Maybe the user have to download subtitle, and be allowed to rate it after 1 hour minimum...? Must provide comment, if rating is lower than Max rating...?

I like both methods.

Must provide comment:
I agree with Explosiveskull. To avoid vague comments like "bad", ask what is wrong (if rating is less than 10/10). Maybe use a simple list understandable by non-subtitlers, like
- translation not accurate
- grammatical errors
- typos/spelling errors
- messy text formatting
- not in sync
- reading speed too high
- missing parts
- other: .........

It should be possible to mark more than one option.
The option "other" must have some description of more than eeeeeh 10 characters including at least one space.

Allow rating after 1 hour:
I think this is a good idea. Too often people rate out of happiness and then it's always a 10/10. Or the IMDb ID is wrong and the subs get a 1/10.

Adjusting the stars/numbers:
And I think the 5-star / 10 points system should be adjusted.
3 stars should mean someting like "just okay", "sufficient enough", while now the NUMBER refers to 5/10 which sounds pretty bad.
The 4 stars should mean somethings like "almost perfect". The number 7 is too little.
I would say the numbers should be 1 - 3.5 - 6 - 8.5 - 10

Do not allow rating for totally fresh new users:
And maybe totally fresh users should not be able to rate, to avoid revenge or competition wars by troll users. Allow a user to rate only after one week and/or after 10 downloads.

Double value for Trusted and Sub Translator users:
Whenever a Trusted user or Sub Translator rates a subtitle, this should be counted TWO times, or maybe even three times to calculate the average.

User avatar
SmallBrother
Site Admin
Posts: 3287
Joined: Sun Mar 04, 2012 12:59 pm
Location: Somewhere on this globe

Re: No point in ratings

Tue Apr 20, 2021 10:06 am

os_dev wrote:I've been giving a lot of efforts trying to make a better way to rate subtitles on the new site, could you give me your opinion on it ?

I saw your post only after posting mine... Looks like we agree a lot :)

As for your checklist, the options "wrong movie" and "wrong language" should NOT be a reason to give a low rating. Because 1. this doesn't necessarily make the subs bad and 2. it can be easily corrected.

If you want to make it really fancy, you could still include those two options, but if this is the ONLY reason, do NOT actually add the rating, but link the user to a correction form, to be reviewed by an admin...

A bit similar for stolen/missing credits... but I wouldn't know how to solve this...
Incomplete translation and missing parts - isn't this the same?
OCR errors and encoding errors - I think the majority of users don't know what this is.

os_dev wrote:if the subtitle has nothing wrong, then the user gives 5 stars, there's nothing more to add

Maybe ask them the question WHY they give 5 stars:
A. because I am very happy with this subtlitle.
B. because I saw the movie/episode and the subs are perfect.

If thy choose option A, do NOT add the rating, but add a "thank you". Display a small message explaining that the rating system is about quality, not about happiness.

os_dev
Posts: 80
Joined: Wed Oct 17, 2018 3:42 pm

Re: No point in ratings

Tue Apr 20, 2021 10:22 am

SmallBrother wrote:
os_dev wrote:I've been giving a lot of efforts trying to make a better way to rate subtitles on the new site, could you give me your opinion on it ?

I saw your post only after posting mine... Looks like we agree a lot :)

haha, you almost got me worried for a second, I was like - "hey, but I just answered that ?!?"


SmallBrother wrote:As for your checklist, the options "wrong movie" and "wrong language" should NOT be a reason to give a low rating. Because 1. this doesn't necessarily make the subs bad and 2. it can be easily corrected.


Yeah my idea there was more to give a signal to the admins to correct the film

SmallBrother wrote:If you want to make it really fancy, you could still include those two options, but if this is the ONLY reason, do NOT actually add the rating, but link the user to a correction form, to be reviewed by an admin...

True, if the user picks "wrong movie" then I should not count the rating

SmallBrother wrote:
A bit similar for stolen/missing credits... but I wouldn't know how to solve this...
Incomplete translation and missing parts - isn't this the same?
OCR errors and encoding errors - I think the majority of users don't know what this is.


for incomplete/missing, I was thinking of films that suddenly switch language, typically a translator who skipped a line (that would be incomplete translation), or bits where there is no text at all (that would be missing parts).

for OCR/encoding, I assumed people would tick that one when they see weird symbols - but normally this doesn't happen much on .com as the buggy subtitles are detected and removed.

User avatar
explosiveskull
Posts: 146
Joined: Mon Jul 17, 2017 4:33 pm

Re: No point in ratings

Tue Apr 20, 2021 3:16 pm

I'll be personally satisfied with just the commenting requirement and possible predefined options, but adding the impossibility of rating for new users up to one week is also a good idea which you see a lot of childish behavior due to the lack of it it on other websites. Other options, I have no opinion on, up to you guys. This was a good brainstorming topic.

User avatar
scooby007
Site Admin
Posts: 659
Joined: Thu Mar 05, 2009 10:49 pm
Location: Scandalous

Re: No point in ratings

Tue Apr 20, 2021 6:29 pm

SmallBrother wrote:As for your checklist, the options "wrong movie" and "wrong language" should NOT be a reason to give a low rating. Because 1. this doesn't necessarily make the subs bad and 2. it can be easily corrected.


Yeah my idea there was more to give a signal to the admins to correct the film


Yes, anything like that, please don't allow them to rate at all, but direct them to report page to make the amendment to change what they think is wrong.

Maybe ask them the question WHY they give 5 stars:


Nope, too much requirement.
It's a good rating. Leave that be, otherwise people will just stop rating as it takes too much time.
Comment only required for low rating. Keep it fast and simple.

No one wants to spend too much time on it.
Only bug them if the rating being offered is a low one.
If it's a good one, the process should be a fast one to encourage a good rating, rather than, "I never rate anything 10, so I'll be different and offer a 7 for no reason."

Keep it fast and simple.
Something like: Pants down, wham-bham, thank you, ma'am! :)
Image

User avatar
explosiveskull
Posts: 146
Joined: Mon Jul 17, 2017 4:33 pm

Re: No point in ratings

Wed Apr 21, 2021 9:47 am

I think if we want to make the rating process quick, having those predefined options is the best in addition to comments, but not having any of those for low ratings doesn't make any sense and I don't consider 5 a good rating personally. I'd say 10 to 8 are good ratings if we don't want to show any options or a comments field, but lower than that requires further explanation in my opinion. The "wham-bham" doesn't always produce the best outcome, if you know what I mean. :D

User avatar
oss
Site Admin
Posts: 5148
Joined: Sat Feb 25, 2006 11:26 pm
Contact: Website

Re: No point in ratings

Wed Apr 21, 2021 3:09 pm

important we all agree the direction, detail if comment is required for rating (1,2,3,4 from 5) or from (1,2,3 from 5) is really detail.

User avatar
SmallBrother
Site Admin
Posts: 3287
Joined: Sun Mar 04, 2012 12:59 pm
Location: Somewhere on this globe

Re: No point in ratings

Wed Apr 21, 2021 5:36 pm

scooby007 wrote:
Maybe ask them the question WHY they give 5 stars:

Nope, too much requirement.
Comment only required for low rating. Keep it fast and simple.
No one wants to spend too much time on it.

Of course, it shouldn't be a dramatic hassle to make a rating. If we can nag about choosing from a list of 10 reasons if rated 1-4 stars, I think it's not too much asked to have the user choose between two options if the rating was 5 stars. Otherwise we will end up having reliable/no low ratings, but unreliable/many high ratings...

Return to “General talk”

Who is online

Users browsing this forum: No registered users and 5 guests