Forum rules
Under no circumstances is spamming or advertising of any kind allowed. Do not post any abusive, obscene, vulgar, slanderous, hateful, threatening, sexually-orientated or any other material that may violate others security. Profanity or any kind of insolent behavior to other members (regardless of rank) will not be tolerated. Remember, what you don’t find offensive can be offensive to other members. Please treat each other with the kind of reverence you’d expect from other members.
Failure to comply with any of the above will result in users being banned without notice. If any further details are needed, contact: “The team” using the link at the bottom of the forum page. Thank you.
User avatar
oss
Site Admin
Posts: 5878
Joined: Sat Feb 25, 2006 11:26 pm
Contact: Website

[NEW] REST search, JSON results

Mon Mar 26, 2018 10:52 am

Hi guys,

More and more clients are connecting to opensubtitles API using XML-RPC which is difficult to cache. We are proud to present you REST search. Idea is simple - client make URL, request it and we cache results on CDN (in Cloudflare), that means, responses are faster and our servers can take care of something else.

Make sure you provide your XML-RPC useragent (API) in http header. (#curl -A 'registeredUA' or <?php curl_setopt($curl, CURLOPT_USERAGENT, $agent); )

OK, how it looks?

Code: Select all

https://rest.opensubtitles.org/search/episode-20/imdbid-4145054/moviebytesize-750005572/moviehash-319b23c54e9cf314/season-2/sublanguageid-eng
to create request you need base URL, which is https://rest.opensubtitles.org/search and then search parameters ordered by name, otherwise you will be redirected (please sort them on client side!). Make sure url is in _lowercase_, otherwise redirect. Valid parameters are:

Code: Select all

episode (number) imdbid (always format it as sprintf("%07d", $imdb) - when using imdb you can add /tags-hdtv for example. moviebytesize (number) moviehash (should be always 16 character, must be together with moviebytesize) query (use url_encode, make sure " " is converted to "%20") season (number) sublanguageid (if ommited, all languages are returned) tag (use url_encode, make sure " " is converted to "%20")
Valid URLs (examples, try yourself to combine):

Code: Select all

curl -A 'TemporaryUserAgent' https://rest.opensubtitles.org/search/moviebytesize-750005572/moviehash-319b23c54e9cf314 curl -A 'TemporaryUserAgent' https://rest.opensubtitles.org/search/episode-20/imdbid-4145054/moviebytesize-750005572/moviehash-319b23c54e9cf314/season-2/sublanguageid-eng curl -A 'TemporaryUserAgent' https://rest.opensubtitles.org/search/episode-11/imdbid-4145054/season-1/tags-web-dl curl -A 'TemporaryUserAgent' https://rest.opensubtitles.org/search/query-matrix%20reloaded/sublanguageid-eng curl -A 'TemporaryUserAgent' https://rest.opensubtitles.org/search/episode-8/sublanguageid-eng/tag-heroess01e08
NEW: for clients unable to change default User Agent it is possible now to set up custom HTTP header 'X-User-Agent':

Code: Select all

curl --header "X-User-Agent: TemporaryUserAgent" https://rest.opensubtitles.org/search/query-matrix%20reloaded/sublanguageid-cze

INVALID URLs:

Code: Select all

# duplicate parameters are not permitted https://rest.opensubtitles.org/search/imdbid-4145054/imdbid-1234567 # duplicate values per parameter are not permitted https://rest.opensubtitles.org/search/imdbid-4145054/sublanguageid-eng,cze
Results are limited to 500 items. This REST method (search) is public and doesn't need any key or any special user-agent. Results are sorted by SCORE, so which are first in list should be better.
Always check for 200 response http code.
Example:

Code: Select all

# https://rest.opensubtitles.org/search/episode-1/imdbid-4145054/season-1/sublanguageid-cze < HTTP/2 200 < date: Mon, 26 Mar 2018 08:48:54 GMT < content-type: application/json < set-cookie: __cfduid=debccacb67b21029477d3d105bd3943131522054134; expires=Tue, 26-Mar-19 08:48:54 GMT; path=/; domain=.opensubtitles.org; HttpOnly < cache-control: public, max-age=43200 < last-modified: Mon, 26 Mar 2018 10:47:00 +0200 < access-control-allow-origin: * < access-control-allow-headers: ETag, X-RateLimit-Remaining, Content-Type, Last-Modified < x-robots-tag: noindex, nofollow < vary: Accept-Encoding < x-cache-backend: web3 < x-cache: MISS < x-ratelimit-remaining: 40 < x-via: fw2.int.opensubtitles.org < cf-cache-status: HIT < expires: Mon, 26 Mar 2018 20:48:54 GMT < expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct" < server: cloudflare < cf-ray: 40185c628ecb6f7e-SIN [{"MatchedBy":"imdbid","IDSubMovieFile":"0","MovieHash":"0","MovieByteSize":"0","MovieTimeMS":"0","IDSubtitleFile":"1955043364","SubFileName":"Shadowhunters.S01E01.HDTV.x264-KILLERS.srt","SubActualCD":"1","SubSize":"42803","SubHash":"66fea5461f173f689c3675c372cd055a","SubLastTS":"00:38:57","SubTSGroup":"1","IDSubtitle":"6461000","UserID":"0","SubLanguageID":"cze","SubFormat":"srt","SubSumCD":"1","SubAuthorComment":"","SubAddDate":"2016-01-13 23:19:41","SubBad":"0","SubRating":"0.0","SubSumVotes":"0","SubDownloadsCnt":"1332","MovieReleaseName":" Shadowhunters.S01E01.HDTV.x264-KILLERS","MovieFPS":"0.000","IDMovie":"406174","IDMovieImdb":"4601794","MovieName":"\"Shadowhunters: The Mortal Instruments\" The Mortal Cup","MovieNameEng":null,"MovieYear":"2016","MovieImdbRating":"7.6","SubFeatured":"0","UserNickName":"","SubTranslator":"","ISO639":"cs","LanguageName":"Czech","SubComments":"0","SubHearingImpaired":"0","UserRank":"","SeriesSeason":"1","SeriesEpisode":"1","MovieKind":"episode","SubHD":"0","SeriesIMDBParent":"4145054","SubEncoding":"UTF-8","SubAutoTranslation":"0","SubForeignPartsOnly":"0","SubFromTrusted":"0","SubTSGroupHash":"a5985421faabcab605e61243794a4d7f","SubDownloadLink":"https:\/\/dl.opensubtitles.org\/en\/download\/src-api\/vrf-19a20c51\/filead\/1955043364.gz","ZipDownloadLink":"https:\/\/dl.opensubtitles.org\/en\/download\/src-api\/vrf-f52d0baa\/subad\/6461000","SubtitlesLink":"http:\/\/www.opensubtitles.org\/en\/subtitles\/6461000\/shadowhunters-the-mortal-instruments-the-mortal-cup-cs","QueryNumber":"0","QueryParameters":{"episode":1,"season":1,"imdbid":"4145054","sublanguageid":"cze"},"Score":10.01332},{"MatchedBy":"imdbid","IDSubMovieFile":"0","MovieHash":"0","MovieByteSize":"0","MovieTimeMS":"0","IDSubtitleFile":"1955231797","SubFileName":"Shadowhunters S01E02 The Descent Into Hell Isn't Easy.srt","SubActualCD":"1","SubSize":"51130","SubHash":"5ec1754f14ed36e087a5c799815fadc9","SubLastTS":"00:40:41","SubTSGroup":"3","IDSubtitle":"6651521","UserID":"0","SubLanguageID":"cze","SubFormat":"srt","SubSumCD":"1","SubAuthorComment":"","SubAddDate":"2016-06-08 15:54:26","SubBad":"0","SubRating":"0.0","SubSumVotes":"0","SubDownloadsCnt":"170","MovieReleaseName":" Shadowhunters S01E02 The Descent Into Hell Isn't Easy","MovieFPS":"25.000","IDMovie":"406174","IDMovieImdb":"4601794","MovieName":"\"Shadowhunters: The Mortal Instruments\" The Mortal Cup","MovieNameEng":null,"MovieYear":"2016","MovieImdbRating":"7.6","SubFeatured":"0","UserNickName":"","SubTranslator":"","ISO639":"cs","LanguageName":"Czech","SubComments":"0","SubHearingImpaired":"0","UserRank":"","SeriesSeason":"1","SeriesEpisode":"1","MovieKind":"episode","SubHD":"0","SeriesIMDBParent":"4145054","SubEncoding":"CP1250","SubAutoTranslation":"0","SubForeignPartsOnly":"0","SubFromTrusted":"0","SubTSGroupHash":"a201acec74b8fdf5f64130bda9a7c5c7","SubDownloadLink":"https:\/\/dl.opensubtitles.org\/en\/download\/src-api\/vrf-19b60c5a\/filead\/1955231797.gz","ZipDownloadLink":"https:\/\/dl.opensubtitles.org\/en\/download\/src-api\/vrf-f5480bb3\/subad\/6651521","SubtitlesLink":"http:\/\/www.opensubtitles.org\/en\/subtitles\/6651521\/shadowhunters-the-mortal-instruments-the-mortal-cup-cs","QueryNumber":"0","QueryParameters":{"episode":1,"season":1,"imdbid":"4145054","sublanguageid":"cze"},"Score":10.0017},{"MatchedBy":"imdbid","IDSubMovieFile":"0","MovieHash":"0","MovieByteSize":"0","MovieTimeMS":"0","IDSubtitleFile":"1955209010","SubFileName":"Shadowhunters - 01x01 - The Mortal Cup (Pilot).KILLERS.English.HI.C.orig.Addic7ed.com.srt","SubActualCD":"1","SubSize":"41037","SubHash":"b67051a0013745caf50c0706c2a954ae","SubLastTS":"00:38:58","SubTSGroup":"1","IDSubtitle":"6628618","UserID":"0","SubLanguageID":"cze","SubFormat":"srt","SubSumCD":"1","SubAuthorComment":"","SubAddDate":"2016-05-17 13:33:06","SubBad":"0","SubRating":"0.0","SubSumVotes":"0","SubDownloadsCnt":"79","MovieReleaseName":" Shadowhunters - 01x01 - The Mortal Cup (Pilot).KILLERS.English.HI.C.orig.Addic7ed.com","MovieFPS":"23.976","IDMovie":"406174","IDMovieImdb":"4601794","MovieName":"\"Shadowhunters: The Mortal Instruments\" The Mortal Cup","MovieNameEng":"","MovieYear":"2016","MovieImdbRating":"7.6","SubFeatured":"0","UserNickName":"","SubTranslator":"","ISO639":"cs","LanguageName":"Czech","SubComments":"0","SubHearingImpaired":"0","UserRank":"","SeriesSeason":"1","SeriesEpisode":"1","MovieKind":"episode","SubHD":"0","SeriesIMDBParent":"4145054","SubEncoding":"CP1250","SubAutoTranslation":"0","SubForeignPartsOnly":"0","SubFromTrusted":"0","SubTSGroupHash":"a5985421faabcab605e61243794a4d7f","SubDownloadLink":"https:\/\/dl.opensubtitles.org\/en\/download\/src-api\/vrf-199b0c49\/filead\/1955209010.gz","ZipDownloadLink":"https:\/\/dl.opensubtitles.org\/en\/download\/src-api\/vrf-f55d0bbe\/subad\/6628618","SubtitlesLink":"http:\/\/www.opensubtitles.org\/en\/subtitles\/6628618\/shadowhunters-the-mortal-instruments-the-mortal-cup-cs","QueryNumber":"0","QueryParameters":{"episode":1,"season":1,"imdbid":"4145054","sublanguageid":"cze"},"Score":10.00079},{"MatchedBy":"imdbid","IDSubMovieFile":"0","MovieHash":"0","MovieByteSize":"0","MovieTimeMS":"0","IDSubtitleFile":"1955105830","SubFileName":"Shadowhunters.S01E01.1080p.WEB-DL.DD5.1.H264-RARBG.srt","SubActualCD":"1","SubSize":"41135","SubHash":"2a012782be0d93f31c4053669d6c2f76","SubLastTS":"00:39:05","SubTSGroup":"5","IDSubtitle":"6524935","UserID":"0","SubLanguageID":"cze","SubFormat":"srt","SubSumCD":"1","SubAuthorComment":"","SubAddDate":"2016-02-25 21:36:46","SubBad":"0","SubRating":"0.0","SubSumVotes":"0","SubDownloadsCnt":"74","MovieReleaseName":" Shadowhunters.S01E01.1080p.WEB-DL.DD5.1.H264-RARBG","MovieFPS":"23.976","IDMovie":"406174","IDMovieImdb":"4601794","MovieName":"\"Shadowhunters: The Mortal Instruments\" The Mortal Cup","MovieNameEng":"","MovieYear":"2016","MovieImdbRating":"7.6","SubFeatured":"0","UserNickName":"","SubTranslator":"","ISO639":"cs","LanguageName":"Czech","SubComments":"0","SubHearingImpaired":"0","UserRank":"","SeriesSeason":"1","SeriesEpisode":"1","MovieKind":"episode","SubHD":"1","SeriesIMDBParent":"4145054","SubEncoding":"CP1250","SubAutoTranslation":"0","SubForeignPartsOnly":"0","SubFromTrusted":"0","SubTSGroupHash":"ad6e0f888438099ed83c2d1a335588bd","SubDownloadLink":"https:\/\/dl.opensubtitles.org\/en\/download\/src-api\/vrf-19a10c4e\/filead\/1955105830.gz","ZipDownloadLink":"https:\/\/dl.opensubtitles.org\/en\/download\/src-api\/vrf-f5510bbb\/subad\/6524935","SubtitlesLink":"http:\/\/www.opensubtitles.org\/en\/subtitles\/6524935\/shadowhunters-the-mortal-instruments-the-mortal-cup-cs","QueryNumber":"0","QueryParameters":{"episode":1,"season":1,"imdbid":"4145054","sublanguageid":"cze"},"Score":10.00074}]
If you will find out this useful, let us know.

More REST methods to play with:

Code: Select all

curl -A 'TemporaryUserAgent' http://rest.opensubtitles.org/suggest/query-a/sublanguageid-eng curl -A 'TemporaryUserAgent' http://rest.opensubtitles.org/getsublanguages/language-en curl -vvv -d 'username=your_username&password=your_password' -X POST -A 'TemporaryUserAgent v1.2' https://rest.opensubtitles.org/auth curl -vvv --user username:password -A 'TemporaryUserAgent v1.2' https://rest.opensubtitles.org/auth
Example passing SID parameter for download URL:

Code: Select all

http://dl.opensubtitles.org/en/download/src-api/vrf-19d50c5e/sid-{YOUR_SID_HERE}/filead/1951976249.gz
If you want to download subtitles as logged in user, either use http basic auth, or add sid parameter into download link.

User avatar
eduo
Posts: 716
Joined: Sat Feb 10, 2007 1:40 am
Location: Information Technology
Contact: ICQ Website Yahoo Messenger

Re: [NEW] REST search, JSON results

Mon Mar 26, 2018 8:38 pm

Nice.

What happens if only bytesize and hash are provided but not IMDBid? Or only IMDBid?

I understand that if I have 50 files (2 tv seasons) I must make 50 queries for each subtitle language. What are the limits (based on IP, I guess) for this method?

What agent should we use? Not sure if you're keeping statistics.
http://eduo.info/
[url=http://eduo.info/soleol/]OpenSubtitles from your desktop: SolEol for Mac/Windows/Linux[/url]
[url=http://forums.plexapp.com/index.php?showtopic=325&st=0&p=2480&#entry2480]My current episode processing work flow[/url].

User avatar
oss
Site Admin
Posts: 5878
Joined: Sat Feb 25, 2006 11:26 pm
Contact: Website

Re: [NEW] REST search, JSON results

Tue Mar 27, 2018 3:50 am

Hi Eduo, long time...

to answer your questions - you can try. Of course you can query just by imdbid:
https://rest.opensubtitles.org/search/i ... uageid-cze

or just moviehash and moviebytesize:
https://rest.opensubtitles.org/search/m ... c54e9cf314

if you have 50 files, you have to make 50 queries (by moviehash for example), yes. If you want 1 language not a problem, if you want all languages not a problem, but if you want 2 languages, you have to make 2 queries per file - it is done for sake of caching on CF (better hit ratio), but more requests...I know.

there is not limits, no UA needed just speed limit: 40 queries per IP per 10 seconds. We not making any stats for now.

This should help to avoid LogIn() in XMLRPC and caching better in CF.

But you are right, maybe we should request setting HTTP UA to actually know what is going on.

EDIT: I added UA checking, it makes sense.

noembryo
Posts: 79
Joined: Thu Nov 16, 2017 12:07 am

Re: [NEW] REST search, JSON results

Tue Mar 27, 2018 2:28 pm

Can you please post an example of an http header with a fake XML-RPC useragent in it?
Do you mean instead of "Mozilla/5.0 (Windows NT 5.1; rv:14.0) Gecko/20100101 Firefox/24.0.1" or whatever, I should use my app's user agent in the http request?

The rest of the calls look really straightforward (even for me ;o)

User avatar
oss
Site Admin
Posts: 5878
Joined: Sat Feb 25, 2006 11:26 pm
Contact: Website

Re: [NEW] REST search, JSON results

Tue Mar 27, 2018 3:47 pm

hi, yes instead of standard HTTP user agent you must use the user agent, which is registered for xml-rpc. I post example using "curl" command, in every language you define different user agent, for sure you will find it.

noembryo
Posts: 79
Joined: Thu Nov 16, 2017 12:07 am

Re: [NEW] REST search, JSON results

Thu Mar 29, 2018 8:03 pm

I tried the rest API today and it looks very nice.
There are some issues though.
One has to do with the subtitles returned.
Alternating between the XML-RPC and the JSON method does not return the same results for the same file.
JSON returns less some of the times.
With XML-RPC I use only the hash and the size (and the language).
With the JSON I also add the title as query, thinking that it might get me more results.
I tried with the title omitted and got the same results some of the times.
This is strange.

The other thing has to do with the limit of queries (40).
Almost all my friends that use Subber always search for at least 2 languages. English and their native one.
If they drop a Series folder to search, the number of files can easily get more than 20.
One season of a series can be more than 20.
Some series can get more than 200.
It might be more helpful to be able to send an array of languageIDs with the REST and get only these returns.
As it is now, I get all the languages and then I filter the ones I don't need.
This works but is a little wasteful.
Perhaps the limit is a tad low...

darkvinill
Posts: 1
Joined: Fri Mar 30, 2018 10:59 pm

Re: [NEW] REST search, JSON results

Fri Mar 30, 2018 11:01 pm

Should I use JSON instead XML-RPC ?

kioo123
Posts: 6
Joined: Thu Aug 10, 2017 5:48 pm

Re: [NEW] REST search, JSON results

Tue Apr 03, 2018 2:41 pm

JSON API means you can issue a JSON request to server just like other JSON requests.
Because the JSON API is PUBLIC that means you will not need LOGIN credentials, and that is one request less to the OPENSUBTITILES servers.
Also, since the requests are cached, there is faster response too. Keep in mind that your USERAGENT must be registered and approved in OPENSUBTITLES or you will get a response "http useragent is not valid"
To make a simple test to the API, use a REST Client such as in Firefox you can install the AddOn RESTED client:
Open it, then enter the following:

URL: https://rest.opensubtitles.org/search/m ... c54e9cf314

Headers -> Add a New Header and on name field enter: User-Agent and on it value field enter "TemporaryUserAgent" don't forget the quotes.
Remember "TemporaryUserAgent" is for Development only and it has limitation; hence don't use it in production and instead use yours.

Click Submit, and Voilah! Your response will be there in nicely formatted JSON.

Hope this helps

kioo123
Posts: 6
Joined: Thu Aug 10, 2017 5:48 pm

Re: [NEW] REST search, JSON results

Tue Apr 03, 2018 5:22 pm

Hi OpenSubttitles,

I was trying to use the new API in my application written in QT, but unfortunately QT doesn't allow change of User-Agent in the request header. It always defaults to Mozilla/5.0 as User-Agent.
Is it possible to use a custom request header name instead in order to pass this restriction?

User avatar
oss
Site Admin
Posts: 5878
Joined: Sat Feb 25, 2006 11:26 pm
Contact: Website

Re: [NEW] REST search, JSON results

Wed Apr 04, 2018 12:05 am

Hi,

for QT set UA, quick google shows this: https://wiki.qt.io/How_to_set_user_agen ... pplication

it is not solution for your problem?

kioo123
Posts: 6
Joined: Thu Aug 10, 2017 5:48 pm

Re: [NEW] REST search, JSON results

Wed Apr 04, 2018 6:32 am

Hi Oss,

Thank you. This actually worked.
Another method is this http://www.misterion.ru/2012/10/qt-qml-user-agent.html
But i prefer yours instead. :D

samsamsam
Posts: 17
Joined: Sun Aug 16, 2015 3:27 pm

Re: [NEW] REST search, JSON results

Wed Apr 18, 2018 10:47 pm

Hi Oss,

I try this new API:
https://rest.opensubtitles.org/search/i ... /season-1/

But unfortunately, when I try to download subtitles I have problem with reCaptcha v2.
When I try to download for example:
https://dl.opensubtitles.org/en/downloa ... 3226300.gz

the server return:
HTTPError: HTTP Error 429: Unknown HTTP Status

add redirect me to:
http://www.opensubtitles.org/en/captcha ... 3226300.gz
and show reCaptcha v2 checkbox.

I understand why you protect opensubtitles.org with reCaptcha v2 and I am not requesting to disable it , but with currect setting of the reCaptcha v2 my application is not able to handle it.
I asked you in this thread
https://forum.opensubtitles.org/viewtop ... &start=180
to provide possibility to solve reCaptcha v2 without javascript. So, my application will be able to show reCaptcha v2 task to the end user for solving it.

Could you please check this?

Thank you,
SamSamSam

samsamsam
Posts: 17
Joined: Sun Aug 16, 2015 3:27 pm

Re: [NEW] REST search, JSON results

Sat Apr 21, 2018 5:14 pm

Any answer please?

There is some hole in the re-captcha v2 protection on this site, so I can use it to workaroud these problems.
But I don't want to do this, because I have respect for opensubtitles.org idea. I hope that you can take a look on the problem with reCaptcha v2 and change it settings.

User avatar
oss
Site Admin
Posts: 5878
Joined: Sat Feb 25, 2006 11:26 pm
Contact: Website

Re: [NEW] REST search, JSON results

Mon Apr 23, 2018 8:01 am

429 http code is meant "too many requests", so you have to calm down.

Your IP can be in captcha mode, so in private message you can send me what are you doing exactly, send me urls where are you getting captcha and also your IP, so I can check

The problem with REST is that you are unauthenticated.

samsamsam
Posts: 17
Joined: Sun Aug 16, 2015 3:27 pm

Re: [NEW] REST search, JSON results

Mon Apr 23, 2018 9:01 am

For sure I did not made to many requests.

Total numbers of downloaded subtitles files in this month are less than 30.
Your IP can be in captcha mode, so in private message you can send me what are you doing exactly, send me urls where are you getting captcha and also your IP, so I can check
I am not doing anything special. I already wrote when I got reCaptcha v2: when I try download subtitles from link returned via REST API.

The problem is not me. I can simple work around this problem. You will check my IP and remove it from reCaptcha database, but this is not what I am asking. I am a developer. I know that many users have such problems I get report form them.
So, I am not asking about solution for me, but general solution to give possibility to solve reCaptcha without javascript.
.
It can be done from reCaptcha configuration.

Please read reCaptcha documentation:
https://developers.google.com/recaptcha ... pt-enabled

Here you have a quote:
Does reCAPTCHA support users that don't have JavaScript enabled?

reCAPTCHA can only provide the optimal experience in terms of security and usability with JavaScript enabled. However, if supporting users who have disabled JavaScript is important for your site, you can enable the alternative challenge with the following steps. Navigate to the admin console and move the security preference slider to "easiest for users". Keep in mind that with this setting reCAPTCHA won't be able to use all of its security features.
My plugin is able to handle reCaptcha task in fallback mode and present reCaptcha task to end user to select valid pictures, but the reCaptcha widget must be configured to allow fallback mode.

Return to “Developing”

Who is online

Users browsing this forum: No registered users and 23 guests