Forum Thread

PSA: How Kodi's infamous streaming add-ons actually work (and what's happening when they don't work)

497 1,443 December 31, 2017 at 06:42 PM
If you read nothing else from this post, please just read the first section, below. It explains 90% of what you need to know with a simple example that doesn't even use Kodi, and anyone can try via most any web browser.

Sections covered in this tutorial/rant:
  • You can be your own Covenant/etc "add-on"
  • When things don't go perfectly
  • Index Sites are easy but Host Sites are much, much harder
  • Ahem... I heard Real-Debrid/etc could magically give you more links

You can be your own Covenant/etc "add-on"...
As mentioned above, Kodi+Covenant isn't doing anything you can't do yourself with just a web browser ... which is because almost all of the most "infamous" streaming addons do little more than background-visit a few key websites and then show you the resulting video.

For this example, let's say you want to watch s44e21 of the PBS series Nova. You would start with something as absurdly simple as googling for "watch nova online free". Among the results would probably be a few websites familiar to many of us: CouchTuner, 123Hulu, WatchSeries, etc. These websites privately curate their own highly-prized lists of links to video files available via OTHER websites, and organize them into a searchable index.

For the purposes of this example, somewhere in those google results would be a link to the site SeriesFree. You could pull up that site on any web browser and navigate to the show, season, and episode and be presented with a variety of links to other websites that actually host the video files. These host sites allow a user to view a video file embedded right there in a webpage. ...and that's it. Boom, you're done, as simple as this image depicts:

kodi-streaming-example.png [slickdealscdn.com]


In this example, SeriesFree only had a few links, and both of the ones I tried were both still valid, but that is certainly not always the case. Host Sites will often remove links due to DMCA threats/etc. Similarly many Index Sites will only choose to a few of the most popular/recent shows, sometimes leaving a user to try many Index Sites & Hosts before landing on a good video stream. Note that usually, both the Index Site and the Host Sites (SeriesFree and OpenLoad/Vidzi in this example) usually have ads and popups absolutely everywhere -- which makes sense because that's how they make money. FYI, visiting on mobile platforms is usually a lot less messy than via a desktop web browser.

This is pretty much all that a "universal" streaming addon (like Covenant) does, inside Kodi. Rather than googling for "watch XYZ online free", streaming addons usually have their own hard-coded list of dependable-ish Index Sites that will include the likes of CouchTuner, 123Hulu, WatchSeries, etc. And instead of having to manually visit many Index Sites to see if it has links to your show, the addon does it for you and compiles a big master list, all in the background.
When things don't go perfectly...
Covenant (like other streaming addons you might have used in Kodi) doesn't have your eyes and years of skill at avoiding deceptive ads, popups, redirects, and other shady tricks that websites employ to get more clicks and make more money... so some kind soul had to write instructions, in the form of python code, to tell a computer exactly how to get the indexed links and video URLs off of any particular website. Here's a screenshot of just some of the ones that were used by the now-defunct (but still very good) Covenant addon:

covenant-index-list.png [slickdealscdn.com]


Covenant is a good example because it's currently in the early stages of deterioration after having been abandoned due to legal scares following the absurdity around the Zem lawsuits [koditips.com]. Many of its original Index Sites will still produce links to add to the master list for any particular search, but for various reasons, many others no longer can. Just taking a quick glance inside these python scripts can often tell you why:

covenant-indexes-dead-and-dying.png [slickdealscdn.com]

......A moment of silence for those Index Sites we have lost..........
Index Sites are easy but Host Sites are much, much harder...
For the purposes of discussion, please keep in mind that you probably have some familiarity with many Host Sites, already. Their ranks include RapidGator, Uptobox, Vidme, Openload, Vidzi and many other sites that are often referred to as "cyber lockers [wikipedia.org]".

Although an addon could just as easily include its own python scripts for both Index and Host sites, most of them only include instructions for Kodi to sort through Index Sites. Host sites really, really don't want to make it easy for you to get at their videos directly: hosting all that data and using all that bandwidth isn't cheap, and they would prefer you accidentally clicked on a ton of ads first!! These sites will regularly make changes to their sites to trip up anyone who has a dependable way of getting to the juicy juicy video links... which is really their only defense since they ...probably... don't have permission to host these files so they can't really complain to anyone. Luckily, the Kodi-addon community has a couple 800-pound code-unscrambling gorillas on its side by the names of NaNScrapers and URLResolver. These "dependencies" are used by most of the "universal" streaming addons and are maintained by a small army of volunteer coders... so even though Covenant is no longer under development, it still mostly works because it delegates most of the hard work to these gorillas. They work in a fashion very similar to that already demonstrated for Index Sites, but you should know they are pretty different, separate functions.

This is very important to keep in mind, if for no other reason than to understand that there are multiple points of failure when using a streaming addon: Index Sites can change and/or go away ... and same for Host Sites. I'd like to think most properly motivated slickdealers now have the knowledge to look a little deeper into why their particular usage case isn't working well themselves, rather than helplessly post on forums.
Ahem... I heard Real-Debrid/etc could magically give you more links...
That's not how it works. Don't expect miracles. And don't craptalk a service for not working the way you want it to, if that's not actually how it works.

I recently posted a deal about a subscriptions sale that RD was having, and got a little itchy reading through the comments. I wrote this whole post so that a reader would have the appropriate background to understand what "debrid" services actually do, since many MANY people seem to look at them like the blackest of magics. Real-debrid and similar sites adjust the video stream links for some, but not all, Host Sites. Why some? Well you probably have, at least once, tried to download a file from a cyberlocker type Host and been greeted with two buttons: one for "SLOOOOWW" download, and one for "FAST!!!" ... the FAST! link is reserved for paying members of that website.

When you pay for premium membership to Real-debrid/etc, you're getting access to the fast lane of MANY cyberlockers all at once. (RD lists their supported hosts here [real-debrid.com].) I don't pretend to know anything about the business side of how they arrange this, but it's pretty neat and works well. A user can copy/paste a URL for a desired file into a box/form on the RD website, and behind the scenes, RD would began a fast-lane download on your behalf. I'm not sure about other sites, but all RD downloads come from RD servers, rather than those of the individual cyberlockers.

Hopefully you see now why RD will not magically create streams out of thin air -- they have to exist somewhere first. ...And that 'somewhere' needs to be a Host Site that offers premium service. ...And that Host Site needs to be one that is supported by RD/etc. Many cyberlocker Host Sites will ONLY offer premium downloads, and that is the one single way you will see *new* sources in your streaming addon, rather than just faster access to many of them. ... it's not an all-powerful downloading panacea but it's a definite improvement!!!

covenant-RD-fast-lane.png [slickdealscdn.com]


Bringing it back to the start of this post, every user has the power to check if Real-debrid is working or not... it's quite simple to look at Covenant's results and pull up the video in question with a web browser. It's just as simple to copy/paste that link into Real-debrid's website and get the debrid-ized link for direct download and skip Kodi altogether. ...and that is exactly what you should do before blaming RD for not doing something you think it should be doing.
edit: 3 of the 4 images I originally attached to this post are inexplicably gone... I'm re-attaching and re inline-tagging them now but ... who knows if they'll stay?

6 Comments

1

Sign up for a Slickdeals account to remove this ad.

This comment has been rated as unhelpful by Slickdeals users
Joined Nov 2010
L8: Grand Teacher
3,007 Posts
1,773 Reputation
#2
Interesting, I rarely use KODI, but when I recently tried. Covenant didn't really work for new streams, Exodus has been dead, and I need to find a new add-on. Repped and will follow this thread to learn something!

Thanks OP!
Reply Helpful Comment? 0 0
This comment has been rated as unhelpful by Slickdeals users
Joined Nov 2014
L3: Novice
529 Posts
352 Reputation
#3
Very informative,I always wondered how the addons found the links.
Reply Helpful Comment? 0 0
This comment has been rated as unhelpful by Slickdeals users
Joined Sep 2007
Deductive Reasoner, Sr.
497 Posts
1,443 Reputation
Original Poster
Pro
#4
Note: hopefully you guys/gals who've already read the first post saw it with the intended images embedded inline so it was less of a wall of text!! But 3 of the 4 images I originally attached to it were inexplicably gone when pulled it up this morning... I'm re-attaching and re inline-tagging them now but ... who knows if they'll stay?

Thanks, guys! Glad to know there are others curious about Kodi/etc's inner-workings!
Quote from trtem
:
Very informative,I always wondered how the addons found the links.
Quote from aznboicn
:
Interesting, I rarely use KODI, but when I recently tried. Covenant didn't really work for new streams, Exodus has been dead, and I need to find a new add-on. Repped and will follow this thread to learn something! Thanks OP!
@aznboicn, I got ya, fam. I spent a while this morning cobbling together a bunch of historical and current-events info on streaming addons. In doing so, I found this aaaaaawesome family tree:
https://i.redd.it/ria7f4ds86701.png



With that as a nice visual reference, let's discuss a little bit of the key players because some of the information is directly related to the most current developers and addons...

A brief recap of the Genesis and SALTS families and developers
Genesis (by Lambda) [ref 1]
Genesis was originally developed by Lambda and was the first popular multi-source Kodi addon.
Exodus (still by Lambda) [ref 1]
* Forked directly from Genesis by (most of) the original devs.
Libraries for unscrambling Provider websites (Host Sites) were moved to a central dependency called URLResolver. This meant that resolvers could be updated in URLResolver and would work in all addons using this dependency. In early 2017, lambda retired from the Kodi addon community and Exodus support was taken over by a collection of developer.
Covenant (by jsergio/Colossus repo) [ref 1]
* Forked directly from Exodus by (most of) the original devs.
The new developers who had taken Exodus over decided to rebrand the addon as Covenant to respect the old legacy and moved the addon into the new Colossus Repo. Despite constant reports to the contrary, Covenant still functions great, aside from the limitations stated previously: Host Site sources not being updated. BUT! ...Other heroic users are independently updating the python scripts themselves and making the updates available to the savvy, as exemplified here: https://www.reddit.com/r/Addons4K...ant_fixes/. Similarly, others are repackaging the updates and most of the original Covenant code into new forks like Incursion and Placenta (see below).
Zen, Elysium, and Bob addons [ref 2]
* Zen was an Exodus fork
* Elysium was directly forked from the Zen addon by (most of) the original devs.
* Bob and Elysium apparently share(d) (some of) the same devs.
The Bob addon was the first, or one of the first, addons to use the NanScrapers library -- which has much of the same power as its cousin, URLResolver. Elysium originally used URLResolver, but later moved to use NaNScrapers instead. (I assume for simplicity between sharing developers.) NaNScrapers ended up being the most important/lasting legacy of this family.
SALTS (Stream all the sources; by tknorris/K3l3vra) [ref 3]
Originally developed by tknorris and later by k3l3vra. In its heyday, SALTS was almost as popular/powerful as Exodus. Like Exodus, it too used URLResolver to unscramble Host Sites.
Death Streams (by Mr Blamo) [ref 4]
* Forked from SALTS
Uses URLResolver, still. My experience with SALTS, and continuing into Death Streams, is that there is some kind of annoying glitch in the search routine which often results in the first run of a search producing zero streams -- even though a second run will then produce many great streams. (!!?!?)
Neptune (by Mr Blamo) [ref 5]
* Forked from the Poseidon addon, which was itself a fork of the old Exodus addon.
The addon uses NaNScrapers as its dependency source rather than URLResolver as Exodus did. My personal experience is that (at least the current version) is (sometimes?) noticeably slower just to navigate its own menu system, almost like there's an un-found bug causing some code to needlessly loop. That having been said, on almost every test I through at it, it produced many more results than Covenant did. (Both using premium RD account.) But the slowness (or at this point, it might just be my own perception of slowness?) has led me to avoid it.
Placenta (by "Team Afterbirth" esp Mr Blamo & Maud-Dib) [ref 6]
* Forked from Covenant (probably) or Exodus (maybe)
Currently uses URLResolver for Host Site unscrambling, but given Mr Blamo's other projects, and the fact NaNScrapers seems to be more actively maintained lately, I wouldn't be surprised if it ends up using NanScrapers instead. Update: I just tested this addon and in all the best ways, it feels like a carbon copy of Covenant. Rejoice!
Incursion (by NixGates) [ref 7]
* Forked from Covenant
Not sure if this developer has any history with other projects, or support from other devs. It doesn't even have an entry (yet) on KodiTips. This fixes a lot of the sources that were present in Covenant and adds some new ones. Keep in mind this is a brand new add-on spun off of unfamiliar (to the dev) code, so will probably take a few iterations to reach the same level of polish while unexpected legacy code is being found and fixed. (Assuming the addon is more than vaporware/stopgapware.)
References:


Try them yourself -- install instructions
So in sort of a conclusion of the above:​ Covenant (still!) most of the time, but failing-over to Death Streams, then Neptune. I haven't yet tried Incursion... but based on the dev legacy of Placenta, I dare suspect it might be the "new Covenant".

These all can of course be installed via each one's home-repository, but figuring out, and subsequently installing each repo can get kinda annoying if you just want to do some quick testing. If you know all the modules you'll need ahead of time, it's often easier to install manually via the repo-agnostic method of just collecting and installing module zips. I'm going to leave myself some install instructions below for the future... feel free to follow along!

Instructions: on your Kodi-running device, download all of the .zip files listed below under the plugin you want. (Note they range from 0.1 to 14MB. If you're on a Firestick, use the Downloader app [amazon.com] to download them.) Once they are all fully downloaded, install them in the order listed by using "the box icon" and sequentially picking each one with "Install from zip file".

Placenta (by Team Afterbirth)
Death Streams (by Mr Blamo)
Neptune Rising (by Mr Blamo)
**A few of the modules above have additional dependencies BUT they are automatically handled by the official "Kodi Add-on repository" which is pre-installed by default on all new Kodi installations I'm aware of.

Reply Helpful Comment? 0 0
Last edited by jasonbuechler January 3, 2018 at 12:24 PM.
This comment has been rated as unhelpful by Slickdeals users
Joined Nov 2014
L3: Novice
529 Posts
352 Reputation
#5
Good stuff there Jason. Back in the day before it went down,my next favorite after Exodus was Phoenix,there was a lot of good stuff in there.
I think some of the developers you mentioned were associated with it.
It wasn't around long but Bennu was a nice addon that was similar to Phoenix.
Reply Helpful Comment? 0 0
This comment has been rated as unhelpful by Slickdeals users
Joined Sep 2007
Deductive Reasoner, Sr.
497 Posts
1,443 Reputation
Original Poster
Pro
#6
Quote from trtem
:
Good stuff there Jason. Back in the day before it went down,my next favorite after Exodus was Phoenix,there was a lot of good stuff in there.
I think some of the developers you mentioned were associated with it.
It wasn't around long but Bennu was a nice addon that was similar to Phoenix.
You used Phoenix/Bennu?? (Bennu was a direct fork by the orig team.)
I *tried* to use it to see what it was all about but I just didn't "get" it. Was it supposed to be a "multi-source" all-purpose media-finder, or something else (that I couldn't quite figure out)? What would be one's usage case for heading to Bennu over other addons? I'm genuinely curious to learn about these other addons I mighta been missing out on!


update: lol way to make myself look dumb. I installed it locally and didn't immediately try opening it since I knew from experience I didn't know "how" to efficiently use it... but I did just open it and discovered that its main listing must have come directly from culling a pastebin entry created by the author (https://pastebin.com/raw/rMSzEpaZ), making the addon pointless. Sorry for the tease!

So, out of curiosity I looked at some info inside Bennu and it uses almost all of the guts of Covenant, so if you still want to use Bennu, there's a good chance it still works pretty well** -- at least as well as my Covenant still does. If you're not sure how to install it since its repo is gone, here are the modules you'll need, in order:**esp if you use a premium account, and/or if you followup your Bennu installation with one of the unofficial "updated" covenant installations I referenced above -- because that updated version will update the guts that bennu depends on.
Reply Helpful Comment? 0 0
Last edited by jasonbuechler January 4, 2018 at 11:36 AM.
This comment has been rated as unhelpful by Slickdeals users
Joined Nov 2014
L3: Novice
529 Posts
352 Reputation
#7
Quote from jasonbuechler
:
You used Phoenix/Bennu?? (Bennu was a direct fork by the orig team.)
I *tried* to use it to see what it was all about but I just didn't "get" it. Was it supposed to be a "multi-source" all-purpose media-finder, or something else (that I couldn't quite figure out)? What would be one's usage case for heading to Bennu over other addons? I'm genuinely curious to learn about these other addons I mighta been missing out on!


update: lol way to make myself look dumb. I installed it locally and didn't immediately try opening it since I knew from experience I didn't know "how" to efficiently use it... but I did just open it and discovered that its main listing must have come directly from culling a pastebin entry created by the author (https://pastebin.com/raw/rMSzEpaZ), making the addon pointless. Sorry for the tease!

So, out of curiosity I looked at some info inside Bennu and it uses almost all of the guts of Covenant, so if you still want to use Bennu, there's a good chance it still works pretty well** -- at least as well as my Covenant still does. If you're not sure how to install it since its repo is gone, here are the modules you'll need, in order:**esp if you use a premium account, and/or if you followup your Bennu installation with one of the unofficial "updated" covenant installations I referenced above -- because that updated version will update the guts that bennu depends on.
Thanx for the tip on getting Bennu to work but I only watch a couple shows a week on Kodi right now and Covenant with Real Debrid is still working OK for me. When that eventually stops working I'll probably just do a clear data and install whatever is the current best option at the time,since these seems to change week to week.
That being said Phoenix was hard to explain. It had a movies and tv section but it also had individual developers listed with their own addon links with their movie sections,playlists etc. Kind of like addons within an addon. Had a good bit of International type content too.
Reply Helpful Comment? 0 0
Page 1 of 1
1
Join the Conversation
Add a Comment
 
Copyright 1999 - 2018. Slickdeals, LLC. All Rights Reserved. Copyright / Infringement Policy  •  Privacy Policy  •  Terms of Service  •  Acceptable Use Policy (Rules)  •  Interest-Based Ads
Link Copied to Clipboard