Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sync with upstream v0.0.10 #3

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Sync with upstream v0.0.10 #3

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
.github export-ignore
.gitignore export-ignore
.gitattributes export-ignore
requirements.txt export-ignore
setup.cfg export-ignore
standalone.py export-ignore
45 changes: 25 additions & 20 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,34 +2,39 @@

This repository is a fork of a [i96751414's](https://github.com/i96751414) https://github.com/i96751414/repository.github project with minor changes. Many thanks for this project.

This add-on creates a virtual repository for Kodi. This way, one does not need to use a GitHub repository for storing add-ons zips when all that information is already accessible from each add-on repository.
This add-on creates a virtual repository for Kodi. This way, one does not need to use a GitHub repository for storing
add-ons zips when all that information is already accessible from each add-on repository.

It works by setting a HTTP server which has the following endpoints:
It works by setting an HTTP server which has the following endpoints:

|Endpoint|Description|
|--------|-----------|
|http://127.0.0.1:{port}/addons.xml|Main xml file containing all add-ons information|
|http://127.0.0.1:{port}/addons.xml.md5|Checksum of the main xml file|
|http://127.0.0.1:{port}/{addon_id}/{asset_path}|Endpoint for serving add-ons assets/zips|
|http://127.0.0.1:{port}/update|Endpoint for updating repository entries and clearing caches|
| Endpoint | Description |
|-------------------------------------------------|--------------------------------------------------------------|
| http://127.0.0.1:{port}/addons.xml | Main xml file containing all add-ons information |
| http://127.0.0.1:{port}/addons.xml.md5 | Checksum of the main xml file |
| http://127.0.0.1:{port}/{addon_id}/{asset_path} | Endpoint for serving add-ons assets/zips |
| http://127.0.0.1:{port}/update | Endpoint for updating repository entries and clearing caches |

## Installation

Get the [latest release](https://github.com/elementumorg/repository.elementumorg/releases/latest) from github.
Then, [install from zip](https://kodi.wiki/view/Add-on_manager#How_to_install_from_a_ZIP_file) within [Kodi](https://kodi.tv/).
Get the [latest release](https://github.com/elementumorg/repository.elementumorg/releases/latest) from GitHub.
Then, [install from zip](https://kodi.wiki/view/Add-on_manager#How_to_install_from_a_ZIP_file)
within [Kodi](https://kodi.tv/).

## Add-on entries

In order to know which repositories to proxy, the virtual repository needs to be provided with a _list of add-on entries_.
In order to know which repositories to proxy, the virtual repository needs to be provided with a _list of add-on
entries_.
An example can be found [here](resources/repository.json).
Each entry must follow the following schema (also available as [json schema](resources/repository-schema.json)):

|Property|Required|Description|
|--------|--------|-----------|
|id|true|Add-on id.|
|username|true|GitHub repository username.|
|branch|false|GitHub repository branch. If not defined, it will be the commit of the latest release or, in case there are no releases, master branch.|
|assets|false|Dictionary containing string/string key-value pairs, where the key corresponds to the relative asset location and the value corresponds to the real asset location. One can also set "zip" asset, which is a special case for the add-on zip. If an asset is not defined, its location will be automatically evaluated.<br>Note: assets are treated as "new style" format strings with the following keywords - _id_, _username_, _repository_, _branch_.|
|asset_prefix|false|Prefix to use on the real asset location when it is automatically evaluated.|
|repository|false|GitHub repository name. If not set, it is assumed to be the same as the add-on id.|
|platforms|false|Platforms where the add-on is supported. If not set, it is assumed all platforms are supported.|
| Property | Required | Description |
|--------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| id | true | Add-on id. |
| username | true | GitHub repository username. |
| branch | false | The github repository branch. If not defined, it will be either <br>1) the commit of the latest release, <br>2) the respective tag, <br>3) the repository default branch, or <br>4) if all the previous are unable to fetch, "main" branch. |
| assets | false | Dictionary containing string/string key-value pairs, where the key corresponds to the relative asset location and the value corresponds to the real asset location. One can also set "zip" asset, which is a special case for the add-on zip. If an asset is not defined, its location will be automatically evaluated.<br><br>Note: assets are treated as "new style" format strings with the following keywords - _id_, _username_, _repository_, _ref_, _system_, _arch_ and _version_ (_version_ is available for zip assets only). |
| asset_prefix | false | Prefix to use on the real asset location when it is automatically evaluated. |
| repository | false | GitHub repository name. If not set, it is assumed to be the same as the add-on id. |
| tag_pattern | false | The pattern for matching eligible tags. If not set, all tags are considered. |
| token | false | The token to use for accessing the repository. If not provided, the repository must have public access. |
| platforms | false | Platforms where the add-on is supported. If not set, it is assumed all platforms are supported. |
10 changes: 8 additions & 2 deletions addon.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
<?xml version="1.0" encoding="UTF-8"?>
<addon id="repository.elementumorg" name="ElementumOrg repository" provider-name="elementumorg" version="0.0.2">
<addon id="repository.elementumorg" name="ElementumOrg repository" provider-name="elementumorg" version="0.0.3">
<requires>
</requires>
<extension point="xbmc.addon.repository">
Expand All @@ -18,9 +18,15 @@
<summary lang="en">GitHub virtual Add-on repository</summary>
<description lang="en">Customizable repository which acts as a proxy for defined GitHub users' add-ons updates.</description>
<news>
- Allow running in standalone mode
- Migrate to GitHub API
- Add support for "tag_pattern" field
- Add support for private repositories by supplying a "token"
- Try to automatically determine zip ref from version
- Multiple minor improvements
</news>
<assets>
<icon>icon.png</icon>
</assets>
</extension>
</addon>
</addon>
130 changes: 78 additions & 52 deletions lib/cache.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,44 @@
import time
from functools import wraps
from operator import attrgetter
from threading import Lock


class _HashedSeq(list):
__slots__ = 'hash_value'
class _CacheValue(object):
__slots__ = ["_value", "_modified"]

# noinspection PyMissingConstructor
def __init__(self, tup):
self[:] = tup
self.hash_value = hash(tup)
def __init__(self, value):
self._value = value
self._modified = time.time()

@property
def modified(self):
return self._modified

@property
def value(self):
return self._value

def expired(self, ttl):
return time.time() - self._modified > ttl

def update(self):
self._modified = time.time()


class _HashedTuple(tuple):
__hash_value = None

def __hash__(self):
return self.hash_value
hash_value = self.__hash_value
if hash_value is None:
self.__hash_value = hash_value = super(_HashedTuple, self).__hash__()
return hash_value

def __getstate__(self):
return {}


def _make_key(args, kwds, typed, kwd_mark=(object(),), fast_types=(int, str)):
def _make_key(args, kwargs, typed, kwd_mark=(_HashedTuple,), fast_types=(int, str)):
"""
Make a cache key from optionally typed positional and keyword arguments

Expand All @@ -26,50 +50,52 @@ def _make_key(args, kwds, typed, kwd_mark=(object(),), fast_types=(int, str)):
saves space and improves lookup speed.
"""
key = args
if kwds:
key += kwd_mark
for item in kwds.items():
key += item
sorted_kwargs = tuple(sorted(kwargs.items()))
if sorted_kwargs:
key += kwd_mark + sorted_kwargs
if typed:
key += tuple(type(v) for v in args)
if kwds:
key += tuple(type(v) for v in kwds.values())
if sorted_kwargs:
key += tuple(type(v) for _, v in sorted_kwargs)
elif len(key) == 1 and type(key[0]) in fast_types:
return key[0]
return _HashedSeq(key)


def cached(seconds=60 * 60, max_size=128, typed=False, lru=False):
def wrapper(func):
cache = {}

if max_size == 0:
# no caching
new_func = func
else:
@wraps(func)
def new_func(*args, **kwargs):
key = _make_key(args, kwargs, typed)
cache_entry = cache.get(key)
if cache_entry is None or time.time() - cache_entry[0] > seconds:
# Check cache size first and clean if necessary
if len(cache) >= max_size:
min_key = min(cache, key=lambda k: cache[k][0])
del cache[min_key]

result = func(*args, **kwargs)
cache[key] = [time.time(), result]
else:
if lru:
cache_entry[0] = time.time()
result = cache_entry[1]

return result

def cache_clear():
cache.clear()

new_func.cache_clear = cache_clear
return new_func

return wrapper
return _HashedTuple(key)


class LoadingCache(object):
__slots__ = ["_func", "_store", "_ttl", "_max_size", "_typed", "_lru", "_get_modifier", "_lock"]

def __init__(self, func, ttl_seconds=60 * 60, max_size=128, typed=False, lru=False):
self._func = func
self._store = {}
self._ttl = ttl_seconds
self._max_size = max_size
self._typed = typed
self._lru = lru
self._get_modifier = attrgetter("modified")
self._lock = Lock()

def get(self, *args, **kwargs):
key = _make_key(args, kwargs, self._typed)
with self._lock:
cache_entry = self._store.get(key) # type: _CacheValue
if cache_entry is None:
# Check cache size first and clean if necessary
if len(self._store) >= self._max_size:
min_key = min(self._store, key=self._get_modifier)
del self._store[min_key]
result = self._func(*args, **kwargs)
self._store[key] = _CacheValue(result)
elif cache_entry.expired(self._ttl):
result = self._func(*args, **kwargs)
self._store[key] = _CacheValue(result)
else:
if self._lru:
cache_entry.update()
result = cache_entry.value

return result

def clear(self):
with self._lock:
self._store.clear()
61 changes: 46 additions & 15 deletions lib/compatibility.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,45 +1,76 @@
import importlib
import os
import re
import sys
from xml.etree import ElementTree # nosec

import xbmc
import xbmcaddon

from lib.version import DebianVersion

PY3 = sys.version_info.major >= 3

_digits_re = re.compile(r"(\d+)")


class UndefinedModuleError(ImportError):
pass


def register_module(name, py2_module=None, py3_module=None):
class InvalidModuleVersionError(ImportError):
pass


def register_module(name, module=None, py2_module=None, py3_module=None, version=None):
try:
importlib.import_module(name)
xbmc.log("{} module is already installed".format(name), xbmc.LOGDEBUG)
except ImportError:
xbmc.log("Failed to import module. Going to register it.", xbmc.LOGDEBUG)
module = py3_module if PY3 else py2_module
module = module or (py3_module if PY3 else py2_module)
if module is None:
raise UndefinedModuleError("No module was defined")
try:
import_module(module)
xbmc.log("{} module was already installed, but missing on addon.xml".format(name), xbmc.LOGDEBUG)
except RuntimeError:
install_and_import_module(name, module)
if has_addon(module):
xbmc.log("{} module is already installed, but missing on addon.xml".format(name), xbmc.LOGDEBUG)
import_module(module, version=version)
else:
install_and_import_module(name, module, version=version)


def import_module(module):
path = xbmcaddon.Addon(module).getAddonInfo("path")
def import_module(module, version=None):
addon = xbmcaddon.Addon(module)
addon_path = addon.getAddonInfo("path")
addon_version = addon.getAddonInfo("version")
if not PY3:
# noinspection PyUnresolvedReferences
path = path.decode("utf-8")
tree = ElementTree.parse(os.path.join(path, "addon.xml"))
addon_path = addon_path.decode("utf-8")
# noinspection PyUnresolvedReferences
addon_version = addon_version.decode("utf-8")
if version is not None and DebianVersion(addon_version) < DebianVersion(version):
raise InvalidModuleVersionError("No valid version for module {}: {} < {}".format(
module, addon_version, version))
tree = ElementTree.parse(os.path.join(addon_path, "addon.xml"))
# Check for dependencies
for dependency in tree.findall("./requires//import"):
dependency_module = dependency.attrib["addon"]
if dependency_module.startswith("script.module."):
xbmc.log("{} module depends on {}. Going to import it.".format(module, dependency_module), xbmc.LOGDEBUG)
import_module(dependency_module, dependency.attrib.get("version"))
# Install the actual module
library_path = tree.find("./extension[@point='xbmc.python.module']").attrib["library"]
sys.path.append(os.path.join(path, library_path))
sys.path.append(os.path.join(addon_path, library_path))


def install_and_import_module(name, module):
def install_and_import_module(name, module, version=None):
xbmc.log("Installing and registering module {}:{}".format(name, module), xbmc.LOGINFO)
xbmc.executebuiltin("InstallAddon(" + module + ")", wait=True)
import_module(module)
install_addon(module)
import_module(module, version=version)


def install_addon(addon):
xbmc.executebuiltin("InstallAddon(" + addon + ")", wait=True)


def has_addon(addon):
return xbmc.getCondVisibility("System.HasAddon(" + addon + ")")
Loading