Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mv es-blockly-games to iiab rsync storage in order to rename #522

Merged
merged 9 commits into from
Dec 29, 2022
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/1.Overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,3 +33,7 @@ The code can be found in the git repo https://github.com/iiab/iiab-admin-console
Globally shared menu definitions are in https://github.com/iiab-share/js-menu-files.

Catalog information is in https://github.com/iiab-share/iiab-content.

## Document Version

This document was last maintaineed on December 29, 2022.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This document was last maintaineed on December 29, 2022.
This document was last maintained on December 29, 2022.

4 changes: 4 additions & 0 deletions docs/2.CMDSRV.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,3 +100,7 @@ When running as a service CMDSRV traps all error, and this can mask problems tha
This will start CMDSRV from the command line and allow python errors not to be trapped.

iiab-cmdsrv-ctl 'COMMAND {"key": "value"}' can be used to test a particular Command. (Note use of single quotes to make the Command and parameters a single argument.)

## Document Version

This document was last maintaineed on December 29, 2022.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This document was last maintaineed on December 29, 2022.
This document was last maintained on December 29, 2022.

6 changes: 5 additions & 1 deletion docs/3.Console.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,8 @@ Wiki content comes in a growing list of languages. In order to translate languag

# Troubleshooting

The best way to see under the hood is to open the Javascript Console in your browser. Many functions write status to the console that can be examined.
The best way to see under the hood is to open the Javascript Console in your browser. Many functions write status to the console that can be examined.

## Document Version

This document was last maintaineed on December 29, 2022.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This document was last maintaineed on December 29, 2022.
This document was last maintained on December 29, 2022.

4 changes: 4 additions & 0 deletions docs/4.JS-Menu.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,7 @@ The second icon makes the output less verbose. This is aimed at users who have b
The third icon leads to a feedback form which will store comments in a local database.

Choices made in the first two cases are particular to the user, or at least the device on which the user is operating.

## Document Version

This document was last maintaineed on December 29, 2022.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This document was last maintaineed on December 29, 2022.
This document was last maintained on December 29, 2022.

79 changes: 79 additions & 0 deletions docs/5.Content.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Content

While IIAB has a number of interactive applications, most of the value of the server derives from the content installed there. Much of this content can be installed through a GUI in the Admin Console.

## Kiwix ZIMS

ZIMS are a compressed form of content published by the WikiMedia Foundation, such as the Wikipedia. There are thousands of these across hundreds of languages and dozens of types of subject matter, and Kiwix attempts to publish a compressed version on a monthly basis.

(Some of the larger ZIMS may not be published every month.)

These ZIMS are deployed in /library/zims/content/ and are locally described by the metadata in /library/zims/library.xml. (Only installed ZIMs are included in this file.)

### Catalogs

The current catalog is https://library.kiwix.org/catalog/root.xml, which is downloaded by /opt/admin/cmdsrv/scripts/get_kiwix_catalog

(The previous catalog was at https://ftp.nluug.nl/pub/kiwix/library/library_zim.xml, which is obsolescent and not used by IIAB.)

get_kiwix_catalog downloads the catalog from Kiwix, parses it, and merges a small catalog of supplementary ZIMS maintained and hosted by IIAB.

The IIAB ZIM catalog is at https://raw.githubusercontent.com/iiab-share/iiab-content/main/catalogs/iiab-zim-cat.json.

This merged catalog is used to display the list of available ZIMS by language in the Admin Console under Install Content -> Get ZIM Files from Kiwix.

The url attribute in the catalog tells CMDSRV where to get the file for download, either directly from Kiwix or from storage maintained by IIAB volunteers. It is always a single file.

### DiY

Creating your own ZIM is not trivial, but Kiwix provides tools such as zimwriterfs and mwoffliner for creating ZIMs. Google these for more information.

https://github.com/iiab-share/iiab-content/tree/main/catalogs contains a subdirectory with a json file for each custom ZIM.

https://github.com/iiab-share/iiab-content/blob/main/catalogs/mk-zim-cat-item.py can be used to create a catalog item json file, which can then be further edited by hand.

https://github.com/iiab-share/iiab-content/blob/main/catalogs/mk-zim-cat.py rolls these catalog items up into the IIAB ZIM catalog.

If you create a custom ZIM you should also create a corresponding Menu Item Definition in Admin Console.

## OER2Go/RACHEL Modules

Modules from OER2Go/RACHEL are deployed in /library/www/html/modules. While the majority are from OER2Go/RACHEL, others have been supplied by IIAB volunteers.

### Catalogs

RACHEL has discontinued the catalog at http://dev.worldpossible.org/cgi/json_api_v1.pl in favor of http://oer2go.org/cgi/json_api_v2.pl. But the latter is missing attributes used by the Admin Console, so we use an archived copy.

IIAB volunteers also maintain supplementary modules under a catalog at https://github.com/iiab-share/iiab-content/blob/main/catalogs/iiab-module-cat.json.

### Module Downloads

The Admin Console presents a GUI based on the combined OER2Go and IIAB catalogs under Install Content -> Get OER2Go(RACHEL) Modules that allows the user to select a given module and cause it to be downloaded.

Two means of download are supported, rsync and rclone. rclone is used for S3 storage, and rsync requires an rsync server.

If a catalog item has a value for rclone, that url is used for the download; otherwise it must have a value for the rsync attribute, which will be used by CMDSRV for the download.

There are currently several cloud storage locations maintained by IIAB volunteers from which modules can be downloaded.

Note that whereas ZIMS are a single file, modules are a directory of files that are downloaded individually. This is done instead of placing the entire directory in a compressed file in order to save file space during the download.

### DiY

Any directory of files that can be rendered by a web server to be viewed with a browser can serve as a module. Note that the directory should contain all files required, and all links should be relative to that directory.

Modules, located in /library/www/html/modules, are expected to have a catalog entry and be installable by all users. A directory of custom content that is not shared is expected to be deployed in the /library/www/html/local_content directory and need not have a catalog entry.

To create a custom module place the required files in a directory under /library/www/html/modules. The module should have a name like ll-nnn where ll is a two character language code and nnn is a meaningful name.

Then create a catalog fragment for this module using https://github.com/iiab-share/iiab-content/blob/main/catalogs/mk-mod-cat-item.py. It will be necessary to manually edit the resulting json file as not all fields can be automated.

The module should have a unique module_id. By convention we have been using IIAB1XX where 1XX is the next 3 digit number after the last item in the catalog.

Manually add the catalog fragment to https://github.com/iiab-share/iiab-content/blob/main/catalogs/iiab-module-cat.json.

As was the case with custom ZIMS you will need to create a Menu Item Definition for your modules.

## Document Version

This document was last maintaineed on December 29, 2022.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This document was last maintaineed on December 29, 2022.
This document was last maintained on December 29, 2022.

25 changes: 1 addition & 24 deletions roles/cmdsrv/files/json/oer2go_catalog.json
Original file line number Diff line number Diff line change
Expand Up @@ -1644,7 +1644,7 @@
"prereq_id": null,
"prereq_note": "",
"rating": "4.0",
"rsync_url": "rsync://dev.worldpossible.org/rachelmods/es-blockly-games",
"rsync_url": "rsync://medbox.iiab.me/modules/es-blockly-games",
"source_url": "https://github.com/google/blockly-games/wiki/Offline",
"title": "Blockly Games",
"type": "html",
Expand Down Expand Up @@ -2733,29 +2733,6 @@
"zip_ftp_url": "ftp://dev.worldpossible.org/zipped-modules/fr-blockly-games.zip",
"zip_http_url": "http://dev.worldpossible.org/zipped-modules/fr-blockly-games.zip"
},
"es-blockly_games": {
"age_range": null,
"category": null,
"description": "delete me someday - this module has been superceded, but is here for people still updating against the old module (after they do, they'll automatically get the new one next time)",
"file_count": "0",
"index_mod_sample_url": null,
"is_hidden": "No",
"ksize": "4416",
"lang": "es",
"logo_url": null,
"moddir": "es-blockly_games",
"module_id": "129",
"prereq_id": null,
"prereq_note": "",
"rating": "0.0",
"rsync_url": "rsync://dev.worldpossible.org/rachelmods/es-blockly_games",
"source_url": "",
"title": "Old Blockly Games",
"type": "html",
"version": "v0.1",
"zip_ftp_url": "ftp://dev.worldpossible.org/zipped-modules/es-blockly_games.zip",
"zip_http_url": "http://dev.worldpossible.org/zipped-modules/es-blockly_games.zip"
},
"id-storybooks": {
"age_range": "children",
"category": "educational,language",
Expand Down