Merge pull request #522 from tim-moody/0.6.1-misc

mv es-blockly-games to iiab rsync storage in order to rename

docs/1.Overview.md

@@ -33,3 +33,7 @@ The code can be found in the git repo https://github.com/iiab/iiab-admin-console
 Globally shared menu definitions are in https://github.com/iiab-share/js-menu-files.
 
 Catalog information is in https://github.com/iiab-share/iiab-content.
+
+## Document Version
+
+This document was last maintained on December 29, 2022.

docs/2.CMDSRV.md

@@ -100,3 +100,7 @@ When running as a service CMDSRV traps all error, and this can mask problems tha
 This will start CMDSRV from the command line and allow python errors not to be trapped.
 
 iiab-cmdsrv-ctl 'COMMAND {"key": "value"}' can be used to test a particular Command. (Note use of single quotes to make the Command and parameters a single argument.)
+
+## Document Version
+
+This document was last maintained on December 29, 2022.

docs/3.Console.md

@@ -49,4 +49,8 @@ Wiki content comes in a growing list of languages. In order to translate languag
 
 # Troubleshooting
 
-The best way to see under the hood is to open the Javascript Console in your browser. Many functions write status to the console that can be examined.
+The best way to see under the hood is to open the Javascript Console in your browser. Many functions write status to the console that can be examined.
+
+## Document Version
+
+This document was last maintained on December 29, 2022.

docs/4.JS-Menu.md

@@ -32,3 +32,7 @@ The second icon makes the output less verbose. This is aimed at users who have b
 The third icon leads to a feedback form which will store comments in a local database.
 
 Choices made in the first two cases are particular to the user, or at least the device on which the user is operating.
+
+## Document Version
+
+This document was last maintained on December 29, 2022.

docs/5.Content.md

@@ -0,0 +1,83 @@
+# Content
+
+While IIAB has a number of interactive applications, most of the value of the server derives from the content installed there. Much of this content can be installed through a GUI in the Admin Console.
+
+## Kiwix ZIMS
+
+ZIMS are a compressed form of content published by the WikiMedia Foundation, such as the Wikipedia. There are thousands of these across hundreds of languages and dozens of types of subject matter, and Kiwix attempts to publish a compressed version on a monthly basis.
+
+(Some of the larger ZIMS may not be published every month.)
+
+These ZIMS are deployed in /library/zims/content/ and are locally described by the metadata in /library/zims/library.xml. (Only installed ZIMs are included in this file.)
+
+### Catalogs
+
+The current catalog is https://library.kiwix.org/catalog/root.xml, which is downloaded by /opt/admin/cmdsrv/scripts/get_kiwix_catalog
+
+(The previous catalog was at https://ftp.nluug.nl/pub/kiwix/library/library_zim.xml, which is obsolescent and not used by IIAB.)
+
+get_kiwix_catalog downloads the catalog from Kiwix, parses it, and merges a small catalog of supplementary ZIMS maintained and hosted by IIAB.
+
+The IIAB ZIM catalog is at https://raw.githubusercontent.com/iiab-share/iiab-content/main/catalogs/iiab-zim-cat.json.
+
+This merged catalog is used to display the list of available ZIMS by language in the Admin Console under Install Content -> Get ZIM Files from Kiwix.
+
+The url attribute in the catalog tells CMDSRV where to get the file for download, either directly from Kiwix or from storage maintained by IIAB volunteers. It is always a single file.
+
+### DiY
+
+Creating your own ZIM is not trivial, but Kiwix provides tools such as zimwriterfs and mwoffliner for creating ZIMs. Google these for more information.
+
+https://github.com/iiab-share/iiab-content/tree/main/catalogs contains a subdirectory with a json file for each custom ZIM.
+
+https://github.com/iiab-share/iiab-content/blob/main/catalogs/mk-zim-cat-item.py can be used to create a catalog item json file, which can then be further edited by hand.
+
+https://github.com/iiab-share/iiab-content/blob/main/catalogs/mk-zim-cat.py rolls these catalog items up into the IIAB ZIM catalog.
+
+If you create a custom ZIM you should also create a corresponding Menu Item Definition in Admin Console.
+
+## OER2Go/RACHEL Modules
+
+Modules from OER2Go/RACHEL are deployed in /library/www/html/modules. While the majority are from OER2Go/RACHEL, others have been supplied by IIAB volunteers.
+
+### Catalogs
+
+RACHEL has discontinued the catalog at http://dev.worldpossible.org/cgi/json_api_v1.pl in favor of http://oer2go.org/cgi/json_api_v2.pl. But the latter is missing attributes used by the Admin Console, so we use an archived copy.
+
+When Admin Console is installed roles/cmdsrv/files/json/oer2go_catalog.json is copied to /etc/iiab/oer2go_catalog.json.
+
+IIAB volunteers also maintain supplementary modules under a catalog at https://github.com/iiab-share/iiab-content/blob/main/catalogs/iiab-module-cat.json.
+
+These two catalogs are merged when iiab-get-oer2go-cat is run and the merged catalog is stored as /etc/iiab/oer2go_catalog.json. So the oer2go portion of the catalog is refreshed any time there is an install and the iiab portion is refreshed whenever iiab-get-oer2go-cat is run.
+
+### Module Downloads
+
+The Admin Console presents a GUI based on the combined OER2Go and IIAB catalogs under Install Content -> Get OER2Go(RACHEL) Modules that allows the user to select a given module and cause it to be downloaded.
+
+Two means of download are supported, rsync and rclone. rclone is used for S3 storage, and rsync requires an rsync server.
+
+If a catalog item has a value for rclone, that url is used for the download; otherwise it must have a value for the rsync attribute, which will be used by CMDSRV for the download.
+
+There are currently several cloud storage locations maintained by IIAB volunteers from which modules can be downloaded.
+
+Note that whereas ZIMS are a single file, modules are a directory of files that are downloaded individually. This is done instead of placing the entire directory in a compressed file in order to save file space during the download.
+
+### DiY
+
+Any directory of files that can be rendered by a web server to be viewed with a browser can serve as a module. Note that the directory should contain all files required, and all links should be relative to that directory.
+
+Modules, located in /library/www/html/modules, are expected to have a catalog entry and be installable by all users. A directory of custom content that is not shared is expected to be deployed in the /library/www/html/local_content directory and need not have a catalog entry.
+
+To create a custom module place the required files in a directory under /library/www/html/modules. The module should have a name like ll-nnn where ll is a two character language code and nnn is a meaningful name.
+
+Then create a catalog fragment for this module using https://github.com/iiab-share/iiab-content/blob/main/catalogs/mk-mod-cat-item.py. It will be necessary to manually edit the resulting json file as not all fields can be automated.
+
+The module should have a unique module_id. By convention we have been using IIAB1XX where 1XX is the next 3 digit number after the last item in the catalog.
+
+Manually add the catalog fragment to https://github.com/iiab-share/iiab-content/blob/main/catalogs/iiab-module-cat.json.
+
+As was the case with custom ZIMS you will need to create a Menu Item Definition for your modules.
+
+## Document Version
+
+This document was last maintained on December 29, 2022.

roles/cmdsrv/files/json/oer2go_catalog.json

@@ -1644,7 +1644,7 @@
       "prereq_id": null,
       "prereq_note": "",
       "rating": "4.0",
-      "rsync_url": "rsync://dev.worldpossible.org/rachelmods/es-blockly-games",
+      "rsync_url": "rsync://medbox.iiab.me/modules/es-blockly-games",
       "source_url": "https://github.com/google/blockly-games/wiki/Offline",
       "title": "Blockly Games",
       "type": "html",
@@ -2733,29 +2733,6 @@
       "zip_ftp_url": "ftp://dev.worldpossible.org/zipped-modules/fr-blockly-games.zip",
       "zip_http_url": "http://dev.worldpossible.org/zipped-modules/fr-blockly-games.zip"
     },
-    "es-blockly_games": {
-      "age_range": null,
-      "category": null,
-      "description": "delete me someday - this module has been superceded, but is here for people still updating against the old module (after they do, they'll automatically get the new one next time)",
-      "file_count": "0",
-      "index_mod_sample_url": null,
-      "is_hidden": "No",
-      "ksize": "4416",
-      "lang": "es",
-      "logo_url": null,
-      "moddir": "es-blockly_games",
-      "module_id": "129",
-      "prereq_id": null,
-      "prereq_note": "",
-      "rating": "0.0",
-      "rsync_url": "rsync://dev.worldpossible.org/rachelmods/es-blockly_games",
-      "source_url": "",
-      "title": "Old Blockly Games",
-      "type": "html",
-      "version": "v0.1",
-      "zip_ftp_url": "ftp://dev.worldpossible.org/zipped-modules/es-blockly_games.zip",
-      "zip_http_url": "http://dev.worldpossible.org/zipped-modules/es-blockly_games.zip"
-    },
     "id-storybooks": {
       "age_range": "children",
       "category": "educational,language",

roles/cmdsrv/tasks/main.yml

@@ -202,7 +202,7 @@
   copy: src="json/oer2go_catalog.json"
         dest="/etc/iiab/oer2go_catalog.json"
         mode=0644
-        force=no
+        force=yes
         owner=root
         group=root
 

roles/common/templates/adm_const.py.j2

@@ -31,7 +31,7 @@ rachel_mods_url = '{{ rachel_mods_url }}'
 iiab_module_cat_url = '{{ iiab_module_cat_url }}'
 
 oer2go_catalog_file = '{{ iiab_etc_path }}/oer2go_catalog.json'
-iiab_catalog_file = '{{ iiab_etc_path }}/iiab_catalog.json'
+# iiab_catalog_file = '{{ iiab_etc_path }}/iiab_catalog.json' - not used
 
 iiab_menu_repo_dir = "{{ iiab_base }}/iiab-admin-console/roles/js-menu/files/"
 iiab_modules_dir = "{{ iiab_modules_dir }}"