Basic documentation outline with some bullet points

doc/basics.md

@@ -0,0 +1,17 @@
+- Vocabulary
+ - Job (blueprint)
+ - Schedule (instance)
+ - Task (callback that implements the job)
+
+Always vs. default task
+
+XML file vs. database
+
+How does scheduling work
+
+What's new compared to native jobs?
+- new fields/features
+	- name, description,... (https://github.com/AOEpeople/Aoe_Scheduler/issues/46)
+	- groups
+	- parameters
+	- disable

doc/communication.md

@@ -0,0 +1,8 @@
+Run status reporting (return values)
+- Code Sample to report results
+- Code Sample to report error messages
+- Code Sample to report back last_seen and ETA
+
+Communicate back
+- ETA
+- Write progress message and last seen for product importer (is this implemented?)

doc/cron-configuration.md

@@ -0,0 +1,15 @@
+cron.sh vs cron.php vs scheduler_cron.sh
+
+Recommendet frequency
+
+always vs default
+
+Not in webserver context (no wget php)
+
+How to correctly configure cron?
+Note: Correct user!
+Also check: http://www.magentocommerce.com/knowledge-base/entry/ce18-and-ee113-installing#install-cron
+NOT: http://stackoverflow.com/questions/5644037/setting-up-cron-job-in-magento?rq=1
+Add note to not run this via wget 
+- cron is not supposed to run in web context
+	- different php settings, max_execution time, memory_limit (and don't increase these globally only to satisfy cron.php)

doc/domain-model.md

@@ -0,0 +1,4 @@
+Schedule Manager
+Process Manager
+Abstract Job
+Factory

doc/faq.md

@@ -63,4 +63,8 @@ This is not so much a question about Aoe_Scheduler, but an issue with Magento's
 - In many cases cron tasks are not designed to run in parallel (race conditions,...). Cron will take care of running jobs sequentially (unless you use the cron group feature and you know what you're doing). Running the job directly doesn't offer this protection.
 - Ideally your webserver user is the same user that also executes cron. If that's not the case you might run into problem with files being generated in cron tasks not having sufficient permissions for the next run from a different environment.
 
-Because of the reasons mentioned above newer versions of Aoe_Scheduler ship with the "runNow" feature being disabled by default. In case you know what you're doing you're welcome to enable this feature in the configuration.
+Because of the reasons mentioned above newer versions of Aoe_Scheduler ship with the "runNow" feature being disabled by default. In case you know what you're doing you're welcome to enable this feature in the configuration.
+
+### I've uninstalled a module that introduced cron jobs and now I'm gettings error messages. What's going on?
+
+TODO...

doc/installation.md

@@ -0,0 +1,8 @@
+Compatible with hackathon installer (or aoepeople installer).
+
+Show snippet
+
+More information (including alternate ways like Magento connect and copy):
+
+http://fbrnc.net/blog/2014/11/how-to-install-a-magento-module
+

doc/job-configuration.md

@@ -0,0 +1,2 @@
+Edit and create jobs
+Explain overlay mechanism

doc/process-management.md

@@ -0,0 +1,3 @@
+- last seen
+- pid...
+- kill job

doc/processing.md

@@ -0,0 +1 @@
+Explain workflow from sh -> php -> events,...

doc/readme_old.md

@@ -0,0 +1,118 @@
+![Logo](doc/images/Aoe_Scheduler.jpg "AOE Scheduler")
+
+# AOE Scheduler for Magento
+
+
+
+## Installation
+
+
+
+see http://www.fabrizio-branca.de/magento-cron-scheduler.html
+
+Running multiple scheduler jobs in parallel
+-------------------------------------------
+
+*INFO*: Running jobs in parallel like this is outdated. Please check the usage of scheduler_cron.sh below.
+
+Running multiple scheduler jobs in parallel is quite handy when you have some jobs which take a long time to run,
+and are not crucial for shop to run properly. En excample could be a job which generates some kind of reports.
+Magento cron.sh script detects if there is some job already running by parsing output of "ps" command.
+If not, it executes cron.php file.
+So to run multiple schedulers in parallel we have to use different file name than cron.php.
+Additionally we want to run only few jobs with second cron task, and exclude them from the original scheduler job.
+Aoe Scheduler comes with whitelist/blacklist feature enabling you to do so.
+
+1. symlink cron.php to cron2.php
+2. add crontab configuration (crontab -e) like this:
+
+```
+* * * * * /usr/bin/env SCHEDULER_BLACKLIST='job_key1,job_key2' /bin/sh  /home/webapps/htdocs/cron.sh
+* * * * * /usr/bin/env SCHEDULER_WHITELIST='job_key1,job_key2' /bin/sh  /home/webapps/htdocs/cron.sh cron2.php
+```
+
+cron.sh script takes name of the php script to run as a parameter (by default it is cron.php).
+This way the first cron job will execute all scheduler jobs except job_key1 and job_key2, and the second one will execute only them.
+
+
+scheduler_cron.sh vs cron.sh
+----------------------------
+Aoe_Scheduler introduces a new shell script acting as an endpoint to execute cron. 
+
+This new endpoint supports including and excluding groups of crons and manages the parallel execution of cron groups.
+
+scheduler_cron.sh then will call shell/scheduler.php --action cron with the correct mode and options parameters instead of running cron.php
+Managing multiple configuration this way is much cleaner and easier to understand.
+
+scheduler_cron.sh or shell/scheduler.php --action cron will not do any shell_exec to run the different modes in subtasks. It's up to you to configure
+multiple OS level crontasks to trigger these individually.
+
+TODO: Add more information and example code.
+
+Parameters: 
+* `--mode [always|default]`
+* `--excludeJobs [comma-separated list of job names]` (deprecated!)
+* `--includeJobs [comma-separated list of job names]` (deprecated!)
+* `--excludeGroups [comma-separated list of cron group names]` 
+* `--includeGroups [comma-separated list of cron group names]`
+
+How to define a cron group
+--------------------------
+
+### Via XML
+
+```
+<my_cron_job>
+    <schedule><config_path>...</config_path></schedule>
+    <run><model>...</model></run>
+    <groups>my_group</groups>
+</my_cron_job>
+```
+
+### Or in a the group field of a record 
+
+TODO: add screenshot here.
+
+Configuring crontask
+--------------------
+
+Please remember to add the following configuration to the webserver user's crontab in order to avoid file permission issues with files that are being generated or need to be accessed from different contexts!
+
+### Split always and default:
+
+```
+* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode always
+* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode default
+```
+
+### Prepend check for maintenance.flag:
+
+```
+* * * * * ! test -e /var/www/magento/current/htdocs/maintenance.flag && ...
+```
+
+### Configure multiple cron groups
+
+This way you can distribute the jobs on different servers. Also this allows you to control which jobs can run in parallel and which don't. Within a cron group only one job will run at a time, while jobs from multiple cron groups might and will run at the same time 
+
+```
+# Always tasks
+* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode always --includeGroups my_queue_jobs
+* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode always --excludeGroups my_queue_jobs
+
+# Default tasks
+* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode default --includeGroups groupA,groupB
+* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode default --includeGroups groupC
+* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode default --excludeGroups groupA,groupB,groupC
+```
+
+If splitting jobs across multiple groups please double check you have every job covered in one of the includeGroups or excludeGroups list.
+
+Watchdog
+--------
+
+Please configure an additional cron for the watchdog. This will ensure that locked up jobs are being cleaned up even if every single cron process is locked.
+
+```
+*/10 * * * * /usr/bin/php /var/www/magento/current/htdocs/shell/scheduler.php --action watchdog
+```

doc/states.md

@@ -0,0 +1,5 @@
+- didn't do anything
+- success
+- error
+
+How to report a state back?

doc/system-configuration.md

@@ -0,0 +1,2 @@
+Walk through all the settings here.
+Advice on how to choose the cron frequency wisely

doc/timeline.md

@@ -0,0 +1 @@
+Cleanup tasks (keep last x)

doc/what-is-aoe-scheduler.md

@@ -0,0 +1,9 @@
+
+Aoe_Scheduler compared to native Magento Cron
+- compatible
+- Features
+	- Visualization
+	- Management: Editing, Creating, Scheduling
+	- Process management
+	- Fine-grained access 
+	- API (Cli, Web Service, PHP)

readme.md

@@ -1,116 +1,42 @@
 # AOE Scheduler for Magento
 
-![Logo](doc/Aoe_Scheduler.jpg "AOE Scheduler")
-
-## Installation
-
-
-
-see http://www.fabrizio-branca.de/magento-cron-scheduler.html
-
-Running multiple scheduler jobs in parallel
--------------------------------------------
-
-*INFO*: Running jobs in parallel like this is outdated. Please check the usage of scheduler_cron.sh below.
-
-Running multiple scheduler jobs in parallel is quite handy when you have some jobs which take a long time to run,
-and are not crucial for shop to run properly. En excample could be a job which generates some kind of reports.
-Magento cron.sh script detects if there is some job already running by parsing output of "ps" command.
-If not, it executes cron.php file.
-So to run multiple schedulers in parallel we have to use different file name than cron.php.
-Additionally we want to run only few jobs with second cron task, and exclude them from the original scheduler job.
-Aoe Scheduler comes with whitelist/blacklist feature enabling you to do so.
-
-1. symlink cron.php to cron2.php
-2. add crontab configuration (crontab -e) like this:
-
-```
-* * * * * /usr/bin/env SCHEDULER_BLACKLIST='job_key1,job_key2' /bin/sh  /home/webapps/htdocs/cron.sh
-* * * * * /usr/bin/env SCHEDULER_WHITELIST='job_key1,job_key2' /bin/sh  /home/webapps/htdocs/cron.sh cron2.php
-```
-
-cron.sh script takes name of the php script to run as a parameter (by default it is cron.php).
-This way the first cron job will execute all scheduler jobs except job_key1 and job_key2, and the second one will execute only them.
-
-
-scheduler_cron.sh vs cron.sh
-----------------------------
-Aoe_Scheduler introduces a new shell script acting as an endpoint to execute cron. 
-
-This new endpoint supports including and excluding groups of crons and manages the parallel execution of cron groups.
-
-scheduler_cron.sh then will call shell/scheduler.php --action cron with the correct mode and options parameters instead of running cron.php
-Managing multiple configuration this way is much cleaner and easier to understand.
-
-scheduler_cron.sh or shell/scheduler.php --action cron will not do any shell_exec to run the different modes in subtasks. It's up to you to configure
-multiple OS level crontasks to trigger these individually.
-
-TODO: Add more information and example code.
-
-Parameters: 
-* `--mode [always|default]`
-* `--excludeJobs [comma-separated list of job names]` (deprecated!)
-* `--includeJobs [comma-separated list of job names]` (deprecated!)
-* `--excludeGroups [comma-separated list of cron group names]` 
-* `--includeGroups [comma-separated list of cron group names]`
-
-How to define a cron group
---------------------------
-
-### Via XML
-
-```
-<my_cron_job>
-    <schedule><config_path>...</config_path></schedule>
-    <run><model>...</model></run>
-    <groups>my_group</groups>
-</my_cron_job>
-```
-
-### Or in a the group field of a record 
-
-TODO: add screenshot here.
-
-Configuring crontask
---------------------
-
-Please remember to add the following configuration to the webserver user's crontab in order to avoid file permission issues with files that are being generated or need to be accessed from different contexts!
-
-### Split always and default:
-
-```
-* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode always
-* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode default
-```
-
-### Prepend check for maintenance.flag:
-
-```
-* * * * * ! test -e /var/www/magento/current/htdocs/maintenance.flag && ...
-```
-
-### Configure multiple cron groups
-
-This way you can distribute the jobs on different servers. Also this allows you to control which jobs can run in parallel and which don't. Within a cron group only one job will run at a time, while jobs from multiple cron groups might and will run at the same time 
-
-```
-# Always tasks
-* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode always --includeGroups my_queue_jobs
-* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode always --excludeGroups my_queue_jobs
-
-# Default tasks
-* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode default --includeGroups groupA,groupB
-* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode default --includeGroups groupC
-* * * * * /bin/bash /var/www/magento/current/htdocs/scheduler_cron.sh --mode default --excludeGroups groupA,groupB,groupC
-```
-
-If splitting jobs across multiple groups please double check you have every job covered in one of the includeGroups or excludeGroups list.
-
-Watchdog
---------
-
-Please configure an additional cron for the watchdog. This will ensure that locked up jobs are being cleaned up even if every single cron process is locked.
-
-```
-*/10 * * * * /usr/bin/php /var/www/magento/current/htdocs/shell/scheduler.php --action watchdog
-```
+![Logo](doc/images/Aoe_Scheduler.jpg "AOE Scheduler")
+
+## Documentation
+
+1. [What is AOE Scheduler?](docs/what-is-aoe-scheduler.md)
+1. [Module Installation](docs/installation.md)
+1. [Cron Configuration](docs/cron-configuration.md)
+1. [Some Basics (Job? Schedule? Task?)](docs/basics.md)
+1. [Faq](docs/faq.md)
+1. [Changelog](docs/changelog.md)
+1. Admin Interfaces
+    1. [System Configuration](docs/system-configuration.md)
+    1. [Job Configuration](docs/job-configuration.md)
+    1. [List View](docs/list-view.md)
+    1. [Timeline View](docs/timeline.md)
+    1. [Instructions](docs/instructions.md)
+1. [Events](docs/events.md)
+1. [Job States](docs/states.md)
+1. [CLI](docs/cli.md)
+1. [Web Services](docs/web-services.md)
+1. [Domain Model](docs/domain-model.md)
+1. [Cron Processing](docs/processing.md)
+1. Features
+    1. [Heartbeat](docs/heartbeat.md)
+    1. [Notifications](docs/notifications.md)
+    1. [Custom job parameters](docs/custom-job-parameters.md)
+    1. [Global enabled/disable](docs/global-enable-disable.md)
+    1. [Disable individual jobs](docs/disable-individual-jobs.md)
+    1. [runNow](docs/run-now.md)
+    1. [scheduleNow](docs/schedule-now.md)
+    1. [Run return values and communication](docs/communication.md)
+    1. [ETA](docs/eta.md)
+    1. [Watchdog](docs/watchdog.md)
+    1. [Cron groups](docs/cron-groups.md)
+    1. [Process Management](docs/process-management.md)
+1. Cookbook
+    1. [Distributed cron](docs/distributed.md)
+    1. [Job workflows](docs/job-workflows.md)
+    1. [Schedule a job programmatically](docs/programmatically.md)
+