Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local scheduler prototype #57

Open
wants to merge 32 commits into
base: master
Choose a base branch
from
Open

Conversation

pgaref
Copy link
Contributor

@pgaref pgaref commented Jan 18, 2016

Local Scheduler prototype implementation:

  • Creates new Local Scheduler Commands
  • Separates workers in groups by ip
  • Elects a new Local Scheduler for each group
  • Connect global scheduler with local
  • Local Scheduler logic including tasks and priorities

TODO:

  • Tracking multiple Stages on global scheduler level
  • Write some proper tests

pgaref and others added 30 commits December 14, 2015 18:00
…line using a static configuration file like: ./seep-worker --properties.file=/home/pg1712/my-config/worker1.properties. As it tourned out the properties file was not being read because by default it was expecting a local file with name properties.file. Solution: I fixed that reading the prorties-filename form the command line, given by the user and then changed the file priorities to have higher priority than command line arguments - otherwise they are going to be ignored every time! I am not sure what was the idea here but the way it works now makes more sense to me.
…ties file - TODO: make sure the properties parameters have higher priority than the DEFAULT values (which are part of command line values)
…an have custom names) - implemented an extra method to check if the command line property has the default value or not
Merge remote-tracking branch 'origin/master' into config_file_fix
…mmands for: Local Scheduler Election and Local Scheduler Stage
…s tracking * Fix worker to local scheduler notifications * Introduce Groupstages concept * proper exceptions when worker receive local scheduler commands or the other way round * Add GroupStage concept and deque accord to priorities
…than one workers as well. TODO: finally introduce priorities
… send multiple independent stages from global to local scheduler
…time - multuple workers tested working - For a clean implementation I need to extend StageTracker to support multiple stages
…time - multuple workers tested working - For a clean implementation I need to extend StageTracker to support multiple stages
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant