-
Notifications
You must be signed in to change notification settings - Fork 4
Ongoing discussion of project and sharing of knowledge
This is a place to discuss where we are with the project, who is working on what, and what we are learning through our work. Please identify each addition you make to this page with your username or name. -clearcom0
So I just figured out that in order to receive email updates about when the repository is updated or (for example) when something is added to this page, you have to "Watch" the repository. Just click the button above that says "Watch" ... so apparently, "watching" a repository does not send you emails when that repository's wiki is updated... -clearcom0
If you see commits by "online", that is me - when pushing via ssh, I did not have my "git config user.name" set. By the way if you want to push via the command line using ssh, you have to upload your public key to github - go to the upper right-hand corner of the page - the wrench and screwdriver ("Account Settings"), then click on "SSH Keys", copy the text from your ssh public key file (youruserhomefolder/.ssh/id_rsa.pub, for example) and paste it into the "Key" where you add a key (if you don't have any keys, use ssh-keygen to create them). Then, from inside your CCVote folder, "git pull"; then commit your local changes and "git push ssh://[email protected]/HackSpaceClt/CCVote.git". I don't completely understand how to handle merges from the command line yet, so if you want to avoid that, maybe you should just continue to use your GUI git client. If you are doing it with ssh on the command line, make sure you set the user.name variable in your repository with "git config user.name "yourusername"" so your commit will show with the username you desire rather than your local user account name on your local OS. You can view the current user.name set in your repository (and other git config variables) with "git config --list".
By the way, you can also push via command line using https with "git push https://github.com/HackSpaceClt/CCVote.git" It will then ask for your github username and password. I cannot always use this method as one of my git installations was compiled without support for http/https. -clearcom0
I have been looking into temporary tables in order to create one that will be used either per vote or per meeting. There are two reasons for this: one is we don't want a query to have to search through all votes ever recorded in the database to, for example display the votes made since the current vote was opened on the videoOverlay or clerk's interface. Also, the voter interfaces will be constantly querying something to determine whether the current vote is open or closed, in order to display the voter interface as active or inactive. Mike mentioned that 0MQ would help with this problem. However, as we do not have 0MQ setup yet, and are just working with the Django development server for now, I am looking to create some additional models that can later tie in to whatever temporary / caching system we end up using.
As of right now, there doesn't appear to be a way to create a temporary table through Django's ORM. Apparently, by default each time Django reads from the database, it opens and closes a database connection. This means a true temporary table created with the "temporary" keyword in mysql would not persist between subsequent reads of the table with Django's normal settings (see here and search for "TEMPORARY keyword"). Some solutions to this problem are suggested here. However, this route would require using some raw SQL (bypassing Django's ORM) to create the TEMPORARY table along with one of the work-arounds to keep the connection open, and creating a separate model and linking it to the temporary table, as far as I can tell. There is also something called Django Cache Machine, with which you can apparently specify certain models to be cached. This might work to create separate models that we can specify as cached with this plugin. Django Cache Machine requires memcached. So we would need memcached on the server. For now, I am going to create some temporary models with "Temp" as a suffix on the end of their names, which we can later figure out how to tie in to a cache or temporary table of some sort. Mike mentioned having a table that is flushed and cleared per vote, so for now I will create a new model called voteTemp with fields userid, motionid, and vote for the videoOverlay to query. These fields can be written to the VoteData model (and thus to the database) when the vote is closed, and then cleared to start a new vote. -clearcom0
Anyone know how to get variables / objects passed from function in views.py to render as html text in template?
I have filled the UserData and GroupData models with some data, including user_names and user_full_names in UserData for the City Council members' names. I am trying to get these to render in the template, but have been having no luck. I have even started by just trying to get data from the page_data object in the 'def home' view in views.py to render in the home.html template by just adding {{ page_data }} in to the template, but when I do this, nothing is displayed. Does anyone know how to get text from the object to display in the template? (I have also tried {{ page_data['users'].user_name(group_id=2) }} with no luck. The UserData model is populated with the city council members' user_full_name, user_name, and group_id. The mayor is the only one with a group_id=2, so that is the reason for that syntax) -clearcom0
So Walter saved me on this one - The reason {{ page_data }} was not showing anything is because when an "item" (python lingo) or object is passed from the view to the template (in our instance, the "page_data" item), the object becomes a "top level variable in the template" (Walter's words).
Walter explained it the best: "So for example if in the view somewhere we did
page_data['myName'] = 'Gob Bluth'
Then in order to access it in the template you would have to use:
{{ myName }}
So that's why {{ page_data }} didn't work because itself isn't a named variable in the template but the dictionary that populates the template.
Now, in the example above a trivial string values was used to populate an item in the page_data dictionary. Django allows more than just strings. Just about any type of Python primitive or object can be put into the page_data dictionary.
For example:
page_data['cousins'] = ['George Michael', 'Maeby', 'Steve']
Above we have just set the item 'cousins' to be a list of strings. We can access them by using looping control structures in the template system.
{% for cousin in cousins %}
{{ cousin }} ::
{% endfor %}"
-clearcom0
So apparently Websockets does not really work well with Django (Walter had mentioned looking into this at one point). See this link. The other similar options for persistent connections appear to require a second server alongside the Django server (is this where 0MQ comes in?). I am going to look into django-orbited as a way to possibly do long-polling in Django without a second server. I think this is what Paul was talking about Tuesday night in place of 0MQ. One thing he mentioned was the possibility that the number of concurrent connections in Apache might potentially be a problem with doing that. With only about 15 users of our system, 150 concurrent connections seems like plenty to handle our needs. This might mean we wouldn't need 0MQ if we can use long polling with something like django-orbited. I don't know enough about it yet to say for sure.
The reason I am looking into this is because I now have the videoOverlay display page refreshing every half second from a template that is based on the VoteTemp model, but it seems a little intensive (this may be because it is also refreshing the background picture, which would not happen using a solid color background with a chroma key setup). The comet / long polling option might be useful for the voter interface screens as well though, so I will look into it.
Yeah, I think that's where the conversation was sorta going on Tuesday night. I think I tried to explain what the original plan was:
multiple "clients" (including the video overlay) <- long polling -> multiple apache/django processes <- subscription/feed -> one process of 0MQ <-> one process of the DB
And of course, since I'm such a slacker at looking into 0MQ (not to mention the fact that it's not a requirement of the demo), for now I plan on implementing:
multiple "clients" (including the video overlay) <- long polling -> multiple apache/django processes <-> lots of calls to the DB
I have an implementation of long polling that I'd created in php/javascript before we went django, so I'm probably going to re-use some of that logic. Rather than using an existing "product" to implement long polling (due to the requirements of an extra service and/or extra hostnames), I opted to roll my own javascript solely-long-polling implementation. It really wasn't that hard.... Basically goes like this:
- Client initiates an ajax request to a long polling page on the server
- That long polling page doesn't do an HTTPResponse until you tell it to
- Check the DB every .4 seconds or so for a change
- if something changes, return a status of changed and the change and whatever else you want ;)
- return a "nothing's changed" sort of response every so often just to make sure nothing times out. IE every 30 seconds
- When the client gets a response (whether something has changed or not), it should either update it's local stuff and re-start the long-poll (if something changed and warrants it), or just restart the long-poll
Side note -- I think what Paul was saying was that if there exists a product that implements what 0MQ is doing above within python (thus implementable without leaving the django world), that'd probably be a better route. He did also talk about finding a long polling implementation that does fallback to different methods of implementation depending on the clients... But as you might've seen as well, there's so many ways to implement it, and so many of those products imply the necessity of extra host names or services.... Just seems cleaner to roll your own....
-Doofus
After reading how difficult it is to work with web sockets from django, I'd say skip it and use our own long-polling, possibly along with caching like mentioned below. -pwhe23
https://docs.djangoproject.com/en/dev/topics/cache/ Coming from the .Net I was just thinking of the built-in caching mechanism as opposed to something like 0mq. It appears that django has a similar mechanism but you have to use memcached for it to work like I was picturing. I'm not sure if this is an option in our environment. The idea would be to have an in-memory representation of the state of things that the client is polling for. Any changes are made to the in-memory object and also written back to the database for long-term persistence, but the reads would all come from memory. Maybe 0mq can be used for this, but I'm not familiar with it. -pwhe23
This is a tutorial I am following right now that goes through installing Twisted and Orbited in a way that works with Django: http://thingsilearned.com/2009/06/09/starting-out-with-comet-orbited-part-1/ Twisted is an asynchronous server written in python. It apparently can host long polling requests (working with Orbited) as well as Django itself (using wsgi?). So this would remove the need for apache, if I understand correctly, and everything would be going through python. As far as what Paul is saying, I think that is where I was mentioning Django Cache Machine (documentation), in combination with some separate models (like VoteTemp) that can be cached. -clearcom0
So after thinking more about your suggestion above, Mike (at first I didn't completely understand how to do what you were saying), and since we are trying to get a prototype up and running, I am going to try to use your idea in my videoOverlay view (it really wasn't that hard to get orbited installed and working, but maybe the less dependencies we have in the system, the better right?). -clearcom0
So I tried your idea Mike, but was unable to get the development server to serve a json or xml file while it still had the connection for the main page open (which it would have to still be open in order to be running the javascript that accesses that file) - it kept giving me 500 errors. I even tried cloning the repository and running a separate instance of the server on a different port, and changing the url for the json/xml file to http://127.0.0.1:newPortNumber/jsonFile.json, but still no luck. I guess the development server cannot serve multiple connections simultaneously to the same ip address. I wonder if Apache can do this. Maybe this is the limitation and the reason why everywhere I looked, it was talking about using an asynchronous server for real-time applications. I will probably try Twisted next. If that works, once I commit it I guess we can see if it works on your server. If you want to install Orbited, just follow the tutorial I mentioned above, but instead of the example.cfg file he shows, use this one here, and alter the variables to fit what he has in his. That's about as far as I have gotten (just to opening the STOMP config page).
-clearcom0
I've just added some code to handle user function authorizations. The easiest way to use it is
just to guard the entry point of views with the decorator @authorize(). There are four
named constants that map to the different access flags in the GroupData record.
They are:
AUTH_LEVEL_ADMIN
AUTH_LEVEL_CLERK
AUTH_LEVEL_REPORTER
AUTH_LEVEL_VOTERSo, for example, if you need to authorize the view foo() in views.py you just need to:
Make sure you're importing the authorization functions/vars
from authorization import *Then you just add the decorator to your view-function:
@authorize(AUTH_LEVEL_VOTER)
def foo(request):
page_data = {}
page_data['bar'] = 'baz'
return render_to_response('main/foo.html', page_data,
RequestContext(request))If you are using the ActionView base class then you can use the same
decorator on either the action(), action_*(), or default() methods.
class MyView(ActionView):
# ...stuff
@authorize(AUTH_LEVEL_VOTER)
def action_submit(self, args):
# more stuff
@authorize(AUTH_LEVEL_VOTER)
def default(self):
return self.something()If a function should be allowed for multiple auth levels then you
can just keep adding them as arguments to @authorize().
@authorize(AUTH_LEVEL_ADMIN, AUTH_LEVEL_CLERK)
def modify_motion(request):
# even more stuffIf you want your template output to gracefully degrade due to permissions
then you can call auth_template_vars(request, page_data). This will populate
the template variables:
AUTH_LEVEL_ADMINAUTH_LEVEL_CLERKAUTH_LEVEL_REPORTERAUTH_LEVEL_VOTERLOGGED_IN
You just call it on request and template data dictionary
@authorize(AUTH_LEVEL_VOTER)
def bananas(request):
page_data = {}
auth_template_vars(request, page_data)
# cont.Then you can test whether the template is being rendered for a user with a particular auth level.
<h1>{{ motion_description }}<h1>
{% for vote in votes %}
{{ name }} - {{ vote_result }}
{% endfor %}
{% if AUTH_LEVEL_CLERK %}
<a href="/motion/{{ motion_id }}/edit">Edit Motion</a>
{% endif %}
Paul recommended we have one class that is used to acquire any and all information from the database. That way when we figure out if / how frequently accessed database information is cached, we only have to change one class to do that. The class can provide different sets of information through different methods, but for now will provide the current motion state. So Mike is going to create a class, MeetingState that will have a method getCurrentMotion, which will return the state of the current motion (a list of dictionaries containing user_id, motion_id, and vote values, for example (and any other values pertaining to the current motion that might be needed like user_full_name for the videoOverlay display).
I talked about Django signals and having a voteCast signal that is sent when a vote is cast. Signals can carry arguments and in this case it would carry the current vote (a dictionary of user_id, motion_id and vote). This way the videoOverlay view will know a vote has occurred which it can send as the vote happens, rather than having to poll the MeetingClass continuously. However, this will require a background process or service of some sort that can receive voteCast signals and store them in a buffer in case signals are missed while the videoOverlay view is sending the current vote. We don't have a way to do this at the moment. There are various ways to do this - Paul suggested memcache. I have glanced at pyro (and a little at Twister). There is RabbitMQ with Celery - ZeroMQ might still be a possibility; but the conclusion is to just create the one class that provides all the database info (which can be changed later) for now to get a prototype working. -clearcom0
To prevent the iPod touch from going to sleep while the council members are deliberating the issue, we should set the iPod touch to never sleep. To do this, we should go to Settings->General->Auto-Lock and change it to Never.