Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix vacuum on py3.6 #66

Merged
merged 1 commit into from
Nov 5, 2018
Merged

Fix vacuum on py3.6 #66

merged 1 commit into from
Nov 5, 2018

Conversation

geaaru
Copy link
Contributor

@geaaru geaaru commented Nov 4, 2018

No description provided.

@geaaru geaaru requested review from Enlik, lxnay and mudler November 4, 2018 16:02
@@ -700,6 +700,7 @@ def vacuum(self):
"""
Reimplemented from EntropySQLRepository.
"""
self._connection().commit()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

self.commit()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please mention the reason why we issue self.commit() before vacuum() in a comment.

@lxnay
Copy link
Member

lxnay commented Nov 4, 2018

Generally LGTM, but I would add a comment before calling commit() and use self.commit() instead of self._connection().commit().

Thanks!

@geaaru
Copy link
Contributor Author

geaaru commented Nov 4, 2018

Copy that. :)

However, exception is this:

Traceback (most recent call last):
  File "/usr/lib/entropy/lib/entropy/db/sql.py", line 166, in _proxy_call
    return method(*args, **kwargs)
sqlite3.OperationalError: cannot VACUUM from within a transaction

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/eit", line 17, in <module>
    main()
  File "/usr/lib/entropy/server/eit/main.py", line 114, in main
    exit_st = func(*func_args)
  File "/usr/lib/entropy/server/eit/commands/command.py", line 237, in _call_exclusive
    return func(server)
  File "/usr/lib/entropy/server/eit/commands/push.py", line 172, in _push
    rc = self._push_repo(entropy_server, repository_id)
  File "/usr/lib/entropy/server/eit/commands/push.py", line 184, in _push_repo
    rc = self.__push_repo(entropy_server, repository_id)
  File "/usr/lib/entropy/server/eit/commands/push.py", line 309, in __push_repo
    sts = self.__sync_repo(entropy_server, repository_id)
  File "/usr/lib/entropy/server/eit/commands/push.py", line 262, in __sync_repo
    enable_download = False, force = self._force)
  File "/usr/lib/entropy/lib/entropy/server/interfaces/mirrors.py", line 1673, in sync_repository
    enable_upload, enable_download, force = force)
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 231, in update
    enable_upload, enable_download, force = force).update()
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 404, in update
    rc, fine_uris, broken_uris = self._sync()
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1873, in _sync
    broken_uris = self._upload(uris)
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1529, in _upload
    self._shrink_and_close(dbconn)
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1234, in _shrink_and_close
    entropy_repository.vacuum()
  File "/usr/lib/entropy/lib/entropy/db/sqlite.py", line 703, in vacuum
    self._cursor().execute("vacuum")
  File "/usr/lib/entropy/lib/entropy/db/sqlite.py", line 58, in execute
    cur = self._proxy_call(self._cur.execute, *args, **kwargs)
  File "/usr/lib/entropy/lib/entropy/db/sql.py", line 173, in _proxy_call
    raise OperationalError(err)
entropy.db.exceptions.OperationalError: cannot VACUUM from within a transaction

And it happens at this point:

https://github.com/Sabayon/sabayon-sark/blob/master/sark-localbuild#L165

pysqlite has a bug on use VACUUM with py3.6 (see ghaering/pysqlite#109)

Hereinafter, exception related to eit push --quick --force <REPO> command:

Traceback (most recent call last):
  File "/usr/lib/entropy/lib/entropy/db/sql.py", line 166, in _proxy_call
    return method(*args, **kwargs)
sqlite3.OperationalError: cannot VACUUM from within a transaction

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/bin/eit", line 17, in <module>
    main()
  File "/usr/lib/entropy/server/eit/main.py", line 114, in main
    exit_st = func(*func_args)
  File "/usr/lib/entropy/server/eit/commands/command.py", line 237, in _call_exclusive
    return func(server)
  File "/usr/lib/entropy/server/eit/commands/push.py", line 172, in _push
    rc = self._push_repo(entropy_server, repository_id)
  File "/usr/lib/entropy/server/eit/commands/push.py", line 184, in _push_repo
    rc = self.__push_repo(entropy_server, repository_id)
  File "/usr/lib/entropy/server/eit/commands/push.py", line 309, in __push_repo
    sts = self.__sync_repo(entropy_server, repository_id)
  File "/usr/lib/entropy/server/eit/commands/push.py", line 262, in __sync_repo
    enable_download = False, force = self._force)
  File "/usr/lib/entropy/lib/entropy/server/interfaces/mirrors.py", line 1673, in sync_repository
    enable_upload, enable_download, force = force)
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 231, in update
    enable_upload, enable_download, force = force).update()
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 404, in update
    rc, fine_uris, broken_uris = self._sync()
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1873, in _sync
    broken_uris = self._upload(uris)
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1529, in _upload
    self._shrink_and_close(dbconn)
  File "/usr/lib/entropy/lib/entropy/server/interfaces/db.py", line 1234, in _shrink_and_close
    entropy_repository.vacuum()
  File "/usr/lib/entropy/lib/entropy/db/sqlite.py", line 703, in vacuum
    self._cursor().execute("vacuum")
  File "/usr/lib/entropy/lib/entropy/db/sqlite.py", line 58, in execute
    cur = self._proxy_call(self._cur.execute, *args, **kwargs)
  File "/usr/lib/entropy/lib/entropy/db/sql.py", line 173, in _proxy_call
    raise OperationalError(err)
entropy.db.exceptions.OperationalError: cannot VACUUM from within a transaction
@Enlik
Copy link
Member

Enlik commented Nov 4, 2018

Have you tested this codepath with Python 2 and 3?

@geaaru
Copy link
Contributor Author

geaaru commented Nov 4, 2018

@Enlik I tested this on ARM with py3.6 and it works. I can't test this with py2.7 for issue #64 :'(

@Enlik
Copy link
Member

Enlik commented Nov 4, 2018

You can test it on another package, just make sure this part of code is executed and doesn't explode. It's Python, so it's easy; you can even change this file on you system's installation of Entropy*. (I'm not sure if tests cover this part - if they do, it's ok too.)

Second, instead of "and pr for details: #66" it's better to link to https://bugs.python.org/issue28518.

* and add a print with the information "hey I'm about to execute this" perhaps

@lxnay lxnay merged commit 1c08347 into Sabayon:master Nov 5, 2018
@geaaru geaaru deleted the eit-push-py36 branch November 5, 2018 09:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants