import bib by zotero translation server

[tshu-w]

Inspired by this pull request jkitchin/org-ref#901, we can import bib entries by Zotero Translation Server with Zotero Translators which support a large range of websites. We don't need to install Zotero but should run a translation server somewhere as describde in the README. Would you mind add following code in ebib(I can make a PR) or mentions this issue in the manual.

  (defcustom ebib-zotero-translation-server "http://127.0.0.1:1969"
    "The address of Zotero translation server."
    :group 'ebib
    :type 'string)

  (defun ebib-zotero-translate (item server-path &optional export-format)
    "Convert item to EXPORT-FORMAT entry through `ebib-zotero-translation-server'."
    (let ((export-format (or export-format
                             (downcase (symbol-name (intern-soft bibtex-dialect))))))
      (shell-command-to-string
       (format "curl -s -d '%s' -H 'Content-Type: text/plain' '%s/%s' | curl -s -d @- -H 'Content-Type: application/json' '%s/export?format=%s'" item ebib-zotero-translation-server server-path ebib-zotero-translation-server export-format))))

  (defun ebib-zotero-import-url (url)
    "Fetch a entry from zotero translation server via a URL.
The entry is stored in the current database."
    (interactive "MURL: ")
    (with-temp-buffer
      (insert (ebib-zotero-translate url "web"))
      (ebib-import-entries ebib--cur-db)))

  (defun ebib-zotero-import-identifier (identifier)
    "Fetch a entry from zotero translation server via an IDENTIFIER.
The entry is stored in the current database,
and the identifier can be DOI, ISBN, PMID, or arXiv ID."
    (interactive "MIDENTIFIER: ")
    (with-temp-buffer
      (insert (ebib-zotero-translate identifier "search"))
      (ebib-import-entries ebib--cur-db)))

[joostkremers]

Hi, thanks for the suggestion. I'm trying to think of the best way to add this. Ebib itself doesn't have any functionality built-in to pull .bib data from the web, so my first thought is that this would be better in biblio.el. Ebib could then include some glue code similar to ebib-biblio-import-doi.

OTOH there's no real reason why it shouldn't be included in Ebib directly, so if you make a PR, I'll certainly consider it.

[tshu-w]

Yes, I also mention in biblio.el. I understand what you are considering, maybe we should add these codes to the wiki？ They are not too long and there are no dependencies but are really useful for those who need them. I want ebib user can find this snippet easily somewhere.

[tshu-w]

Added to the wiki. @joostkremers maybe you can mention wiki page in ebib manual or readme.

[joostkremers]

Thanks, that's certainly a good start for now. I'll add a note about the wiki to the manual.

I'm reopening this issue because I'm still considering adding this to Ebib proper.

[tshu-w]

Another thing to mention is that if adding to ebib, we need to consider whether to use curl or Emacs' built-in url-* function

[tshu-w]

@joostkremers Hi, I have a questions, how to run (ebib-generate-autokey) automatically after import. I tried the following but failed.

  (defun ebib-zotero-import-url (url)
    "Fetch a entry from zotero translation server via a URL.
The entry is stored in the current database."
    (interactive "MURL: ")
    (with-temp-buffer
      (insert (ebib-zotero-translate url "web"))
      (ebib-import-entries ebib--cur-db))
    (ebib-generate-autokey))

[joostkremers]

ebib-generate-autokey can only be called interactively in Ebib's index buffer. There is currently no way to do it non-interactively for one or more entries. It would be possible to write a function to do that, but currently such a function does not exist.

[tshu-w]

ok, I will write some functions to do it myself.

[localauthor]

@tshu-w thanks for putting together this code, it's working great so far.

I'm wondering if you've managed to solve the matter of running ebib-generate-autokey automatically after import?

I've hacked the following onto the end of ebib-zotero-import-identifier to get the job done, but it's pretty crude, so I wonder if you've managed something more elegant?

(defun ebib-zotero-import-identifier (identifier)
  "Fetch a entry from zotero translation server via an IDENTIFIER.
The entry is stored in the current database,
and the identifier can be DOI, ISBN, PMID, or arXiv ID."
  (interactive "MDOI or ISBN: ")
  (kill-new identifier)
  (unless (get-buffer "*Ebib-entry*") ;; check that ebib is running
    (ebib))
  (with-temp-buffer
    (insert (ebib-zotero-translate identifier "search"))
    (ebib-import-entries ebib--cur-db))
  (progn
    (ebib)
    (ebib-db-set-filter `(contains "any" ,identifier) ebib--cur-db)
    (ebib--update-buffers)
    (ebib-filters-cancel-filter)
    (ebib-generate-autokey)))

The above uses the input identifier as a filter to locate the entry in the index, then cancels the filter, leaving the entry at point. Surely this isn't the best way to jump to the newest entry.

When discussed in #232, it is suggested to use ebib--goto-entry-in-index, which takes a key, but that would require extracting the key from the entry, if available --- perhaps in the temp-buffer? I'm still working out how that might be done. Any thoughts?

Thanks in any case.

[localauthor]

This seems to work so far, using ebib--goto-entry-in-index this time, but perhaps there's a better way.

(defun ebib-zotero-import-identifier (identifier)
  "Fetch a entry from zotero translation server via an IDENTIFIER.
The entry is stored in the current database,
and the identifier can be DOI, ISBN, PMID, or arXiv ID."
  (interactive "MDOI or ISBN: ")
  (let (entry-type key)
    (kill-new identifier)
    (unless (get-buffer "*Ebib-entry*") ;; check that ebib is running
      (ebib))
    (with-temp-buffer
      (insert (ebib-zotero-translate identifier "search"))
      (goto-char (point-min))
      (setq entry-type (ebib--bib-find-next-bibtex-item))
      (setq key (cdr (assoc-string "=key=" (parsebib-read-entry entry-type))))     
      (ebib-import-entries ebib--cur-db))
    (progn
      (ebib)
      (ebib--goto-entry-in-index key)
      (ebib-generate-autokey))))

cc: @Hugo-Heagren Maybe something from here could be incorporated into ebib-biblio-doi-import and ebib-biblio-selection-import?

[tshu-w]

I'm wondering if you've managed to solve the matter of running ebib-generate-autokey automatically after import?

Not yet, I've been busy recently, if I finish it I'll append it here or in the wiki.

[gkowzan]

I prepared a pull request implementing this functionality in biblio.el proper, see cpitclaudel/biblio.el#50. It uses either url-retrieve-synchronously or url-retrieve depending on biblio-synchronous setting. My zotero-server ebib branch adds support for Zotero translation server and could be merged after the biblio.el pull request gets merged. I also added org-protocol handler, so that one can click on a bookmarklet and add the entry based on URL to the current database

[tshu-w]

@gkowzan Well done! Using org-protocol is really convenient!