Highlevel Architecture Overview
There are a number of pieces to this, and some of this might require some experimentation to determine if what I'm proposing here is practical. I will highlight the areas where I'm unsure.
Here's what I'm proposing, at a very high-level:
Some components are numbered, and I will now attempt to explain each numbered component.
I think of this as the core of the new address book. This component is the "grand central station" where all requests to create, read, update or delete contacts will go through.
I want this to be a SharedWorker, because it's important to remember that multiple things (multiple Thunderbird windows, parts of Thunderbird) might want to communicate with the address book at different times. A SharedWorker allows us to contact the same worker across different windows and iframes, which makes it a "global" service. Note that in order to access the same SharedWorker, the things accessing it must share the same origin. This is one of the things I'm not entirely certain about: when Thunderbird loads these local scripts, I believe the origin will be the "System Principal", and I think it should work correctly, but I'm really not certain. We should experiment to make sure that chrome-privileged pages in Thunderbird can communicate over a SharedWorker.
Note that communication with the SharedWorker by its very nature has to be asynchronous, and use message passing.
In general, the SharedWorker should listen for messages for creating, reading, updating, and deleting contact records that it's holding in its cache (I'll get to the cache next).
The Mork database that Thunderbird currently uses for the address book has only one strength that I know of, and that's speed. It is very fast to query it. That's not because Mork is particularly clever, but because the database is loaded into working memory at startup, and all queries occur there; it's essentially an in-memory database.
I think we want to keep that performance characteristic if we can. What I'm proposing is that the new address book holds an in-memory cache of contacts that it can create, read, update and delete (synchronously from within the SharedWorker, asynchronously from clients that message the SharedWorker).
The SharedWorker and its Storage Cache will not have direct access to the disk - SharedWorkers simply don't get access to this. What I suggest is that when the SharedWorker is initialized, it gets passed a message that sends along the entire contents of the address book, which it then shoves into the cache.
Periodically then, the SharedWorker will send a current snapshot of the cache to something more priviledged (like the Hidden DOM Window, which I'll get to soon), which can then write those snapshots to the disk.
When Thunderbird starts to be shut down, it's important that we cease all writes to the cache, and then ask the SharedWorker for the last snapshot, which can then be written to disk.
It's important for various components within Thunderbird to access the address book. In order to do that, they'll need to communicate to a part of the address book using something called XPCOM. XPCOM is powerful and also complicated. I'm hoping we won't have to interact much with it for the majority of this project. However, there are certain touchpoints to the rest of Thunderbird where XPCOM communication just cannot be avoided.
I think I've devised a way of exposing our SharedWorker for Contact Management to XPCOM, and that's through the Hidden DOM Window (which I'll talk about in the next section).
For now, what I imagine this component to be is basically an asynchronous, minimal-as-possible API to the SharedWorker for those bits of Thunderbird that will need to talk over XPCOM. This should probably also be used to act as the "pipe" between the SharedWorker's Storage Cache to the on-disk persistent storage that will be read from at address book initialization (I'll get to that in section 5).
NOTE: It's possible that this, as well as the Hidden DOM Window will not be necessary, if the SharedWorker can be accessed directly from the JSM that I talk about in the next section. This is something worth investigating.
4. Hidden DOM Window (Singleton)
Thunderbird has something called the Hidden DOM Window that is not visible to users, but is essentially an invisible browser window. HTML can be inserted into that Hidden DOM Window just like any other browser window. How I imagine we expose the SharedWorker to XPCOM is through the Hidden DOM Window.
What I imagine is that, on address book initialization, a <script> tag will be loaded into the Hidden DOM Window which will load the minimal XPCOM API surface I talked about in section 4. We will need some singleton code in Thunderbird to do this, and this is probably the entrypoint from which Thunderbird will be initializing the entire address book service.
That singleton code in Thunderbird will likely be a JSM. JSM's are privileged, and what I think should be possible is that the JSM can communicate to the Hidden DOM Window to call the API functions for the XPCOM services. We can then expose the JSM to XPCOM, and this is what the XPCOM-relying bits of Thunderbird will talk to.
NOTE: It's possible that this, as well as the IFrame will not be necessary, if the SharedWorker can be accessed directly from the JSM. This is something worth investigating.
Because the JSM is privileged, it also means it has access to the disk, which means it can be used to retrieve (and persist) the contacts records to whatever storage format we end up deciding on (I'm still rooting for a JSON file, maybe gzipped or something).
This is something that I think we might want to iron out up front - I have a high-degree of confidence that we can get the right performance and simplicity characteristics if we use a JSON file to store the contacts on the disk. I think SQLite is not a great choice, even though the JSM would have access to this. There are a myriad of reasons not to use SQLite. I also can't think of any great reasons why SQLite would be any better than a JSON file, so that's why I'm leaning this way so heavily.
I want the team's input on this to get their thoughts.
The new address book should open in a tab instead of a window, and that tab should render web content (XHTML, JS, CSS). Getting access to the contacts to render should be straight forward assuming the tab can talk to the SharedWorker.
I suspect that something like React will be useful as a framework for implementing this front-end. I also suspect the Flux pattern will be useful for how updates are sent to the SharedWorker, and then broadcast to all contact tabs (since there might be many open at one time, as well as other things that want to hear updates about contacts).
- Can a SharedWorker hosted at a chrome:// URI be accessed from another page at a chrome:// URI? How about from a JSM?
- What does the contact schema look like?
- Suppose we have 500 contacts in the address book. Given the schema from 2, how quickly can we read this data in from a (compressed?) JSON file and do a linear search through it? How about with 5000 contacts? How about with 50,000 contacts? What are the performance characteristics as we increase the number of contacts?
- What is the minimal API surface that the XPCOM components will need from the new address book?