The plan

[thesamovar]

I've been thinking a bit more about how to do this and I think I've got a pretty clear idea in mind now. This issue supersedes previous issues talking about this.

The first thing is that we want a clean, minimal UI. I'm thinking two or three panels, arranged like this:

In other words, the main text you're reading is on the left, and on the right is one or more panels of related information. That might be a figure, some text like definitions, relevant equations, etc. This information scrolls into view automatically and adapts to what you're reading.

The whole idea is that 99% of the time you shouldn't have to think about it, the relevant stuff just comes into view automatically. I want this to be ultra low friction, low cognitive overhead. Of course, you should always have the option of overriding the default choices, either by pinning items, or options, etc., but the default should mostly just work. These defaults are set by the document author (but in many cases can be generated programmatically without manual intervention).

Navigation through a document will most of the time be linear, scrolling up and down through the text. Why this rather than some of the more outlandish linear ideas before? Well, I think on reflection it has a slightly lower cognitive overhead. For the moment, our training at reading is linearly arranged text and our brains are very good at scanning and quickly reading this sort of text. Let's not mess with this. Future generations may change how they do this, but for the moment let's stick with people as they are to some extent.

With that said, I still like the idea of expanding detail and partially non-linear reading order. Here's my thoughts on a design for that. Firstly, some details could be text that appears on the right as a related item. Secondly, some details could be attached to a paragraph or figure and be collapsible, e.g. a figure caption, or an explanation of an equation, etc. There could be a default value of whether to expand or not per item (globally overridable by the user of course).

Thirdly, you should be able to have links within a document that point to a subset of the document and open that up in the left pane. For example, you might start with an abstract that has a background sentence, a sentence on the main result, one on a secondary result and then a final concluding sentence. Clicking on the background sentence would pop up just the introduction section. Clicking on the primary result would pop up the relevant subsection of the results section as well as the relevant subsection of the methods section. Similarly for the secondary result, and so on. So you could either read the paper through in traditional linear order (abstract - intro - results - discussion - methods) or you could start from the abstract clicking through to new sections built from subsets of those other sections.

Here's a couple of designs for how that might look on the left:

I think I generally prefer the second one. In both cases, you have a sort of breadcrumb trail for how you got there. When you scroll down, it will go through in linear order until you reach the end of the section you're currently viewing, and then if you keep scrolling down it closes that section and goes back to your previous view. In other words, you're sort of recomposing your own linear view by where you click.

So that's the overall vision of how it should look, now for details.

Details

The way I see it is that there are three primitives here:

The basic object is a Box, that would correspond basically to a div in html. This can contain any valid HTML, e.g. text, figure, etc. Each Box can have a collapsible Detail box that can expand.

Then we have Links, basically a mapping from one Box to another A->B. When Box A is shown on the left, Box B will float into view automatically on the right. Each Link can have a priority. If all the boxes pointed to by A can be shown on the right they will be, but if there are too many to fit on the screen (dynamically determined) then they're included in order of priority. Ones that don't fit get collapsed to a title bar / icon / preview that can be clicked to enlarge.

Finally, we have a Section. A Section is just a linearly ordered list of Boxes (and Sections, for convenience). These determine what is shown on the left and in what order. There's a default Section that is shown when the document is opened, and clicks can open new Sections with the breadcrumb trail as discussed above. You can also have a Link from a Section to a Box for convenience (equivalent to a link from each Box in the Section).

And that's it. The design is pretty simple, and honestly I think the implementation won't be too hard. The trickiest bit will probably be getting the layout to work nicely, and ideally animated in a nice way (e.g. new Boxes probably shouldn't appear suddenly on the right because that's a bit jarring).

In my mind, I'm thinking that the document would be specified in HTML, since you can do everything above with div and a tags and they nest in exactly the right way. Playing around with React, it seems really nice for specifying separate Components that can be re-used, so this seems a good idea. What I tinkered around with so far is putting the document you want to work with in the public/index.html file created by npx create-react-app and then pulling divs out of that with document.querySelectorAll. This can probably be done better, but the nice thing is that if javascript is disabled it renders nicely.

Thoughts? @synek

[thesamovar]

A lot of this looks like it could be done with existing HTML Semantic elements like section, aside, detail, figure, figcaption, etc. They won't do the layout we want, but they do have the right semantics. So my feeling is we should probably make the most of them. The big thing that is missing is that HTML only seems to define a single linear flow for a document, and there's no way to define the multiple flows that I'm thinking about here. So I'd suggest that we always have a single main flow defined using HTML semantic elements and this would be the fallback if javascript is disabled, and then we could have additional sections defined by a set of section elements with a tags pointing to section/div ids from the main flow. Something like this:

<section id="main">
    <section id="abstract">
        <a href="#intro" class="notpaperlink">Background sentence.</a>
        <a href="#primary" class="notpaperlink">Statement of main result.</a>
        <a href="#secondary" class="notpaperlink">Statement of secondary result.</a>
        <a href="#discussion" class="notpaperlink">Concluding sentence.</a>        
    </section>
    <section id="intro">blah blah</section>
    <section id="results">
        <section id="result-primary">...</section>
        <section id="result-secondary">...</section>
    </section>
    <section id="discussion">blah blah</section>
    <section id="methods">
        <section id="methods-primary">...</section>
        <section id="methods-secondary">...</section>
    </section>
</section>
<section id="primary">
    <a href="#results-primary"/> <!-- is this valid HTML? -->
    <a href="#methods-primary"/>
</section>
<section id="secondary">
    <a href="#results-secondary"/>
    <a href="#methods-secondary"/>
</section>

[rorybyrne]

I like the focus on primitives and their relationships, and I think it's good to simplify the non-linearities so that we can make sure the tool is coherent and practical. When it comes to how we implement it, I have a few thoughts:

1. I think the Box, Section, and Link concepts can be represented really well by a graph. Boxes and Sections would be nodes (with appropriate node attributes), and Links would be edges (with appropriate edge attributes). We could dynamically create new Links to add new narratives through the content like you mentioned. Graphs are a flexible data structure that we can expand later, and they are used for similar apps with transclusion like LogSeq, Roam Research, etc..
2. I'm all for using semantic HTML5 tags, but as far as I know they don't have any functional impact. It just makes it easier for machines to parse the HTML.
3. I don't think we should worry about situations where Javascript is disabled, at least not in the early days of the project. The overlap between people who want a new digital research format and who browse the web with Javascript disabled is surely quite small, and trying to accommodate them would hold back the rest of the project IMO.
4. Overall I think that the data structure and UI is the easier part of the project. Getting research papers into the correct format to be used in Notpaper sounds a lot more difficult to me. We still need to figure out how to parse a PDF/LaTeX doc into a usable format, right?

Some ideas I have:

1. What if we allowed users to create new Links between Boxes and Sections while they're reading? On medium.com you can highlight passages of the text, and it then shows you which passages were highlighted most by users
   
   
2. If we could achieve the above, then we could allow users to upload a PDF or LaTeX document, and then they can add the Boxes, Sections, and Links dynamically via the UI. It would save us the hassle of having to automatically convert documents, since the community would be doing it for us.

[thesamovar]

One of my concerns for the project is archivability. One of the reasons that PDF is such a widely used format for papers is that it is absolutely stable: PDFs just work and you can view a PDF that was created/downloaded 10 years ago, etc. Having a non-javascript fallback, and making sure the data structure is viewable HTML means that even if the fancy interactive features break at some point, the document will still be viewable for decades to come. It also means that if everything can be dumped into one HTML file it can be downloaded as a single file and viewed offline.

Love the idea of readers being able to construct their own new narratives for the paper as they read, although I'd say it should be a secondary objective as relatively few will do it. It's a bit more complicated than highlighting I guess.

Also agree that conversion is going to be a big issue. I think reading from PDFs is likely to be too challenging for us, so we should probably target importing from latex/html/word. We should look into the workflow being used by journals to generate JATS, because that would be a natural workflow for us to adapt.