Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-organize econ-ark information #13

Closed
3 tasks done
shaunagm opened this issue Mar 1, 2019 · 42 comments
Closed
3 tasks done

Re-organize econ-ark information #13

shaunagm opened this issue Mar 1, 2019 · 42 comments

Comments

@shaunagm
Copy link

shaunagm commented Mar 1, 2019

I'm putting pause on this until I've got a few more weeks with the project, at least, but I've already made some notes on how to improve our information organization so it's more logically laid out, more publicly accessible, and less redundant.

Key steps include:

  • re-organizing the private google drive to make more sense and contain as little information as possible, and no redundant organization
  • move SGM's personal google docs into appropriate places in google drive
  • remove unnecessary repositories from this org, some of which have instructional/presentation content which may end up on the website (in progress, see below)

We'll also need to update the website (see issue #27) and re-organize documentation (see issue #16).

Stuff that needs to move to separate sphinx documentation repo or to restructured HARK/documentation branch:

  • Hark's gh-pages branch
  • HARK/documentation
  • NARK (probably)
  • either the site should directly host content related to the HARK Manual, or it should clearly link to it as a "tutorial"
@shaunagm
Copy link
Author

shaunagm commented Apr 5, 2019

List of repositories

Repo Name Description Keep/Delete Public/Private
Hark HARK code keep public
Remark "Replications and Explorations Made using the ARK" keep public
Remark-make generates remarks content, may be able to move to source folder, see item 1 below ? private
OverArk organizational/administrative repo keep public
Quark question sets using econ-ark keep public
Quark-make generates sets & solutions, needs to be separate to hide solutions, may be able to move to source folder, see item 1 below ? private
Demark demonstrations of how to use econ-ark keep public
Demark-make I'd like to move most of this to a source folder in demark and then delete, see item 1 below delete private
ExArk Super-repo with submodules for DemARk, Remarks, Quark, Titlark - confused about the purpose of this repo, see item 2 below ? public
ExARK-make delete if deleting exark, otherwise explore if can move this to source folder in exark and delete this repo ? private
Titlark "Teaching and Instructional Tools for Learning the ARK" - clearly we should keep the content, but do we need a separate repo, see item 2 below ? public
Titlark-make generates Titlark files, recommend we move this to source folder in Titlark then delete, see item 1 below delete private
HARK-make generates HARK manual in pdf and markdown, need to implement alternate approach to HARK manual before deleting keep (for now) private
NARK notation information; we should consider incorporating this directly into documentation but leave as is for now keep public
PARK presentation materials, possibly worth keeping in version control but should at least be featured on the website too, they're easy to lose in a github repo keep public
PARK-make generates files for Park, should move to source folder in PARK, see item 1 delete private
interARK stores grant submission and similar stuff, I think this is better as folder in google drive delete private
econ-ark-make looks like this was used as a todo list by Nathan and Alex in summer 2018, I recommend emailing them the contents and deleting the repo (also this file might make a good wiki page) delete private
jobs job descriptions - there are better ways of sharing this info deleted public
scipy_proceedings fork of scipy's version of this, doubt we need it anymore deleted public
Ballpark papers that are "in the ballpark" of Ark - empty, there are better ways to capture this info than a github repo delete private
Ballpark-make see description for "ballpark" delete private
Postmark a mostly empty repo with a small text file describing plans for a journal - moved to google drive & deleted deleted private
Plutark "a list of links to completed papers that use the Econ-ARK project" - empty, and we're using a new system to track this anyway delete private
Plutark-make contains files to generate SciPy paper; I feel like this could be folded into one of the 6 other "using hark" repos - see item 2 below delete private
econ-ark.github.io simply redirects to econ-ark.org delete public

General Thoughts

1) -make repos

Generally speaking, I dislike the use of the private -make repos. If you're generating content, I recommend keeping the "generator" and the "generated" files in the same repository, just in separate top level folders. I've typically seen these called "build" and "source". In the README, you then say "to look at existing files, read the build, to change what the build looks like, edit the source".

The only reason you should need a separate repo is if there's something secret or private in the build, which doesn't seem relevant here, except possibly in the case of QUARKs, which have hidden solutions for question sets. (In an email, Chris said REMARKs have secret solutions as well? I'm not sure whether the scripts delete the solutions or merely temporarily hide them - if they're merely temporarily hidden, then we don't need a separate -make repo.)

The other reason that seems to be driving the use of these -make files is the difficulty of versioning jupyter notebooks as described in demark-make, so it's worth learning more about those constraints before making a hasty decision.

Otherwise, keeping build and source files in the same repo makes it easier to keep them in sync and produces less clutter.

2) Repos demonstrating usage of HARK

There's a bunch of repos for tracking usage of HARK. The ExARK repo uses some of them as submodules (DemARk, Remarks, Quark, Titlark) but not others (park, plutark, ballpark). It's really not clear to me how ExARK is meant to be used, though. Why 7 separate repos and a meta-repo, and not one repo with 7 folders? Couldn't nearly all of them go into DemARK as specific kinds of demonstrations?

There's also the question of what can be taken out of github altogether and put onto the website via a content management system. I think Park, Plutark, and Ballpark are the most likely candidates for this.

3) Documentation

When it comes to organizing our documentation, we may want to move the docs out of the gh-pages branch of HARK and into their own repo. I don't have a strong opinion of this yet.

Current Recommendation

  1. Move all the content of remarks/demarks/quarks/plutarks/titlark/park/ballpark into a single repo, using separate directories as needed to keep the content distinct. Delete all old repos, including interark.
  2. Move most of the -make content into a source folder in this single repo, keeping one -make version to handle any "secret" content (if any is necessary).
  3. Delete the other repos marked delete above, like jobs and scipy_proceedings.

@shaunagm
Copy link
Author

shaunagm commented Apr 11, 2019

Concrete Proposal for Consolidating Repos

There are currently seven (overlapping, ambiguous) types of content that we're tracking across 16 repos. They are:

  • Remarks "Explorations (Use the Econ-ARK/HARK toolkit to demonstrate some set of modeling ideas), Replications (Attempts to replicate the results of published papers written using other tools) and Reproductions (Code that reproduces the results of some paper that was originally written using the toolkit)"
  • Demarks "Demonstrations of how to use material in the Econ-ARK" (not clear how this is different from an "Exploration" style of Remark)
  • Titlarks "Teaching and Instructional Tools for Learning the ARK"
  • Quarks "Questions (teaching exercises with the statement of "Problems" that can be solved in place)" (not clear how this is different from Titlarks - maybe a subset of titlarks? they both focus on teaching)
  • Parks "presentations"
  • Plutarks "a list of links to completed papers that use the Econ-ARK project" - currently empty
  • Ballparks ("means papers or models that are closely related enough to be of interest to the kinds of people who are interested in the Econ-ARK") - can be used to create partial demonstrations (demark) and fullscale replications (remark)

I would like to propose a new set of categories:

  • Demonstrations: jupyter notebooks demonstrating usage of HARK
  • Replications: jupyter notebooks implementing existing papers in HARK
  • Instructional tools: notebooks constructed for teaching purposes, including problem sets like would be previously found in "quarks"
  • Presentations: notebooks and non-notebook content from presentations given related to HARK/econ-ark

Rather than keeping lists of papers that use the Econ-ARK project, and related papers, in a repo, we can use two different lists in the Zotero account (one for the "plutarks" and one for the "ballparks"), both of which can be linked to from the repository readme.

All of the notebooks have both "source" and "build" files. The source files are currently in separate -make directories, but they should be moved into "source" subfolders within the main repository. The only issue with doing this is the "quarks"/problem sets. I propose we retain the private quark-make repository for this and only this content.

Thus, the final structure of whatever-the-newly-named-repo-is would be:

  • Demonstrations
    • build (generated files)
    • source (corresponding -make files)
  • Replications
    • build (generated files)
    • source (corresponding -make files)
  • Instructional
    • non-problem sets
      • source (corresponding -make files)
      • build (generated files)
    • problem sets
      • build (generated files)
      • [NO source folder, source is in private -make repo]
  • Presentations
    • source (corresponding -make files)
    • build (generated files)
    • non-generated presentations (standalone pdfs or powerpoints, etc)
  • README with, among other things, links to Zotero lists of papers that use HARK, papers related to HARK, and "wish list" papers that we'd like to see implemented in HARK and added to this repo

Once the content was moved, and any links to the content from other places (such as the website) are fixed, and we've verified that mybinder/colab works with the new notebook structure and that the problem sets are being generated appropriately, we'd delete all folders except demark (or remark, or whichever repository we use as the main repository), and except quark-make.

A note on naming: if Chris is very attached to the names, we can name the demonstrations folder "demarks", the replications folder "remarks", teaching as "titlarks", and presentations as "parks", although I think a more readable name would be better. The repository as a whole could keep the name demark, as all the subtypes are in some way demonstrations, but I don't really care what we name the repo as a whole.

@shaunagm
Copy link
Author

@llorracc @mnwhite @pkofod - as promised, here's a proposal for re-organizing the bulk of our content. See the comment above this one. Let me know if you have questions, concerns, suggestions, counterproposals, etc.

@shaunagm
Copy link
Author

shaunagm commented Apr 18, 2019

Chris notes: things besides Quarks will need to be secret, plus other instructional stuff. (Caveat - want to have papers, that's none instructional.)

@pkofod
Copy link

pkofod commented Apr 19, 2019

* README with, among other things, links to Zotero lists of papers that use HARK, papers related to HARK, and "wish list" papers that we'd like to see implemented in HARK and added to this repo

I think it good to use the software/services that exist (and we use) to keep track of papers for a problem of keeping track of a list of papers, so I fully agree here, versus keeping a "manually" written list in a readme to a repo.

@shaunagm
Copy link
Author

shaunagm commented Jun 13, 2019

Okay, I'm preparing to merge the repositories soon.

My plan is to use the DEMARKs repository as my base, because it's the main repo with open PRs, and I don't want to have to reissue those. So I'm planning to:

  • fork the DEMARKs repo
  • restructure the repo according to the above plan
  • merge the other repositories into the base/demarks repo, conserving their git history using this methodology
  • generate list of references to materials (primarily from website I think) so I can quickly update once changes are merged
  • push my fork to the econ-ark repo, which implements the changes
  • the open PRs to the DEMARKs repo then need to be rebased so the changes (largely new files) are going to the right directories
  • check for other things to manually rebase/resubmit, for instance there's a branch in remarks, Liqconstr
  • finally, once we're sure everything has been done correctly, and once the -make repositories have been consolidated into a single private repository, we can delete the now-redundant repositories.

@llorracc
Copy link
Contributor

@shaunagm, I've finally had a chance to read through this and think about it.

I think this is a good draft of a plan, but am not sure that it is ready for pulling the trigger. In particular, I want to process and absorb some of the ideas in the various extremely interesting presentations at PASC19 from various different research organizations -- esp the presentation on documentation that broke things down into categories like "tutorials" and "getting things done" and "retrieving information" (there was a fourth category that I can't remember at the moment but was also persuasive -- see the "issue" I posted in HARK -- which maybe should have been in OverARK).

A point that was emphasized over and over at PASC, and which I agree with entirely, is the importance of setting things up so our "meta" content is generated as automatically as possible. (The presenter strongly endorsed Sphinx for this purpose). For example, as Patrick suggested, we don't want to manually maintain a README file that links to papers; we want to generate that content. After the session I asked the assembled presenters whether they knew of ways to integrate Jupyter notebook content like what we have in our DemARKs and REMARKs with the documentation; for example, if Sphinx could present not only the docstrings, but also some index of where the object in question has been used in practice in Jupyter notebooks.

Connected to this, both the RFE presentation and the documentation presentation advocated the integration of content with Zenodo, Zotero, and the cff file format, and said that their technology for doing that was available and open source. A key seems to be to use the CFF file format because that is something that Google knows how to read and uses in its indexing of sites.

The session organizer promised to make sure that the presentations were all posted, but I don't think that has happened yet. When they are, I'd like you to scan them and then we need to have a long talk so we can process them together. Maybe we can find time to do this at SciPy.

Some further specific responses:

  1. There are two reasons to "hide" material:
    1. It needs to be kept private (like the QuARK content with answers)
    2. It is internally useful (like, "for PASC19 I want to start with Matt's presentation at SciPy last year and adapt it") but would be distracting clutter for users of the site. It's not particularly useful to them to be able to see all 15 versions of a particular presentation that we have given, but it is is useful for us to be able to retrieve that history.
      • I'm not (yet) persuaded that the value of exposing the "-make" material (basically, reduction of the number of private repos), outweighs the cost (clutter and confusion that might be caused by "too much information."). I'm not saying I couldn't be persuaded, just that it does not seem obvious to me. Probably it would be useful to look further into how QuantEcon does this with the generation of its content from rst files.
  2. One consideration suggests that we should have even fewer repos than you have proposed: Some content may be appropriate for more than one of the categories. For example, a good number of the notebooks in the DemARK are both "instructional" in the sense of being potential fodder for introductory PhD courses teaching the substance of economic concepts, and "tutorial" in the sense of teaching how to use the Econ-ARK toolkit. Similarly, some of the DemARK content is basically explicating key ideas from some REMARKs. This cross-pollination is the reason I am keen on having some kind of metadata system based on tags, so that the econ-ark.org website can find material related to a particular tag across multiple repos. That probably does not mean that we need to pursue a "one repo to rule them all" strategy but I think it DOES mean we should try to work out some system of tags that could serve our purposes before making a decision about how to structure the repos.

A final thing that I haven't figured out is whether (and if so, how) to include material that does not make direct use of Econ-ARK but is Econ-ARK adjacent. For example, I've developed a number of Jupyter notebooks for my first year PhD course, and will be developing material to present at my course in Budapest, that does not use the HARK toolkit at all. But in teaching, that material is interwoven with material that does use Econ-ARK tools. This material is now part of what is in the TITLARK but that is not a particularly good name for it. I'm still pondering this.

@shaunagm
Copy link
Author

Do you want to just table this issue - and, necessarily, the issue of how Andrij will implement the website-side of notebook discovery - until we have a chance to view these presentations? One option is to email the presenters directly and ask for copies of their slides. Or were the sessions videotaped?

I have some replies to the other questions you have, but maybe it makes sense for me to wait until I read/view the presentations and understand where you're coming from before engaging.

@llorracc
Copy link
Contributor

llorracc commented Jun 17, 2019 via email

@samdbrice
Copy link

Everything I was thinking of is already mentioned above.

It's important to be mindful of your audience on GitHub - developers. A regular user may get to Econ-Ark.org, find out how to install the library, open a few notebooks, and never visit the repositories.

Generalizing the repository names is critical because it's not something you can easily change in the future when you have hundreds/thousands of fork or hardcoded values in scripts/urls etc.

The naming currently being used for the repos make more sense as "pages" or "products" on the website because TITLARK conceptually is not a "repository" but rather a "data product." Ultimately the TITLARK webpage could pull/reference "data" from econ-ark/courses or econ-ark/demos (or wherever) but you have the flexibility of changing the content and presentation of TITLARK without having to spend a lot of time on the backend wondering if a new piece of data belongs in DemARK or TITLARK. And if you ever decide to change the name from TITLARK to something else (or discontinue the data product) you don't have to "break" your repositories/urls because the generic repo names would still be representative of their data/content.

Based on the above recommendation and what's already been mentioned here's what I was proposing.

https://github.com/econ-ark/*-make -> tools
https://github.com/econ-ark/DemARK -> demos
https://github.com/econ-ark/TITLARK -> courses
https://github.com/econ-ark/REMARK -> replications
https://github.com/econ-ark/PARK -> presentations
https://github.com/econ-ark/OverARK -> management
https://github.com/econ-ark/ballpark -> papers
https://github.com/econ-ark/QuARK -> exercises

The less the better, and the more generic the names the more flexibility you have as things change.

@shaunagm
Copy link
Author

Thanks, Sam. I had thought of combining all of the "using HARK" content into one repository - what do you think of that idea?

@samdbrice
Copy link

In principle, I am for reducing the number of repositories.

Which ones were you thinking of specifically and what would the repo be called? (sorry if you already mentioned above)

@samdbrice
Copy link

samdbrice commented Jul 23, 2019

I notice above you mentioned generalized names (Demonstrations, Instructional, etc), but it would help to see your mapping of (current-nicknames --> proposed-names) as that's easier for me to digest since I'm still not familiar with the nicknames.

@shaunagm
Copy link
Author

shaunagm commented Jul 23, 2019

I'm not fussy about the names - your proposed ones are very similar to mine - but more about the general idea of consolidation. Although I'm worried now that that's a non-starter, if having a nested directory structure will mess with mybinder displaying our notebooks.

In terms of which repos specifically, this subset (taken from your names above):

https://github.com/econ-ark/*-make -> tools
https://github.com/econ-ark/DemARK -> demos
https://github.com/econ-ark/TITLARK -> courses
https://github.com/econ-ark/REMARK -> replications
https://github.com/econ-ark/PARK -> presentations
https://github.com/econ-ark/QuARK -> exercises

OverArk (renamed perhaps) would remain separate, and we'd move ballpark (and plutark, which you don't mention) out of github entirely and use an existing paper-storage tool

@samdbrice
Copy link

Looks good to me. Yeah, if nesting becomes an issue for mybinder we can flatten accordingly.

@shaunagm
Copy link
Author

shaunagm commented Jul 23, 2019

Well it's a lot of work to merge 7 repositories into 1, it would be good to know if it's going to be an issue for mybinder ahead of time. If you're up for looking into this, that'd be great, otherwise I'll check myself before implementing anything.

@samdbrice
Copy link

Ok, let me know what you find out - I'll also do some research.

What else remains to reach a consensus on this and get the work done?

@samdbrice
Copy link

Doesn't seem to have a problem with displaying notebooks in nested directories whatsoever https://hub.gke.mybinder.org/user/sbrice-setup.py-9i8djjr7/tree

Although you are still only limited to the same version of Econ-Ark for all the notebooks.

@shaunagm
Copy link
Author

shaunagm commented Jul 23, 2019 via email

@samdbrice
Copy link

Are we clear to execute on this?

@shaunagm
Copy link
Author

shaunagm commented Jul 24, 2019

What would you execute? You are very welcome to create a new repo that copies the existing repositories with new, clearer names and a nested structure, but please don't change anything on the existing repos as there's stuff pointing to them in our website, docs, wiki, etc that we need to change first.

@samdbrice
Copy link

Yes, exactly what I wanted to do. Thanks, I'm on it!

@llorracc
Copy link
Contributor

llorracc commented Jul 24, 2019 via email

@shaunagm
Copy link
Author

@sbrice Did you over go ahead and make the alternate/renamed/consolidated repo as suggested by this comment?

@shaunagm
Copy link
Author

@sbrice can you please let me know either way?

@samdbrice
Copy link

Hi, somehow missed the first comment. Yes, I did create the 'demos' repo that's currently linked with the latest docs in HARK but wasn't sure if y'all decided to go a different direction since DemARK etc were still heavily being used.

@shaunagm
Copy link
Author

shaunagm commented Sep 14, 2019

Well, I don't think any of us noticed that you created the demos repo. I guess we somehow missed it too. :)

Were you planning to do any more work on this? Our roadmap above was to merge all of the following into a single repo:

https://github.com/econ-ark/*-make 
https://github.com/econ-ark/DemARK 
https://github.com/econ-ark/TITLARK 
https://github.com/econ-ark/REMARK
https://github.com/econ-ark/PARK 
https://github.com/econ-ark/QuARK 

Looks like you've put part of DemARK in this new repo - were you planning on moving the rest over once you got a thumbs up from us?

@samdbrice
Copy link

Yes, I can continue the move and finish it over this weekend.

Are other related pieces (e.g. HARK --> Sphinx --> DemARK) fairly stable at this point? I haven't noticed much activity there but we'd have to reconfigure anything setup with DemaARK once everything is consolidated.

@llorracc
Copy link
Contributor

By HARK-->Sphinx-->DemARK I think you mean that the Sphinx documentation incorporates some of the notebooks in the DemARK as examples in the readthedocs autogenerated documentation?

It's been on my to do list for quite a while to identify which notebooks should be so treated. I've kind of been waiting until it is clearer how we are going to go about serving up the notebooks in the future; Shauna and I have a call at 9am on Monday morning about that, so let's touch base maybe next Wednesday or Thu and see if the future of the notebooks has clarified.

@samdbrice
Copy link

Correct. In that case, I'll do whatever I can now and we can figure out next step when we touch base. Thanks!

@shaunagm
Copy link
Author

shaunagm commented Sep 16, 2019 via email

@samdbrice
Copy link

samdbrice commented Sep 16, 2019 via email

@samdbrice
Copy link

I've created the codebase econ-ark/extras to consolidate the following repos:

econ-ark/DemARK is still in econ-ark/demos as I'm not sure if that should be consolidated. My intuition is to keep it separate because that's a high visibility/commit repo and keeping it as simple as possible reduces confusion and improves sustainability.

The following codebases seem to be stale or duplicates - and I've consolidates them into an econ-ark/archives repo:

The following codebases seem empty and should probably be deleted:

With the above proposed changes the GitHub org would go from 24 repos to 8. You can see an illustrated before/after attached:

untitled (2)

@samdbrice
Copy link

As for the private/solutions notebooks that should not be public-facing - I can create an econ-ark/private repo that contains an econ-ark/private/solutions directory but i'm not sure what exactly should be private.

@shaunagm
Copy link
Author

@llorracc - when you get a chance, can you specify for @sbrice what elements of the titlark-make repository need to remain private?

@llorracc
Copy link
Contributor

llorracc commented Sep 18, 2019 via email

@samdbrice
Copy link

Let me start by saying I like the naming convention and believe it should remain.

My recommendations boil down to how it seems that Econ-ARK currently uses GitHub more as a website than as a software repository, which is perfectly fine. I explain further above (Jul 23rd) the drawbacks associated with that pattern but for a more concrete example, you could check out the DemARK-make/notebooks folder. The way to avoid such problems is by storing data and content separately. In the case of DemARK and QuARK it could look something like this (using three repositories):

quark-demos-demark

In the abstract, you're already doing this. The difference is you're using *-make as the level of indirection - which, implementation wise, doesn't solve your data duplication/homelessness problem. Ideally, the implementation looks more like this (using five repositories):

make-demos

(#A) ...which I don't think is ideal because you can achieve the same thing with fewer repositories.

(I didn't have any visibility into the *-make repositories back in July and so my recommendations didn't fully take those into account)

Having DemARK listed adjacent to DemARK-make only in the filesystem is somewhat an anti-pattern and you end up with a situation where the DemARK-make repository is referencing a sibling not tracked in the same version control tree. For example, this old remove_Problems..-fails.sh#line6 script in DemARK-make is referencing something it doesn't inherently know how to track (e.g. ../DemARK). I understand the reason for this is the fact that DemARK-make contains sensitive information and there's no clear way of mixing "private" and "public" repositories (pretty much the first question you asked).

There is a way of mixing "public" and "private" repositories but that only scales if you're separating data and content appropriately (and I wouldn't exactly call it a superpower 😄). As hinted in (#A) above, you can achieve this while also reducing the number of repositories. That starts by consolidating the *-make repositories into something (e.g. MakeARK) that references data (e.g. demos) and content (e.g. DemARK) as sibling submodules within the same version control tree. Within MakeARK is where you store your make scripts that (e.g.) pull from the demos submodule to generate the DemARK submodule. The make scripts should exist within the same repo because they all very much do the same thing. In the end, you're getting a double bonus by (1) reducing duplication of data, and (2) reducing duplication of scripts. Ultimately the implementation would look something like this:

MakeARK

Notice how MakeARK only pulls the pieces it needs from demos to generate DemARK. That's how you can cater DemARK (or QuARK) to different user groups (e.g. Students vs Economist) by grouping the right presentation with the right demos/exercises without duplicating the data. For example, you could have DemARK/For-Students and DemARK/For-Economist with different/evolving layouts/representations (see Jul 23rd).

That's also how you'd go about generating static content for the website.

The way you'd go about implementing this (and to finally answer your question) is by having MakeARK be a private repository only accessible by a select team (e.g. maintainers) and having all the relevant repositories within MakeARK as submodules. The submodules themselves can be mixed in terms of "public" vs "private". Whenever you run the appropriate make script (e.g. remarkify.py) it would pull from the appropriate submodules (e.g. demos and presentations) and generate the latest REMARK (sub)module. Once the updated REMARK submodule is ready to be published you'd move into the REMARK submodule then git-commit and git-push as normal.

I'd actually recommend that you go one step further and have MakeARK (and the maker scripts) be public, and only keep a solutions submodule as private (i.e. within a private repo). That way the maker scripts are open source and can be improved like any other software. When implemented as such anyone can clone MakeARK but they would not be able to clone solutions submodule even from within MakeARK (only maintainers can clone/modify the solutions submodule).

-- I believe the above sufficiently addresses your question and comments although I can go into more detail if you'd like.

-- Below is mostly semantics which is somewhat relevant to this issue but not exactly significant.

I mentioned how Econ-ARK uses the GitHub organization more like a website than a repository. If you visit the GitHub organizations github.com/QuantEcon or github.com/openjournals or github.com/shogun-toolbox or github.com/numpy you'll notice they use common nouns for repositories containing mostly static data and they use proper nouns for repositories containing software, tools, or products.

Part of the reason is that there are programming tools that reference GitHub URLs directly as an "install" target. Most often, from scanning a GitHub organization, you'll find that proper nouns are installable whereas common nouns are clonable. For example, the idea of pip install git+git://github.com/econ-ark/HARK.git is perfectly intuitive but pip install git+git://github.com/econ-ark/presentations.git is not (not to say there can't exist such a use case). For me pip install git+git://github.com/econ-ark/DemARK.git falls somewhere in between and that's why it took me some time to understand how and where data vs. software is organized within Econ-ARK.

@shaunagm
Copy link
Author

@sbrice can we have a call about this? It feels stalled out. I'd like to push forward, but I don't want to just make changes and mess with your plans.

@samdbrice
Copy link

Yes, will email you now to setup a time.

@llorracc
Copy link
Contributor

llorracc commented Oct 24, 2019 via email

@shaunagm
Copy link
Author

shaunagm commented Dec 9, 2019

After a lot of discussion, we've decided that making the Econ-Ark organization's repositories easily navigable is not a priority. We're going to pin the most important repos, and provide good descriptions in our documentation, but we're no longer aiming to make it easy for someone to come to the Econ-Ark org, browse our repos, and understand what's there.

(Context: Chris and Mridul believe the best way to handle remarks involves forking individual remarks to the repository, which is going to make the org so difficult to navigate that it's not worth optimizing the other folders.)

That said, we did tackle some low-hanging fruit:

  • deleted econ-ark.github.io, ballpark-make, and plutark without saving or moving any content (there was none to save/move)
  • moved park-make's contents to a source folder within park, and deleted park-make
  • moved plutark-make's contents to the source folder in park, and deleted plutark-make
  • moved econ-ark-make's contents to google drive archives, and deleted econ-ark-make
  • moved NARK to HARK documentation folder, and deleted NARK
  • updated pinned repositories
  • updated repo descriptions to be more readable

Stuff that we're waiting on:

  • Interark's contents were moved to the google drive, but have not been deleted yet due to internal Github error
  • Chris and Sebastian need to save what they want from remark-make before it can be deleted
  • Chris is going to move demark-make to his personal account
  • HARK-make will eventually be moved into HARK's documentation folder but we need to determine what's happening with the HARK manual first
  • we need to decide about renaming the repos we're keeping to be more readable (for instance, "presentations" instead of "PARK)

@shaunagm
Copy link
Author

We're still waiting on remark-make, but I'm going to close this as done - I've made a note to finish with remark-make at the next meeting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants