RFC: yet another local module loading proposal

[Roger-luo]

related issues #4600 and some other long-existing discourse posts and slack discussions. Here is another proposal for them, it is inspired by a lot of previous discussions and languages including Rust, Python & TypeScript

The gripes

I'll pick a few cases explaining why in terms of UX (thanks to Andy Dienes who raised it up in slack)

Gripe 1

currently include does not enforce any structure on project module/submodule, if you have a deep module inside the directly, e.g include("a/deep/path/to/module.jl"), this will make it very hard to figure out where the names are defined at visually. One has to use grep, linter or other external tools to figure out where it is defined, they do not always work.

Gripe 2

names are included in the current namespace without caution, they may conflict, with or overwrite previous definitions. One has to go through the included file making sure every names inside do not conflict with what has been imported.

Gripe 3

to resolve gripe 1 manually, we have to adopt the following module structure, where one at most does include("Foo/Foo.jl") to include a sub-module Foo, and no deeper path. Or include("Foo.jl") to include module Foo, or have to adopt a completely flatten directory structure.

to resolve gripe 2 manually, we have to define a module for every 1-level file/directory that may have a name conflict. And this has to be the same as the file name otherwise it reads inconsistently, which means we will write the same thing 3 times!

Once at the file name, once at the module declaration, and once in the include statement, that gives you a much higher chance of spelling it wrong.

---

The proposal

This is basically adopting the rust syntax on declaring a module outside of the current file.

New syntax: incomplete module declaration

Based on gripe 3, we extend the syntax of the current module declaration, and one can declare an incomplete module, e.g

module Foo

this means wrapping the file Foo.jl with a module Foo and including it in the current scope, thus equivalent to

module Foo
include("Foo.jl")
end

or

module Foo
include("Foo/mod.jl")
end

so one only writes the name twice: once for the file, and once for the module declaration, and it forces one to use the same module name as the file.

The compiler should be able to read these module declarations before including the files so that the order of this does not matter. Thus no need to worry about who gets included first anymore!

This will also force one only include single-path files/directories (if they choose to use it), so we will be able to enforce this consistency on project structure across the ecosystem.

Relations with Python

Python uses files as implicit modules which may look similar to this proposal at first glance, but in this proposal, we still require an explicit declaration of the module, and in using/import statements they still require a valid relative module path, thus things like importing the wrong file by accident won't happen.