I should just probably say, clearly you haven't seen Common Lisp's defpackage, modules are actually first class objects there and are completely decoupled from the file system.
But most importantly as Barbara Liskov mentions in this video[1] we don't know what is a module exactly or how to use them yet. Which is an specific statement aligned with Alan Kay's famous "We don't know how to design systems so let's not turn it into a religion yet."[2]
tl;dr; 1) Innovation is good. 2) Javascript's module is a half assed implementation of Common Lisp's defpackage. (Don't get me wrong is still way better than Python's abhorrent ninja goto: import)
Research is a bit too strong a word for it but I'll summarize my conclusions.
Most people I talked with seemed used to and very happy with the Python model: modules are distinct namespaces for symbols and their referenced objects. Pretty much everyone I talked to had no difficulty understanding modules, it's always the package management that makes things complex no matter which language it is.
Something I spent quite a bit of time thinking about was whether it was worth reifying that model into the language as first class objects. In other words, should import be a special keyword that makes the interpreter magically bind symbols or a function which returns a regular everyday normal object? At the end of the day, Python's modules are just dictionaries, just like Javascript modules. Should I make that fact apparent or hide it behind language keywords? Interestingly, both languages went in the opposite directions: Python went from special import syntax to allowing you to access modules as a dictionary, while Javascript went from a require function that returns an object to an import statement. I ultimately chose a somewhat weird mix of both, powered by lisp's flexibility.
I also tried to figure out how compiled languages approached modules. So I dug up literature on Modula, Modula-2 and Oberon and tried to figure out how they represented modules. This ended up having a significant influence in my design in the form of isolated modules with export control. Each module contains its own table of symbols and their references. Importing is just setting a local symbol to the value of the other module's symbol, and only symbols in the exports list can be imported. I also liked the qualified and unqualified names: added the option to prefix the module name to the local symbol.
I also thought a lot about how to map modules to the file system. The idea of program folders from Windows has been an inspiration for a long time now. The idea is if the module itself is reachable then all of its submodules are also reachable by the loader. I also think it's important that no file escapes the package directory. Python and Ruby frequently have thing.{rb,py} scripts and thing/ directories side-by-side, I sought to eliminate that for the module's root directory only which results in a main file like thing/thing.{rb,py}.
To enable module-oriented development, I've found the most important feature of the modules and packaging system is editable libraries. Like pip's editable installs and npm link. When I develop a project, I often end up with several supporting libraries. Languages should support linking these local versions to the main project so they can be developed simultaneously.
Lisp is the easiest classic example that took ages to settle on a module standard and has had different competing standards over the years.
PHP and Perl are probably more recent examples comparable to JS here that provided mostly only low level plumbing (include/require) support and nearly wound up with multiple incompatible module loading patterns.
Even C/C++ the "module" system started as a text processing hack in a macro pre-processor that used to be a separate tool (though I don't think any modern C/C++ toolchain uses a separate macro pre-processor, as there are tons of optimization layers now and some of the macro pre-processor knowledge makes its way all the way through to the linker these days).
Historically, modules were always just "smash these two files together in this order" and it was only later that languages started asking hard questions such as "which symbols from this other file are available".
The modern concept of modules as a fixed "shape" of self-describing public symbols is as much a later abstraction out of Object-Oriented Programming as anything else. Our idea of what a module should be in 2020 is much more informed by "recent" notions such as Component systems like COM and languages such as Java and C#.
> The packaging systems people use nowadays over CL are not a part of CL
Packaging systems were never a part of CL.
> The big hurdle in CL modularization is something very trivial: the fact that a file cannot refer to neighboring files easily by a short path. There is no
(load (merge-pathnames "test1" *load-pathname*))
If that's not short enough, define a function L which does above...
> This steers package management toward the external mode, whereby some definition exists outside of all the files and handles their inter-dependencies and the manner of actually locating the groups of files.
This is the way how to do it.
> I fixed this in TXR Lisp
Oh no..., well there have been zillions of similar attempts...
> You don't need both modules and files. Decades of Smalltalk development experience backs this up.
While decades of elisp development shows us that without modules, nobody will want to use your language to write programs spanning more than one file. =\
I disagree. Knowing what format a module is written in is no different than knowing it's API. You can't blindly start using a module without knowing stuff about it and how it's intended to be used
You and I might expect a module to be both of those things, probably based on experience with languages such as Python.
Julia's position so far has been that files and modules are entirely unrelated to one another. Modules are namespaces, and files are messy things on a disk. Hooray! You can organize your code however you want!
The problem is that I don't want to decide how to organize my code. I want to write the code to some best practices that have been thought up by someone who deeply understands their implications. Julia doesn't have these yet. I hope that it does one day.
I don't think he's conflating modules with objects. He's saying that if you have (class-based) objects, you don't need modules because object classes already provide superset of the functionality of modules. In particular they provide open recursion, which is something he thinks you're going to want to have anyway.
There’s a spectrum of what different programming languages call “modules” and what they use them for, with languages like ML having “true” modules in the conceptual sense, where a module is a unit of abstraction, the way that classes and interfaces in many languages are units of abstraction, though when you dig into it, I believe modules are more fundamental and more powerful… then on the other hand you have the situation of, “Programs are collections of text files, people don’t want to put all their code in one file, let’s call a file a module.”
Point being, some ways that programming languages use the concept of a module are deep and let you do things you couldn’t otherwise do, some are more dispensable.
Then there’s Smalltalk where I believe programs aren’t collections of text files, you actually browse your code in a sort of code browser and it’s stored as runtime objects in a virtual machine image, basically!
Then there’s the matter of how code is released and distributed… packaging and libraries. (In practice, the words “module” and “package” are used in overlapping ways by different languages.) Are the single-function modules/packages of NPM a good thing? In practice it doesn’t seem that way.
It’s sort of analogous to trying to “giggify” all the jobs. Every individual function outsourced.
By module system, do you mean the ability to do something like "import sys; sys.function(blah)". As in having namespaces? Julia obviously has modules..
I was confused by rusts module system too, after having used es6 modules in js and python but then I realised the module systems couldn't be compared.
In dynamic langauges like python or js u can individually use modules but in rust everything has to link back to either main or lib if I'm not mistaken.
This was what enabled me to get rid of preconceived notions of how a module system should work.
As a matter of fact _module_ is a pure theoretical concept. The fact that we’re discussing at least two different implementations of modules alone proves that it’s at least more abstract than _file_. It’s certainly not strictly related to Node, plenty of languages (and their associated runtimes, if applicable) have modules. Further, it wouldn’t matter what a specific implementation of modules within Node is, nor would 14 years of experience with Node.
You’re missing the point entirely, perhaps intentionally? If you’re using modules you’re already using a “build step” - code that runs before your actual code. Your “perfectly working paradigm” is only such if you intentionally disregard the core reason for modules: modularity.
I actually the module system is one of the few exceptions where it's actually poorly designed. It could very easily have a 1:1 mapping to the filesystem with no need to declare modules to use them. Lot's of people apparently don't like that idea, but most other module systems do it that way, and IMO it would be a lot more intuitive.
But most importantly as Barbara Liskov mentions in this video[1] we don't know what is a module exactly or how to use them yet. Which is an specific statement aligned with Alan Kay's famous "We don't know how to design systems so let's not turn it into a religion yet."[2]
tl;dr; 1) Innovation is good. 2) Javascript's module is a half assed implementation of Common Lisp's defpackage. (Don't get me wrong is still way better than Python's abhorrent ninja goto: import)
[1]: http://www.infoq.com/presentations/programming-abstraction-l... [2]: https://www.youtube.com/watch?v=oKg1hTOQXoY
reply