Common Lisp usually stores code in form of normal source files that are then possible to load into a clean-slate Lisp image. It's possible to dump images and restore them, but it's not the norm of working with CL.

Unlike in Smalltalk, I currently know of no tools that allow one to easily edit source code of functions/methods that have already been compiled into the system, and therefore the dependency on the filesystem source files is heavy and immediate.

AFAIK dumping images in CL is mostly used for shortening load times by preloading code and data, and for application delivery, but not for live editing support.

Lots of Common Lisp systems support that in some way. Those record the location of the source code together with the machine code. These locations are stored in the image.

For example the standard CL function ED takes a function name and in many Lisp systems this will edit the function in some implementation specific way. In Emacs one uses M-. to get the definition of a function. In those implementation it will ask the Lisp for the location of that function source code.

The one system that worked similar to Smalltalk is Interlisp. See https://interlisp.org . It recently has been open sourced and is in the process of making it more accessible. It's a glimpse into an alternative world of computing from the past.

I also know that many implementations store the forms in FUNCTION-LAMBDA-EXPRESSION, but as I said, I know of no easy way to edit these in-memory forms either. Interlisp has had a structure editor to work with those, but that utility was not ported back into the CL workflow. I hope it eventually will be, with Interlisp being open sourced now.

Just make source available on the file system. Mount it. Copy it.

That's why Common Lisp has logical pathnames. Typically one uses logical pathnames in source locations. When I set up my application on a machine, I define a translation table to point to the source code.

One can also just use standard pathnames and mount the source code to a standard path. That's how one also has set up Lisp systems in clusters. Every machine mounts the source in a standard path (or a logical path) and a client system will then have access to that source code. The image still has the recorded locations.

Btw., even Smalltalk stores its source code not in the image. The Smalltalk source code is stored OUTSIDE of the image in the file system and it has to have the files available and needs to know the location of these source files.

See: https://squeak.org/downloads/

    The Squeak/Smalltalk programming system consists of three parts:

    * a virtual machine for your platform,
    * both image and changes files of a particular version, and
    * a sources file for the particular image file.

Smalltalk usually can decompile byte code, so one can edit code which has no corresponding source code, with some loss of original source information.

In general though, there's little good reason to edit Common Lisp code in core. Common Lisp is a language with structure at the character level (reader macros); reading in a form loses all this. This is unlike Interlisp, where everything, even comments, were s-exprs and could be edited in memory with sedit.

It is all a matter of tooling.

I think the point of the article is that tooling can only get you so far. The abstraction they attempt to provide is leaky if the support for this style of development isn't firmly designed into the entire system.

I have seen very few people actually using these kind of workflows back when Smalltalk and Lisp were more relevant (I used Smalltalk/V back then).

It is like using gdb, many don't go beyond step, next, print, run, breakpoint and discover how powerful it actually is (same applies to other debuggers).

I take you at your word that you didn't use the sorts of workflows I've described, but I did, and I still do, to the extent that I can, and it was common place among, for example, the programmers working on the bauhaus OS or the ones working on the SK8 development environment.

We kept images around as a matter of course for various purposes, for example.

In an ideal repl-driven world you could write the test in the repl entirely and commit it to disk once you're ready.

But AFAIK neither provide the feature described in the article where you can continue running after an unhandled exception.

Whenever I think about it I can't quite put my finger on exactly what Python would need to make it the same. It's something to do with the way imports and namespaces work, though.

Python has a fundamentally broken module system, and Python 3 did nothing to fix it:

https://nedbatchelder.com/blog/201908/why_your_mock_doesnt_w...

https://docs.python.org/dev/library/importlib.html#importlib...

What would be very helpful next would be a video comparison of writing a program that is just complex enough to not be trivial or too artificial, first using the common approach, and next using repl-driven approach.

Your choices are:

* Common Lisp: install emacs, SBCL and set up SLIME,

* Clojure: install emacs, Clojure and set up CIDER,

* Scheme: install emacs, racket (or guile) and set up Geiser,

* Emacs Lisp: install emacs.

You'll notice that they all involve emacs. There are probably other ways but you want something that is quite tightly coupled, which all of those emacs packages provide. REPL driven programming is better served by an editor like emacs rather than vim. You want something where you can quickly write/edit code in one buffer and send it to the REPL with a couple of keystrokes. The lower the "cost" of this operation the better; it should be as easy as typing (as it becomes that frequent of an operation).

To help you choose: Common Lisp and Clojure are the most practical. They are both general-purpose languages with a wealth of libraries available. Common Lisp, as the older language, has far more literature available including some of the best programming books ever written. Scheme is the most beautiful language and has one of the best textbooks ever made: Structure and Interpretation of Computer Programs (SICP). Emacs Lisp is the most fun, practical but also the most quirky. Luckily, learning any Lisp will put you in a better position to learn any other Lisp.

Emacs Lisp is fun because it presents the most exciting part of REPL based programming: hacking a live, running system. Most of the time you run a lisp instance just for the purpose of development, but if your program is working and doing something, then why not hack on it while it's running? There's no difference between running a REPL in a "development" instance and a live, production instance. When you hack on emacs you are doing just that: hacking a live, running instance. It's not often you get to use a tool to hack on the tool itself while it's running.

Whatever you choose, good luck on your journey. I truly envy those who have yet to experience the beauty of Lisp programming. It changes you forever.

In the same way there are guides that teach TDD, I would enjoy seeing a RDD (repl-driven ...) tutorial.

Because one is a 30 minute endeavor and the other is a weeks/years endeavor.

I have never experienced such a productive programming environment. https://twitter.com/tomlarkworthy/status/1345321532650905601...

For there I can change variable at runtime. Change the implementation and have the tests auto run. Correct the tests, set a breakpoint in the test or implementation with "debugger;".

I think it might be more productive than smalltalk because of the spreadsheet-like reactive recomputation. Plus it supports real markup as inline documentation.

[1] https://observablehq.com/@tomlarkworthy/rate-estimation

I'm thankful for this article shedding light on what it really is. It's sad that most programmers haven't experienced.

You cannot do everything from the Clojure repl in the way you can from a Common Lisp repl or from a Smalltalk worksheet.

I don't know that the Clojure language design forbids it; you might, for example, implement Clojure on top of a Lisp or Smalltalk environment and hook up their tools to Clojure through its interop. That might work, and ruricolist has been working on such an implementation called Cloture:

https://github.com/ruricolist/cloture

But existing Clojure implementations don't have the full set of tools.

Python doesn't have that, and it's mentioned explicitly as not having that.

It's not about "existing in the real world" or "real enough to make money with".

That means the code you enter at the prompt is converted from a String into literal data that can be evaluated (read). The data is then evaluated to produce a result (eval), and the result is then printed (print).

If that's not what your REPL is doing then it's not a repl.

That is not what's going on in e.g. python or ruby "repls". There is no "read" step here converting the text of the program to data. The program text is simply passed to an "eval" function that produces the result directly.

For example we can write a macro in Lisp and play around with it giving it code as data and see the result as code as data.

    CL-USER 1 > (defmacro while (condition &body body)
                  `(tagbody start
                            (if (not ,condition) (go end))
                            ,@body
                            (go start)
                            end))
    WHILE

    CL-USER 2 > (setf a 1)
    1

    CL-USER 3 > '(while (< a 4) (print a) (incf a))
    (WHILE (< A 4) (PRINT A) (INCF A))

    CL-USER 4 > (macroexpand-1 *)
    (TAGBODY START (IF (NOT (< A 4)) (GO END)) (PRINT A) (INCF A) (GO START) END)
    T

    CL-USER 5 > (pprint *)

    (TAGBODY
     START   (IF (NOT (< A 4)) (GO END))
             (PRINT A)
             (INCF A)
             (GO START)
     END)

    CL-USER 6 > (eval ***)

    1 
    2 
    3 
    NIL

Common Lisp source code (and the source code of its immediate ancestors) is not made of text strings. It's made of S-expressions, which are made of cons cells, symbols, numbers, and so on.

A text file of "Lisp source code" does not actually contain Lisp source code. It contains a text-based serialization of Lisp source code. Other serializations are possible (and there are things you can do in a Common Lisp repl to see some of them).

The "read" in "read-eval-print" means "deserialize the text into the source data that it's meant to represent".

This point is not trivial pedantry because the full power of the Lisp language is available to the read process, and can be brought to bear on how reading is done and what happens when you do it. Compilers for other languages certainly do read text strings and convert them into tree structures and so forth, but the difference is that those data structures are private to the compiler; the data structures that Lisp reads into are standard parts of Lisp's public API, as are the read function, the compile function, the eval function, the print function, and so on. It's all on the table for you to work with.

The same is true of the disposition of the s-expressions produced by the read process; you have an opportunity to bring the whole of the Lisp language to bear on those s-expressions before they are ever passed to (compile or) eval. Then, once again, what eval produces is S-expressions, and those, not strings, are passed to the print function. You once again have the opportunity to intervene in the process that produces the text serialization.

It so happens that I've spent the past six months working on an AI machine-control system written in Common Lisp, and every one of these capabilities was an important part of the work we were doing.

As a long-time but now lapsed Lisper, I never understood some people's elevation of the REPL. A command-line REPL is a poor man's development environment. If you're doing interactive development in Lisp, you'll be far more productive if you use a normal buffer with eval-defun, eval-buffer and friends. When debugging, you'll primarily want auto-updating watch expressions and object inspectors. And when you do have good use for a REPL, you'll still want it to exist within a proper buffer with persistent history, inline object inspection, etc, instead of the low-effort rlwrap terminal experience which usually passes for a REPL.

All of this holds doubly true for Smalltalk environments.

The REPL, often called a 'Listener' in Lisp is an explicit dialog with the system, where the dialog is visible and parts of the dialog can be reused and inspected. Also the dialog can be suspended temporarily and we interact with the running code itself in break loops until we resume the original dialog in some way.

This usually is the horror for people wanting predictable code - where in an interactive Lisp, one can change code interactively at runtime.

Long ago, I was doing Objective-C development with an embedded F-script REPL, hot code swapping, and XCode's built-in debugger. It was very nearly as nice as Smalltalk in many ways, only without the whole IBD situation.

By "repl", I do not mean the window or the buffer or the shell program. I mean the loop of: read an expression, evaluate it, and present the results, in the context of a runtime that is designed to comprehensively support it.

Moreover, you can have all of the tools that you listed and still not be working with a properly-designed repl-driven environment. For example, I built a 3D interactive environment on the JVM that worked quite well--it launched into the 3D environment ns no more than a second or two, and could build whole procedurally-generated scenes in a few seconds. It supported networked multiuser interactions. I could start it from a repl and dynamically alter scenes and objects and their behavior in the environment by talking to them.

It still wasn't a proper repl-driven environment because the underlying runtime could not correctly handle dynamic redefinition of classes and methods. That meant that if I decided that I needed to change a representation or something, I had to kill the environment and rebuild it. It meant that there was always a gratuitous barrier that I might run into at any moment. It meant that some abitrary set of things I was working on was always on the other side of that barrier.

Contrast that to working with, for example, SK8, where I could redefine absolutely everything in the environment (including, for example, the system-level procedures used to draw window frames) without ever restarting the code that was under development.

I have never found the REPL (the Python/Ruby REPL which is not a "real repl" according to this article) to be that useful beyond quickly playing with API of a library with sample data and getting a feel of it and its return types. So this version of repl described in the article does sound interesting.

I’ve always liked the python repl, breaking out a django repl and being able to manipulate the db via the api always felt like a super power to me when coupled with the expressiveness of basic python but last year i started dabbling with Clojure. My first “real” repl driven development experience.

2 things stood out for me:

1. You end up with some genuinely useful executable documentation and it just makes sense to me to preserve some of what i did in the repl while developing. I saw this first here: https://github.com/stuartsierra/component/blob/master/dev/ex...

2. Integration with your editor is essential - having a keystroke to send a form to the repl or evaluate in-line is a real productivity boost. It’s like writing code while an intelligent debugger is permanently live. You don’t need to wait for a compiler run then parse the feedback, it’s just instant in-line runtime feedback. “Paredit” is really sweet. I used calva in vs code for this and tbh it’s as attractive as the language in some ways. I have no idea how to meaningfully apply those concepts to something like python or javascript though. They just don’t lend themselves to that kind of structural editing.

Also, Forth sports a repl.

While not as great as working with a proper Repl-oriented language (as the article explains), this is still so much more pleasant than having to re-run the whole program every time you change a function.

[1] https://github.com/jpalardy/vim-slime

The ability to interactively develop functions in almost any language inside the same environment had been revolutionary for my productivity given my short attention span.

Defining keybindings for splitting the window, opening a terminal buffer and copying a paragraph does not really require a plugin. That's one of the features I like most in Vim, its ability to just map any input to another sequence of keypresses can replace whole scripts and plugins provided you're fine with readability comparable to regular expressions.

[1] https://github.com/jalvesaq/Nvim-R

Python IDE tooling on Microsoft stack.

https://docs.microsoft.com/en-us/visualstudio/python/python-...

Juno IDE for Julia

https://junolab.org/

https://ipython.org/ipython-doc/3/config/extensions/autorelo...

[1] https://www.youtube.com/watch?v=N1oMRw04W3E

edit: fix typo

[0] https://skyalt.com/blog/repl.html

  (when (in-state-i-care-about)
    (break))

It's not exactly a repl, but it shares similarities to the idea of interacting with your source code itself via software, not just by typing bytes into a file.

Testing Forth and ASM code snippets in the REPL before committing to them helps eliminate those nasty assumptions about what "should" work.

In the late 1980s I had a group of friends at Apple that included Smalltalk, Lisp, and FORTH programmers. We certainly found plenty of things to admire and attempt to steal from one another, and everyone accepted the basic goodness of building systems by engaging in conversation with them as they run.

Lisp and Smalltalk cross-pollenated each other more than each did with FORTH, but maybe Slava Pestov's Factor is a glimpse of what you get if FORTH is more in the mix.

I find it better to think of Forth as a clever Macro Assembly language for a two stack VM.

When used as designed, one does not actually program in Forth. You first write a tiny "language" and program the App in that.

Not a popular choice today.

Around the time that MacFrames morphed into SK8, Apple hired an arcade-game programmer named Dave Vronay to work on the SK8 graphics subsystem. When I first met him, he was over the moon about MCL and how great it was to write assembly code in a repl and run it right now.

"Eric Bier Demonstrates Cedar"

https://www.youtube.com/watch?v=z_dt7NG38V4

"Emulating a Xerox Star (8010) Information System Running the Xerox Development Environment (XDE) 5.0"

https://www.youtube.com/watch?v=HP4hRUEIuxo

"Documents as User Interfaces Video Demo"

https://www.youtube.com/watch?v=0-_zVkrWCOk

"SYMBOLICS S-PACKAGES 3D GRAPHICS AND ANIMATION DEMO"

https://www.youtube.com/watch?v=gV5obrYaogU

"Alto System Project: Dan Ingalls demonstrates Smalltalk"

https://www.youtube.com/watch?v=uknEhXyZgsg

"Action!, the worlds first dynamic interface builder - 1988" (Interface Builder percursor, written in Lisp)

https://vimeo.com/62618532

"The Interlisp programming environment"

http://larry.masinter.net/interlisp-ieee.pdf

And the cherry, how Lucid used Lisp ideas for their Energize C++ IDE, including an image based format for AST storage

https://www.youtube.com/watch?v=pQQTScuApWk

https://www.dreamsongs.com/Cadillac.html

I find superior languages like Lisp or Standard ML most enjoyable in text files.

- macOS, Windows, Android, ChromeOS

- InteliJ, Visual Studio, XCode/Playgrounds

as well as the whole series of things that Tudor Girba & co. are doing with Glamorous Toolkit https://www.youtube.com/channel/UClLZHVq_-2D2-iI4rA2O8Ug

I'd still recommend folks use Pharo over GToolkit for most things, as GToolkit lacks a bunch of polish and documentation (not to say that even Pharo documentation approaches anything of the quality of the Rust and Python ecosystems).

This notion if “correct” is “finished the current example as intended”. Running a suite of unit tests would give me more confidence.

Even that is only for programming in the small and that is comparatively easy to programming in the large. I don’t see how a REPL would help there.

That means a REPL makes easy things easier and hard things are unaffected. Does not sound like a competitive advantage to me.

Because the right way of programming in the large is programming in the small and combining things (e.g. functions, components, classes, whatever, etc.), not building some huge monolithic monstrocity (this is orthogonal to monoliths vs microservices btw).

Plus, this "programming in the large", when it gets to, say, 10.000.000 lines of code will still have bugs and behavior you need to check/search for/and fix in individual parts, and this "restarting/dynamic change" REPL will still be a great tool here.

You actually can run a bunch of unit tests in Smalltalk with a single-click, get results, and be dropped into a full authoring environment when tests fail, with the bonus that you can just fix the code and resume execution to have the test go green. Modern Java IDEs can probably do this (I've no experience with Java), but you've to understand that this was possible in Smalltalk several decades ago.

When I use Smalltalk or Lisp, I don't miss pry or the other tools I have when working with Ruby.

I do like Ruby, though. I probably like it better than any other language that isn't made of s-expressions. (Well, I like ML a lot, too.) But the whole time I'm working with it it just makes me want to write another Ruby implementation, but this time on top of a proper interactive runtime.

There is one, actually: MagLev, which is built on a Smalltalk runtime. As far as I can tell, though, it's dormant.

It works by sending highlighted text to vim's terminal buffer. I wrote a short blog post with a demo and the vimscript code here:

https://mcastorina.github.io/posts/vim-repl-driven-developme...

https://github.com/jpalardy/vim-slime

https://vimeo.com/230220635

The author uses the Clojure REPL to walk through the process of developing some non-trivial functionality (calling an API and parsing the results).

It’s a good intro to what it looks like to interactively build a program while it’s running.

Personally, I’ve found that having such a tight feedback loop makes development a lot more enjoyable.

I'm slightly disappointed that it's already something I do day to day. I had hoped that the power of the REPL wasn't overstated.

Most developers are using languages where this sort of thing isn’t possible, and for them, experiencing a REPL driven development flow can be an eye opening experience (even if it’s just to add it to their tool box along side more common approaches like attaching debuggers and using TDD to shorten the development feedback loop).

I don’t know enough about Elixir to understand how your approach is the same/different than using something like a REPL with Clojure, but I did come across a pretty interesting discussion on the topic:

https://elixirforum.com/t/what-do-you-all-think-of-clojures-...

TL;DR - you can accomplish something similar with Elixir, but the underlying technical details are different.

Which language can't do this?

That’s just one characterization I came up with off-hand and it may be inaccurate. I’m mostly trying to paint a picture where tools affect the thinking of the programmer.

What are people’s thoughts on this? Does using a REPL or not using a REPL change your thinking and how so?

Amusingly, to me, this is something people describe as the difference between having to submit punch cards to (or schedule a job on) a mainframe versus having a compiler on your own machine. That having such quick access to the compiler would lead to the "throw stuff at the wall" approach.

IME, faster feedback loops do increase the "throw stuff at the wall" approach, but for those of us who still (mostly) sit back and think, it's an enabler and not (just) a crutch. I can get in the flow much better in a language like Lisp and stay there. If, for instance, there's some confusion in my mind about how a function or data structure works, in C++ it takes me longer (more of a constant factor longer rather than orders of magnitude longer) to write something and test it out (assuming documentation doesn't clear it up for me). But while the time to explore it isn't huge, I've been pulled out of my focus for longer. With Lisp, I can test it in seconds and get right back to whatever I was doing.

But also, when I don't have a REPL, I break my programs into smaller programs (libraries/modules) which can be composed into the larger program I want. This lets me, partially, recreate the REPL experience. Using a test runner (a proper one or an ad hoc one) and a set of small CLI apps that let me use the smaller modules directly I get something approaching the feedback speed of the REPL. With fewer dependencies for any part under test (or as a CLI app), I get much faster compilation speeds vs needing to recompile a much larger program.

The other way around is true, too, of course.

But the intensely-interactive style of programming-by-teaching is the less common approach. In pretty much every thread I've been part of about the topic there have been commentators who've said they just weren't even aware of it as an option, who had no idea that full-featured repl-driven environments had ever existed or were a possibility.

What are the odds that such an obscure approach to programming has already attracted all the programmers who would benefit from it? Long odds, I'm guessing.

So my guess is that it's worthwhile to point out the option for the sake of people who would benefit from it but don't know they have the option.

For my benefit too, of course. I want more people to know about it, because that increases the chances that demand will rise. Rising demand increases the chances of greater investment in those kinds of tools, and reduces the chances of their extinction.

Is it common to write Clojure or other REPL-capable languages in a more "traditional" manner (like how one would write say, Java or C)?

I don't know of anyone who is experienced in Clojure and works in a "write-compile-test" way like you'd do in C or Java. While it's certainly feasible, it's not how you're supposed to do it.

Yes, it’s nicer than Python’s. Yes, it’s convenient. No, I never ended up doing my development in the REPL first. Why? Because editing mistakes in a REPL usually sucks, because a REPL is not a text editor.

What I ended up using heavily was REPL to file integration, which gave me the ability to write a function normally, evaluate it in the attached REPL session, and then play around with it in the REPL. This is far short of the “REPL driven development” that’s commonly discussed, and frankly something that’s probably possible with the Python REPL if they wanted to.

Editing data and functions in the REPL is a neat trick, but it’s a double edged sword, because a REPL can crash, and it provides incredibly rudimentary support for diffing current state and migrating changes back to permanent files. I would never start with anything more than a trivial “how do I manipulate this list” in the REPL for that reason. Oh, and we had a lot of issues with REPLs getting into a bad state with Clojure due to multi methods and protocols; if you’re doing your primary work in the REPL, then having to restart it due to it becoming unstable really sucks.

I spent years in Clojure and found that writing code in the REPL just... isn't fun. This has been true for every interactive system I've ever used (including... the shell).

The shell as a REPL for a succinct language is nice for interactive workflows, but for trying to build less ephemeral pieces of code, the editor is the first class citizen I care about. Sending code to a REPL to be interacted with? Cool, and I think this is what a lot of people mean when they say REPL-driven.

I'd imagine convenience around that in a language or its tooling would be what convinces me to do more REPL-style things.

I think this is a huge barrier of entry to languages like Clojure. The syntax and dynamic nature of the language really lend themselves to an extremely interactive and productive workflow, but that is a hard thing to communicate. It seems most people think that when Clojure programmers refer to the joy of the REPL they imagine writing code into Python or Ruby's REPLs (e.g. https://repl.it/languages/python3), but that is about as far from what is meant as is possible. I mean, who would enjoy programming in an environment that deleted your work every time you finished a line of code?

When I do REPL driven development in Clojure, that's exactly what I do and that's what other folks that I know mean by REPL driven development (in Clojure): define types, functions and variables in a source file, often in a (comment) form, eval them one by one, and copy the code out of the (comment) form when it's stable enough. I wouldn't type code directly into the REPL; that's not a pleasant experience in Clojure - but might be pleasant in Common Lisp or Smalltalk for all I know. The process that I and others use in Clojure most definitely differs from the process that the OP describes and I lack enough familiarity with Common Lisp to know if the process is more pleasant in that language. I assume it is and I must make time to learn Common Lisp properly some day.

For any test/scratch code, just create a scratch file outside your project and write the experiments there, eval'ing as you type. That file could even be an "official" unit test you include in the project.

When I'm working, there is little or no migrating of code from the repl buffer to a file because it's already all in a file. I almost always work in a file. I write snippets in a file, send them to the repl with a keystroke, and build up the world incrementally as I discover what it needs to be. As the contents of the file get larger and more complicated, I move things around and organize them. It's a conversation with the repl, not an editing session in the repl window.

I don't consider any Clojure tools I know of to constitute a proper repl-driven environment, precisely because the language and runtime lack support for the kinds of programming and debugging that I've taken for granted for decades. If I can't inspect and edit and redefine everything in the runtime, it's not the full, proper set of tools.

I've written a good bit of Clojure code, and I'll happily do it again if I need to do something that isn't convenient in, say, Common Lisp, but I consider it an acceptable alternative to things like Haskell and Scala and F# and Swift, not an attractive alternative to Lisp and Smalltalk.

I do occasionally need to restart a Lisp environment because of some gnarly breakage I've committed, but it's pretty rare. Moreover, killing and restarting my favorite lisps takes about--wait, let me check--okay, a second and a half to kill a live app in staging and have it back up, fully-functioning.

I also tend to often just reload changed namespaces and resetting the (component/mount/integrant based) application too. Or letting shadow-cljs live reload my clojurescript. But having the REPL directly accessible is still super useful.

You calling anything else REPL is just wrong, misleading and confusing.

It is not “interpreting or compiling a file” if the tool you are using to evaluate code in the file is sending isolated blocks of code to a REPL and not, well, interpreting or compiling the file.

> REPL is opening interactive mode (if the language supports it) and entering "2+2".

No, a Read-Eval-Print Loop (REPL) is not exclusively used by a user typing keystrokes at a terminal; like other terminal software, it is a candidate for being driven by other programs.

> You calling anything else REPL is just wrong, misleading and confusing.

You insisting a REPL isn't a REPL when the “Read” part is reading something other than keystrokes directly entered by a human user is just wrong, ignorant, and confused.

If I'm going to enter "(+ 2 2)" using my normal development tools, I'm probably going to start Emacs and tell it to start a SLIME session with one of the Common Lisps I use. When I do that, it'll create a repl buffer, because that's the way I have slime configured.

I mean, I don't have to load slime-repl. I could just interact with the Lisp's read, eval, and print functions directly from an arbitrary buffer without involving the SLIME repl display at all. But, y'know, force of habit. Also, I do occasionally type something in SLIME's repl buffer, especially if I want to see some big wad of output and don't want it spewed into the middle of the expressions I'm working with.

Most likely I won't type "(+ 2 2)" in the slime-repl buffer, though. I'm more likely to type it into a scratch buffer and send it to the Lisp by hitting C-x C-e. As soon as I start typing Lisp code, I know I'm probably going to want to add some more and maybe edit it and probably send it to the Lisp again. That's just more convenient if I have it sitting in a buffer in front of me, and not scrolling off the top of the repl buffer into infinity.

So am I working in a repl? Lisp is running a loop waiting for input. I'm sending expressions to it. It's reading them, evaluating the s-expressions produced by READ, and then printing the resulting values to a stream I can look at. Sounds like a repl to me. It's what I've always understood a repl to mean, including when I'm building them.

Does it count as working in a repl when I use a keystroke to send "(+ 2 2)" to the Lisp from the scratch buffer, or does it only count if I actually physically type the text in the buffer where the Lisp displays its prompt? What if there is no prompt? What if I start Lisp and SLIME but don't load the slime-repl extension? Does that mean that the loop that is reading, evaluating, and printing stops being a repl?

I'll probably save my scratch buffer to a file at some point, if there starts to be enough text in it that I think I might forget some of it. Does it stop being an interaction with the repl the instant I save the buffer to a file? I can save the slime-repl buffer to a file, too; does it stop being a repl when I do?

What about Lisp and Smalltalk environments where the read-eval-print loop doesn't display a prompt? Take a Smalltalk or INTERLISP worksheet, for example. Is it a repl if there's a loop reading, evaluating, and printing expressions, but no prompt? Does it stop being a repl if the incremental inputs and outputs get saved to a file? Does it count if it's not a text file, but the environment automatically saves the state of the worksheet to an image file that it automatically deserializes the next time you start the environment?

I don't think I'll adopt your lexicon, but I am sort of curious where its boundaries are.

The main thing that stands out to me is playing with small snippets of code in real-time as you're writing it. Contrast that with writing a class and methods, then writing unit tests, then running them. There might as much as ten minutes between the time you start writing your code and when you run any portion of it, and always running large chunks of changes.

Test-driven development as a discipline emerged from the Smalltalk community, and I don't think that's an accident. I think it's a formalization of the way that Smalltalk and Lisp programmers naturally tended to work.

I generally start on something by making a naive model of what I'm trying to accomplish and interrogating it. In successive iterations, I modify the model and the interrogations in the direction of what I need it to be, discovering details along the way.

I'd say that a majority of my actual activity is testing. That's one reason I prefer to interact with a repl from a file: because I want a record of my interactions. I want to be able to do them again and again.

The difference between that record and formal tests is a simple matter of copying the trail of my interactions into a test framework, and that, too, is how I normally work: make model; test it with interactions; keep the interactions around for later.

As I make progress, the models turn into data structures, and the interactions turn into functions and tests.

It may also be faster to find an issue in the REPL, using various tracing tools ("hm, this function returns the wrong thing when I feed it X, let us enable tracing on a few things and see if anything stands out", followed by "aha, that other function over there is actually the problem, let me add some test cases, so we don't have a regression there").

None of these are impossible without a REPL, but may require extensive modification of the source and a recompilation to get the same effect (followed, probably, by either converting your tracing to debug logs, guarded by a verbosity flag, or by painstakingly track down each print and rip it out again).

I mean, yeah. You just import the Python module you're writing and you can call any function or inspect any object you want. iPython even has hot reload

Unless you've figured it out? In which case please say, because I want to code python like this.

Specifically I write code in my editor, hit a keystroke and all the code at the cursor gets sent to my running python and evaluated. The editor needs to be aware of python's indenting rules so it can do this, but once again if you've worked out a nice way to do this please say. Minimal keystrokes required would be great =)...

I'm not really requiring that the evaluated output gets sent to the editor, though of course that would be ideal, because in the worst case I can side-by-side the editor and the terminal.

From my perspective jupyter notebooks get kind of close, but I'm not coding in a file, so it's not quite ideal.

...I call this "copy/paste". And since I use X clipboard and a tiled window manager, it's only a couple keystrokes and no mouse. Highlight the code in the editor, "super+<direction>" to switch windows, "shift+insert" to paste.

There's a reason I'm specifying specific actions, I can easily see increasing the cognitive burden breaking the ease of that workflow. You can't tell me that putting your cursor to a code block and hitting a key is the same as selection, copy, swap to console, paste...

I've certainly had situations where I've done what you're describing and then still taken the extra time to setup the workflow I've described above because it's worthwhile when you're trying to debug a knotty problem.

Is there a shortcut to evaluate the code in the correct module/namespace for Python? Maybe to switch the namespace in which the Python REPL evaluates the code you paste in?

Docs: https://code.visualstudio.com/docs/python/jupyter-support-py A short demo on YouTube: https://www.youtube.com/watch?v=lwN4-W1WR84

Here's the fork that I'm currently using https://github.com/grusky/slimux

I use it with python and node.is and MySQL and psql

It's great because it can work with any REPL.

I used to use jupyter notebooks but almost exclusively use slimux now unless I need to some data visualisation or computer vision.

VS code's ipython REPL is quite good and has support for images and graphs.

I will say that the main thing that triggered a REPL driven workflow was dealing with large state. When you do machine learning and you have to deal with large models and datasets, loading them each time you save your code and triggering a unit test really gets tedious. You can definitely do it with caching and reducing the dataset size, but it only gets you so far IMO. There's nothing like being to inspect the state of your environment. Particularly when it takes a long time to create that state.

One important thing to keep in mind though is that it is not as dynamic as Lisp, so things like re-evaluating a class definition does not update existing objects. I end up restarting the python process and evaluating the buffers I'm working on often enough that I've bound it to its own key.

[1] https://pastebin.com/ctz5erD4

I've not used emacs for python, but could give it a spin for this!

Bit unfortunate that you need to restart the process for re-evaluating classes though.

It's one of the main reasons why I love Common Lisp so much.

Still, it's not too bad and IMO a lot better then restarting the process on every single change

For adhoc testing code you can type on the repl buffer.

I can't imagine writing a program in its entirety in a REPL.

For what it's worth, there are some videos around of people actually doing it with Lisp and Smalltalk systems, and pjmlp already posted a pile of them elsewhere in this thread.

I can add a few more:

Kalman Reti walking through some interactions with a Symbolics LispM repl:

https://www.youtube.com/watch?v=o4-YnLpLgtk

Brian Mastenbrook demonstrating Interlisp's SEDIT structure editor in the Xerox Lisp environment:

https://www.youtube.com/watch?v=2qsmF8HHskg

Rainer Joswig (lispm here on HN) showing us a little bit of repl and Zmacs interaction on a Symbolics Lisp Machine:

https://www.youtube.com/watch?v=LIGt5OwkoMA&list=PLN1hNlVqKB...

Rainer again, showing some simple interactions with Macintosh Common Lisp, which was my daily driver for years:

https://www.youtube.com/watch?v=GKG8cJl70mo

Ruby programmer Avdi Grimm shows some things that he found cool about Pharo Smalltalk:

https://www.youtube.com/watch?v=HOuZyOKa91o

Dan Ingalls (one of the original authors of Smalltalk) in a 2017 demo of Smalltalk 76:

https://www.youtube.com/watch?v=NqKyHEJe9_w

There are some other things I'd like to find for lists like this, but haven't been able to. In particular, a good demo of Apple's SK8 would be great.

If you can imagine a full-color Hypercard that could crack open and reprogram absolutely everything on the screen, including the machine code that drew the window system's widgets, all in a repl while the code was live; in which you could grab an arbitrary widget and drop it on the repl window to get a live variable reference to the widget, and then inspect it, operate on it, and reprogram it, again, while everything continued to run; in which you could build new window-system widgets by snapping together shapes and telling them to become widgets; in which you were not limited to HyperTalk for coding and text strings for data, but had a full Common Lisp at your disposal plus a Minsky-style frame system for representing data and knowledge, then you have some idea of what SK8 was like.

Nevertheless I felt it was wrong. The approach of designing a data structure, making a few functions to manipulate it, designing another, making sure they work together, etc isn't really that different in lisp from C++. The loop is tighter when you can interactively test.

Either way you have to think ahead.

I find the other differences to be minor in comparison - to be able to recover and continue execution following post-mortem debugging is amazing.

MATLAB and R both have options to drop you into a debugger if there is an error. Not sure if Python's debugger can do this, but other than that, this style of programming is perfectly doable outside of Lisp and Smalltalk.

Define a function foo() in one module that calls a function bar() in another one, but don't define bar().

Now call foo().

Now, in the debugger that you break into, give a definition for bar(), and resume the execution of foo(). Don't quit the program and restart it; that's cheating!

If that works fine, please tell me what tools you're using, because they'll materially improve my tools for some work I need to do.

Also, try this:

Define some classes and make a bunch of instances of them. Write a few methods on the classes. Start your program running with code that uses those classes and methods.

Now change one or more of the class definitions. Don't stop and reload the program! Again, that's cheating.

When you land in the debugger (you do land in an interactive debugger, right?) use it to inspect the stack and find out which classes and methods are on the stack and what's responsible for the breakage. Ask the Python runtime to show you all the functions, types, and values that are on the stack, and when you find the likely culprit, ask it to take you to the source for the version that's currently on the stack.

Now, while you're still suspended in the call stack, redefine the offending classes and methods and tell Python to use the new ones. Now resume the broken computation. If you hit another problem, use the same tools to find and fix that one, too.

After you've done all that, please let me know what your Python toolchain is so that I can use it, too.

Also, if that's all perfectly doable in Matlab and R, that's good to know. Let me know about those tools, too. I haven't had much occasion to use them, but that might have to change if they work that way, too.

1. Define an S4 class.

2. Create an instance of the S4 class (let's call it A).

3. Redefine the S4 class.

4. Examine A.

R complains that there's an error in slot(object, what), because A is missing a slot from the (re)definition of the S4 class.

This is promising, in that R realizes that A is supposed to be an instance of the redefined class. So far, so good.

What I would expect to see next is that either R automatically reinitializes A with the new slot (it has exactly as much information about the new slot as it had about the old ones when I created A), or, if it wants intervention from me, it presents an interactive session I can use to tell it how to initialize the new slots.

It didn't do that, but maybe I just don't have the options configured properly to make it happen.

Smalltalk and Common Lisp meet my expectations: they notice that A is an instance of a redefined class and they either reinitialize the instance automatically, or they present an interactive session that I can use to tell them how to do it.

It's entirely possible that R has this feature and I'm just missing it because I'm a noob.

With this mode https://github.com/nnicandro/emacs-jupyter one can connect to a jupyter kernel running locally or remote (would mostly prefer SSH port forwarding or kubectl port-forward the remote jupyter server). It makes life so much easier to interact with cloud environment (e.g. spark).

You can get similar integration with ruby and python, but there’s way more friction because that kind of development is not the blessed way and so nobody bothers to make it work.

The reason people like REPLs is because they can experiment to discover the right way to write the code in a very fast feedback loop. This alleviates a huge pain when you are operating with uncertainties or at the edge of your knowledge - 'does this function return null if no matching entries or an empty list?' - etc.

The problem is that the reader of your code is not going on that journey with you. They are coming in naively without knowing all of that experimentation you did. The writer now has the opportunity to create code that "just happens" to work by a magic of coincidences and conveniences based on very subtle non-obvious properties of the APIs and systems they are working with. The reader is severely disadvantaged and will have a much harder time to reach parity with the state of knowledge that the writer had. For example, some property of a function or object may not be documented at all, but through the REPL the writer has ascertained it is true. How can the reader know this?

So its a double edged sword in that the more you empower the original creator of the code, the greater imbalance you create to the later readers and maintainers of the code. Used well and with great discipline I could see it being very powerful and positive force to make better code. But in the worst case scenario, you'll end up with extremely unmaintainable code.

REPL gives you more tools to shoot yourself in the foot, but you're not required to use those. Most programmers can write spaghetti code in many Turing-complete languages - not that they do that all the time.

In other words, REPL gives you powerful tools - which, yes, can be misused. They are still powerful, and can bring success when used well.

After some on the field experience, my conclusion is that this is not a large problem. Yes, you were not there on the initial design and does not have all of that acquired knowledge, but you can always run your own tests and formulate and verify your hypothesis. The REPL is there for debugging too, not only for the original design.

ha ha.

unless it's giant sql with 5 sub queries and poorly formated for bonus points

good luck

I don't see the correlation.

Well, the writer could document their code.

Or, the reader could discover behavior of undocumented code the same way the writer did; presumably the “reader” will also have a REPL available.

In fact, one of the benefits of your REPL being integrated with one's editor, and being in the language of the system, is that the kinds of ad-hoc scripts you write when doing exploratory / debug coding which one would typically discard after use can often times simply be committed, either as utility functions or test cases. Far from pulling up the ladder on devs who come later, having a REPL makes it more likely for them to have access to the same tools used to build and understand the system in the first place.

(There's an aside to be said about building binaries from a LISP compiler, but most binaries I have seen produced by such compilers still need access to the required libraries, which sort of hoses it as a distribution solution. This could be a result of the build system used on the projects I work on.)

And your documentation concern is valid, but I think even worse is the design concern: when designing an API, you have to stop and design it and consider the various concerns. If you develop it as you need it / as you go, it is possible to arrive at sub-optimal designs. Hacking the "next piece" out at each step is likely to lead to slanted designs with poor APIs, requiring future refactoring. You see this with programmers at every level: if you ask them to write a program to do a single thing, then add more (related) capabilities, and continue ad nauseum, at some point the program will need to be refactored. This suggests that, without prior planning, REPL-based design is set up to incur technical debt. Unfortunately, my real-life experience confirms this. Moreover, refactoring something in a REPL is... a chore, to say the least.

Your alarm seems a little overblown. Maybe it reflects a particular bad experience? I've delivered a bunch of products built in the way I prefer to work, and it's been a long time since I've had the problems you're describing. Maybe that's because I ran into such such problems early in my career and learned ways of handling them.

And yeah, if you just iterate and explore, you can wind up in lacunae. You do need to develop some actual goal and a vision of how to get there. Interactive development can be really helpful for the exploring part, but it can't pick your goals for you.

In short, interactive development is a way of working that's good for some cases and some people (me, for instance). It's not a cure-all or an objectively superior methodology for all people or all cases. I don't want it to take over the world. I just want it to continue to be an option because it's the way I prefer to work.

Thanks for sharing your experience amd positivity with the world.

This seems like an incidental property (symptom of poor engineering discipline, which manifests itself in other ways in other development paradigms e.g. lack of documentation from "agile" teams) of the particular applications you've worked with and not an essential property of the (architecture decisions resulting from the) REPL-driven development style. What makes you think that this is actually due to REPL-driven development?

I develop several tools from the REPL and was able to easily convert each one to a standalone tool (when I attempted to do so).

I'm interested in your experience in avoiding this, and the process you used.

Additionally, my development environment (SLIME) talked with my application over a network socket, so there's no linkage between it and the application - at least, any more so than the usual problem of "Common Lisp binaries are hard to make small" but that's not specific to REPL-driven development.

The simplicity of my "solution" makes me think that we might be talking at different levels here, but I can't think of what the disconnect might be.

Also, I do not quite buy the simplicity of the standalone tool conversion; it assumes the relevant functions are naturally well-suited as entry points, including handling malformed inputs, etc., very cleanly. In my experience, many things that make sense at a REPL in a live system need to change dramatically to mature into robust command-line tools.

As for deployment, the interactivity I am describing is exactly the linkage you mention, and it really does come down to shipping a development environment as part of the deployment environment. Shipping a binary with a swank server or similar introduces binary size issues, portability issues (you have to ship dependent libraries along with the binary), and some serious security implications. And while this may be the "lisp way", modern languages manage to avoid these issues just fine.

Actually, isn't it far worse than that? Say you're stopped at a function call, you call it, and it does the wrong thing. You edit a variable that's passed as an argument to the function and call it again. Now it does the right thing. Great! You fixed the bug, right? Except that you don't actually know that your program can actually ever end up in that state naturally.

Where did that variable originally come from? Was it entered by the user? Was it read from a file? Was it from a table? If so, does the table contain the modified value? This seems like a recipe for problems to me. I'm sure it's fine in simple cases where you're just changing the value of a constant or something, but it seems really likely to lead to incorrect reasoning about the code while you're working with it in any but the simplest of cases. Am I blowing this out of proportion?

When you say "Great! You fixed the bug, right?" it sounds like a paraphrase of a passage in my essay that I had some misgivings about when I wrote it. But I thought, surely nobody's going to think I really mean the problem is definitely solved? Surely, people will realize its a bit of hyperbole.

Maybe not. Maybe I should have instead written it more soberly. Maybe I should have just said "Now you've collected a bit of data about the nature of the bug."

For me, the whole point of the repl is that I want to build something incrementally, interactively, building it a little at a time by making small changes to it. I want it in memory, running and responding to my changes as I make them. Put this here; put that there. Change that around. Let me look at it; nope. Put that back where it was. Now add this.

If the repl is stateless then the whole purpose is defeated.

This is about the development phase more than the production phase. Do something like this (directly in the REPL or in a buffer as mikelevin has described doing elsewhere):

  (defparamater *db* (connect-to-test-db))
  (query-db *db* (make-query some-query-description))
  ;; see that it pulls out the desired record
  (let ((record (query-db *db* (make-query sqd)))
    (update-record record)
    (write-to-db *db* record)) ;; update the entry
  ;; rerun the query above to see that things changed as expected
  ;; rewrite that let as a defun:
  (defun make-update-to-some-record (sqd db)
    (let ((record ...))
      ...))
   ;; test that it works
   (make-update-to-some-record sqd *db*)
   ;; see that it does in fact do the same as the let above.
   ;; move that to a permanent source file, and build it
   ;; into the system.
   ;; move tests into test file.
   ;; repeat with next feature/task

NB: You can connect to production environments. You can even connect to live, production lisp images. This could be useful for: profiling real-world activity, debugging real-world problems, applying hot-fixes. But especially that last one should be done carefully, and ought to have been validated using a test environment first.

As you're rummaging around through the dynamic state of the running system, you'll discover changes that need to be made, and you might discover them absolutely anywhere. Sure, you could always kill the running system, make the change to the sources, and rebuild the system, but that's exactly what we're trying to avoid.

Consequently, old Lisp and Smalltalk systems are allergic to restrictions on runtime changes. Loosely speaking, if I find something I can't change while my program is running, that restriction is a bug in my development environment.

Old systems like this will discourage certain kinds of changes because they're usually ill-advised, but will not forbid them, because forbidding them is anathema.

As an example, several Common Lisps implement package locks on certain system packages. A package lock prevents you from changing the definitions of system-defined constructs.

But it's Lisp, so it doesn't really prevent the change. It just makes it more inconvenient. You have to say "Mother, may I?" first, which gives you the opportunity to soberly consider whether making that specific change is really really what you want to do.

Turn the testing code into unit test as I go along.

Rinse. Repeat.

Building a script from the REPL might mean running different parts of the code out of order (such as reloading a function definition). As you mentioned, it's easy to lose track of state, writing 'clever'/unmaintainable code, and forgetting the order and purpose of the code.

I don't have REPL-driven development experience, but I think we are all familiar with StackOverflow. This is a great, great resource, but it lead scores of programmers to the "style" of development, where they mindlessly copy solutions from SO, without understanding of how and why they work.

I guess one can extend the old maxim "there is not now, nor ever will be a programming language that makes it easier to write a good program than a bad one"...

(edited grammar)

There is a use-case for a throw-away interaction with a REPL. For example, how does $builtinFuncX work, or how would $data best be imported into a structure?

A REPL can also be a good initial approach to a more ambitious problem. In this case, a REPL can be good for focus and discipline.

If the second case is going to answer your concern and be constructive, it's necessary to be able to build the code for sharing and cleanly export the code for re-use.

I've had success tackling challenges using REPLs for Python and Perl [1] in both ways. But no tooling is going to solve the problem of a sloppy teammate who claims success just because "it compiles" and "it works on my box". A person who knows how to build good tooling goes further.

[1] https://github.com/viviparous/preplish

That should help with maintenance no?

One big theoretical advantage I see in repl driven development is, I can be sure that _all_ the code has been executed at one time atleast. (In a normal edit-compile-link-test language cycle, I can not be sure of that)

This requires no more discipline than, say, actually committing files to a version control system to collaborate with others. Versus shouting from the hills, "It works on my machine!" and being confused when it turns out that you didn't commit "foo.c" and compiled it manually rather than updating your Makefile and committing both changes so others could use it.

Oh man, you are so wrong about that.

It's worth noting that this style of development also hamstrung quite a few Python web app projects in the late 90s because the Zope application server encouraged the use of this style of development (except through a browser rather than the command line) and beginners eventually got to a point that they were stuck and needed to make the leap to a completely different workflow of development based on files on the filesystem and in version control instead for their code, while content (ie. application state) for each deployment of their app remained in the embedded object database.

A frequent lament was "if you knew I would eventually have to switch to filesystem based development, why didn't you have me start out that way?"

Nothing forced you to a saner development model except for wanting to include 3rd-party libraries and being able to distribute and version your code.

Remember, this was when servers were definitely pets and scaling a web app meant scaling up to a bigger server, not scaling out to more servers. It wasn't immediately obvious (especially to brand new developers) that having all the code for even a simple CRUD web app, which you only ever expected to have a single deployment to a single server, live as pickles inside an object database that had a transactional history of edits, was inherently a bad idea.

A lot of interesting stuff was created that way, and integration with the facilities that were missing such as version control led directly to capabilities like versioning content in cvs and svn (since code was just another kind of content).

Eventually, Zope's so-called "Z-shaped learning curve" hindered adoption and other Python-based web-app platforms surpassed it in popularity.

But if you understood my point about developing new functionality this way, why the non sequitur regarding content?

Y'know what? Nevermind. We're on the same page now.

I don't really see how a REPL makes any difference there.

If these sorts of things aren't documented then not having used a REPL won't save you.

Someone ran a function with a println or in a debugger, figured out the answer, then removed the debug stuff... but did it by repeatedly compiling/running instead of in a REPL.

(Or, my personal favorite... someone didn't bother to run that particular code, but just made some assumptions, so has no idea those edge cases are even there...)

But of course you don't want to do this in the live, production database unless you're a DBA who is doing an upgrade manually for some (probably bad) reason.

So, you can try out your ALTER TABLE commands in a scratch database, but you'll need to save them to a migration script, and test the migration.

It seems like a weakness of this sort of live updating? Sure, you can modify your own running instance, but upgrading the production instance(s) will often still be a lot of work. Migrating a production database schema tends to be a tricky thing.

Interestingly, while Ruby’s default REPL (irb) doesn’t have this, and Ruby’s (and irb’s) designers obviously didn’t think this facility through in the planning stages, Ruby does have an alternate REPL (pry) which has a lot of the advanced REPL functionality irb is missing but also doesn’t have this function built-in, but which does have a separate add-on (pry-rescue) which provides it.

And the compiler messages (once you get the hang of it; and the investment in learning the syntax is so trivially worth it in the long run) are so much richer and more helpful than other languages. So it really does "feel" like a REPL or most of the way there as a REPL (and I have one of the few wikis at our work advocating for Common Lisp that I wrote a year ago, and still actively follow Clojure groups, etc.).

It's just Rust starts and runs so fast (and Cargo makes it easy and clear like Go); so that being able to re-run periodically really fast is like a REPL. Yeah there's no retained state or ability to keep a REPL running for days, etc. (though Tmux.. and nix as a repl.... more on that later), but it's reached a point where compilation is an OOM faster than before, so the loop of: read, eval, print now can include the compiler.

And you get this amazing boost where if you're operating e.g. on large data (if you're a senior engineer, like Dan Luu blogs https://danluu.com/, you're often scanning codebases or log files or other metadata to answer questions); so being able to tweak and re-run even without caching at OOM faster speeds than other languages is wonderful.

I can create complex refactoring scripts in Rust on GBs of data and constantly re-run them until it's just right.

To my knowledge you can learn the ins and outs of CL or Rust; both fairly specific-knowledge-heavy languages at first (Rust with some syntax, CL with its non-uniformity of functions). It just seems like Rust is the future. It sort of "merges" dynamic languages with REPLs and *nix (with tmux) as the REPL. Pretty exciting.

And if someone wrote a CL-like REPL with something like SLIME/SLY for debugging for Rust it'd be kind of game over. And I could see that coming in the next few years (https://github.com/google/evcxr seems cool, but perhaps the dependency on Jupyter complicates it a bit).

E.g.: Imagine you're in a breakloop as described in the article. You find that local variable X is 5 when it really should be 4, so you quickly set it to 4. You find function foo() is not defined, so you define it. You continue and your program works. Great!

At the end of the day, you quit your environment and shut down. How do you ensure your interactive work is not lost and the environment is still what you expect it to be when you start again the next day. How would you compile such a program?

Is there some command that dumps the whole environment to a file as source code? Would you save your REPL history? Would you manually copy/paste relevant bits to s code file?

Also, if significant parts of the source code are written inside the REPL, wouldn't the lack of modern IDE features be a hassle? No syntax highlighting, no code completion, no code inspections etc. Or are there tools that offer those?

So any modifications you make stay in your files, just like you would normally do with any other language. But you write your program while it’s running, and you grow/change it piece-by-piece, until it’s done (by that time, your source code is exactly your finished program).

If you want to re-start your REPL session, you just run your current file in the REPL as a whole, and maybe run some “Rich comments” (from the same file) to set up a limited test environment (to isolate the piece of program you’re working on).

This way, you make use of syntax highlighting and all the IDE features.

Edit: I’ve re-read your example with breakloop — I don’t really have experience with this style of REPLing. I agree that it raises a lot of questions, but thankfully functional nature of Clojure discourages this kind of programming.

Suppose I'm in my REPL and I do this:

  > (foo 'bar)

  (defun foo (symbol)
    (format t "Baz ~A~%" symbol))
  ;; this will print "Baz BAR" in the example

My source file and my Lisp image are both synchronized.

In the case of changing a variable. Say I'm somewhere deep in some code and this tries to run:

  ;; in context x = 0
  (/ y x)

1. Examine the back trace in the debugger to see if I can locate where x got an incorrect value (maybe it was the line above, maybe 10 functions earlier). I make a note of what to fix, maybe I fix it now.

2. Change x to its proper minimum value and resume.

If I was able to trace how x got a wrong value, it's fixed already. If not, I'll have to explore more. But I don't have to restart way earlier, I can just restart at this point (assuming nothing else was broken by x having the wrong value here) and still get the result of my program (assuming x being wrong didn't break a lot of other things). Or I can quit back to the REPL and start exploring the problem.

The reason data scientists do it this way is load times are gruesome for data science work. Sometimes it takes days to process something. Imagine every time you change a line of code having to wait days to see if there is a bug in code or what the output is. Instead we run it in an environment that loads it once and keeps it in ram (a REPL), so when working on code below what is already loaded there are no load times.

There are other kinds of environments though. Sometimes video game devs develop on a REPL where the game is running, they pause it, update some code, and then go back into the game with the updates automatically put in. No load times.

Modern Smalltalks have solved this problem. Smalltalk has the concept of Packages just like Java, and as you go along building your environment, even though you are modifying the Smalltalk image, you can export these packages to plain-text files, and put them in Git, just like any other language. The environment itself supports Git integration (called Iceberg in Pharo).

> Also, if significant parts of the source code are written inside the REPL, wouldn't the lack of modern IDE features be a hassle? No syntax highlighting, no code completion, no code inspections etc. Or are there tools that offer those?

The command-line REPLs that other languages have are NOT what you get in Smalltalk. I believe the author means the entire interactive environment, and the "style" of development is REPL, not the actual UI. The Smalltalk "IDE" is just as powerful as any other IDE, including code completion, automatic generation of certain getters/setters, renaming methods/classes, finding uses, jumping to declarations and even refactoring within methods. The difference between a normal IDE for Java is that this "IDE" is pervasively available, including in breakloops and the debugger. Since the system is live, there is no separate notion of debugging, the debugger is always there, and you can use all the editor IDE features when stopped in a debugger. You no longer have to deal with a crippled debugging environment way different from your authoring environment. It truly is mind-blowing!

I highly recommend giving Pharo Smalltalk a spin (by following their MOOC or similar). This video is also worth a watch - https://www.youtube.com/watch/baxtyeFVn3w

I did most of this year's Advent Of Code in Smalltalk and saved it in Git just like any other language. Someone else can then import it into their image. https://github.com/nikhilm/AdventOfCode2020.

Note that the source code looks very verbose, but you never actually interact with the source like that. The source is just a serialization. Your actual environment only ever shows you UI elements and entire IDE windows describing your classes and individual methods.

The only thing I miss in Pharo is that it doesn't have Vim keybindings :) Apart from that there is a significant lack of OS integration and polish, but these are due to the small community and priorities than fundamental deficiencies.

Yes, you're right: as I've repeated several times, when I say "repl-driven programming", I am not talking about a repl window or a command-line shell. I'm talking about the actual read-eval-print loop, the runtime feature that reads input, evaluates it, and presents results. I'm talking about a language and runtime that is designed to be modified while it runs by evaluating incremental changes offered interactively by the user.

It's not about a prompt in a terminal. It's about a software-construction system that is designed to be modified interactively as it runs. Smalltalk is quintessentially that.

It seems like some folks have encountered a repl UI with a prompt in a terminal and jumped to the conclusion that this specific kind of UI is the repl. It isn't. It's one particular kind of UI for a repl. There are others, and there have been others since forever. For heaven's sake, Smalltalk 76 had other kinds of UI for its repl in, well, 1976.

"Because your language is different than mine, therefore it's not as good as mine."

I've heard people say that Java or C++ isn't a language, because it doesn't have features x-y-z when Racket does. Garbage. Similarly, this guy is saying that because Python's shell doesn't have breakloops, it isn't a real REPL. Also garbage. 99.9% of the time when using REPLs/shells, if something doesn't run, I correct the code, copy and paste it into the shell, and run it again. Breakloops don't exist in the Python shell because they don't need to. Python shell works about as good as Clojure REPL for me.

I get really tired of these elitist holier-than-thou arguments. Functional programming is a thing. Procedural programming is a thing. REPL is a historical name... for an interactive language shell. They really are quite equivalent.

I am not saying that your language or your way of working is not as good as mine. I'm not saying that you or anyone else should discard your ways of working and adopt mine.

I'm not saying that Python's repl isn't actually a repl. I'm not saying that everything needs to have breakloops. I'm not saying that I or anything I like is any holier than anyone else.

What I'm saying is I like a specific style of programming and it makes me happy. I think more people should know about it because it will make some of them happy, too. I would like that.

Besides just liking to make people happy because that's the kind of personality I have, I would also like it if demand for tools that support my preferred way of working increased, so that there are more of them available over time rather than fewer.

And, finally, I like to use the phrase "repl-driven programming" for the style I like because some time ago someone asked about what distinguished old-fashioned Lisp and Smalltalk environments from others that the questioner was more familiar with, and "repl-driven programming" was a phrase that person used. So I adopted the terminology. I think it's a reasonably good piece of jargon for its purpose.

We can sensibly distinguish a repl-driven environment--that is, an environment whose design is driven by the requirements of interactive programming with a read-eval-print loop--from an environment that merely has a repl as one of its affordances.

If the distinction isn't meaningful to you, then presumably "repl-driven" design isn't either, and you're presumably not one of the people I'm talking to.