For similar reasons, we similarly can't keep making sleep-sort faster by scaling the sleeps down.

Agreed. Just note that it can sometimes (often?) be a good idea to add a small stochastic element to the sleep time, and/or do something similar to the exponential back-off[1] approach that Ethernet (among other things) uses.

[1]: https://en.wikipedia.org/wiki/Exponential_backoff

If anything, people underestimating the ramp up of exponential back off(or not capping it) has left me waiting much longer than necessary

Technically, we can, at least to a certain limit - as long as the total time needed to schedule the sleeps is less than the shortest sleep duration, sleepsort will still work. Though we're just offloading the sorting to the scheduler, which has to implement the sorting algorithm anyway...

``` interval = interval*1.6 # fiboroughly ```

The biggest headache we had was to figure out when everything was up. There simply was no way to detect this programmatically, so the only way would have been to alter the code to give some kind of signal on the console like "System is up." similar to a linux box showing the login prompt. So I resorted to wait loops too.

We ended up canceling the project.

I've been endlessly frustrated by how prevalent "just add a <x> second sleep" is in the Kubernetes world.

Examples:

If you're using IRSA to provide pods an AWS IAM Role, it can take some time on first-run for the role to become usable. So your app has to just wait until it can successfully talk to AWS.

If you're using the AWS Elastic Load Balancer controller, you need to add sleeps during pre-stop for a pod. The controller has to see the shutdown event, then start target deregistration on the ELB Target Group. That's a minimum of 30 seconds, during which you can be getting new connections.

Service meshes on Kubernetes add some fun on both startup AND shutdown.

Linkerd, for instance, will only inject it's linkerd-proxy container into your pod after startup. So it's entirely possible for your main application container to start, begin talking to the world and then a moment later all your connections get reset when Linkerd-proxy takes over the pod's networking configuration.

You can fix this by injecting into all of your containers a linkerd-await utility that replaces the container entrypoint. It will wait until linkerd starts before starting your regular entrypoint. It's basically a more advanced version of sleep.

Shutdown is just as fun - linkerd-proxy will immediately terminate, dropping all network connectivity for all the containers in your pod. There's a fix for this, too, which involves the above linkerd-await and another option.

Actually it's a generalisation of what OP is doing here. Move the listener _out_ of the program and you never have to know if a service is up or not. Connecting to a service just blocks until it's up.

Is then the waiting for the availability/existence of anything a "more advanced version of sleep"?

Is a readiness probe a sleep?

I don't follow at all.

The problem is often relying too much on cold-starts; these polls amortize well in a real system, where you can afford (and want to afford!)

The alternatives have a ton more complexity for not much benefit.

The dynamic is: developers write the the test and it usually works on their machines. If it does not there is a chance that they figure out what's wrong and rewrite it in the proper way. Tests often work locally, where the CPU and disk are nearly 100% available to run them. Sometimes they fail on CI systems. Developers scratch their heads and attempt the easy fix of adding a sleep of 1 second. That almosts always work and it took 5 minutes, code, commit, test run. They know that it stinks but they have stuff to do. Every few months somebody attempts to remove some of those sleeps with mixed and often unsuccessful results. Integration tests are particularly nasty.

When you run the test in isolation, 1s wait here and 2s wait there is not a big deal. But when you run 200 tests sequentially (because some of them don't work when they run in parallel), then it starts to add up, and your test suite starts feeling sluggish.

Why wait in the first place and how can I fix the issue of tests not parallelized

Both should be red flags in a codebase

Re waiting: Test are integration tests that wait for some event. Example: test changes a file on disk, waits for UI to update. We rely on the OS to report file system changes. We don't know how long it takes, because it depends on system load. We could just wait for the next file system event, but we might receive multiple file system events, and since they are processed asynchronously in our app, we don't know how long it takes for the UI to update. We could add some kind of notification when the UI updates to speed up tests, but that would mean adding extra code that might slow down the app just for integration tests.

Re parallellizing: Tests are integration tests that run against different databases. Not all databases have perfect transaction support, so sometimes one test interferes with another (eg. one test creates a table, another test fails because that table exists). So parallelising these tests would require multiple copies of each test database, which would cost more money, and spreading tests over multiple test databases would make the test execution code even more complex...

Go has excellent primitive for this, a channel:

    // Setup a channel. We'll send one and only one messsage over it, then close it.
    // The message will be either an error (if something went wrong)
    // or nil, if everything is fine and we can proceed.
    ready := make(chan error)

    go func() {
        // No matter how this goroutine exits, when it exits the channel will be closed
        defer close(ready)

        // Do whatever we must: open a socket, run a subprocess, or something else
        lis, err := net.Listen(...)
        if err != nil {
            // Let it be known we've failed
            ready <- err
            return
        }
        defer lis.Close()

        // Announce the "parent" goroutine we're ready, without errors
        ready <- nil
        ...
    }()

    // Here, we block and wait for our "child" goroutine to tell
    // us something or at least die and close the socket.
    // You can extend this and e.g. use `select` if you want to implement a timeout
    err, ok := <-ready
    if err != nil {
        return fmt.Errorf("failed to start listener goroutine: %w", err)
    } else if !ok {
        return errors.New("listener goroutine died unexpectedly")
    }

.NET and Rust are the first two I can name off the top of my head.

Just addressing the case of spawning an external program: if you have control of the external program you’re spawning it can be better to pass the listening socket in using the systemd $LISTEN_PID protocol. It’s easy too implement, language independent and you get systemd integration for free.

See also: https://blog.williammanley.net/2014/01/26/Improve-integratio...

i know this is not the best way. but i added a timeout of 10sec. also while asserting i am polling to know if i got a connection.

i understand and i can make it way better. but for now it's good enough. more importantly, it's better not scatter these kind of codes everywhere. also making things explicit is better always. like now server.Start is Blocking. and in my tests i am spawning a goroutine, rather than it doing implicitly.

another thing, testing dependencies. i am going to test concurrent handler. and it's not dependent on tls or tcp. i can just use a fake and test the concurrency. i like this design. scattering these throughout will make every other test integration tests. increasing time to execute and that is bad.

btw, it would have been better if go lang runs tests in parallel.

And I guess I feel that we should be accepting of a little weirdness as an audience, especially considering the quality? I don't have that idea all figured out but the two concepts feel related.

As an aside, don't put everyone in the same bucket, it's really not great for e.g. trans people that don't want to be associated with that kind of tech enthusiasm. This is like the new variant of strict gender roles, not fun.

Thats what I mean, I don't think the criticism is at all valid

I had to open both snippets in two tabs so I could flip between them and spot all the actual differences, which is that 1) the listener creation was moved out of the go-routine, 2) the port was changed to auto, and 3) the sleep call was removed.

Making the server support multiple instances at same time is a distraction, because you could do the same with the old code. The trick here is that creating the listening socket in the original thread means you can also use auto-port assignment and then fetch the port from the returned object.

This would be unnecessary if you queried for a free port first, so perhaps the point here is that an atomic port reserve-and-listen system also prevents race conditions over the port number.

These are all good angles that could be naturally woven into the explanation. Instead you get furries discussing moldy bread, which doesn't tell you anything, but I suppose makes both reader and writer feel like they got something out of it.

Personally, I feel like the post is cheating by suggesting "in order to synchronize a server and a client, just create the server inside the client!" ... ummm. If you could do that, you wouldn't have a synchronization problem in the first place.

"Please don't pick the most provocative thing in an article or post to complain about in the thread. Find something interesting to respond to instead."

"Eschew flamebait. Avoid generic tangents."

https://news.ycombinator.com/newsguidelines.html

If you didn't make everything noreferrer I could just add a "no fun allowed" bit of JavaScript or server-side Rust code to replace the "objectionable images" with random emoji or pictures of innocuous food or something, but I can't do that.

It is possible to detect if the user came from HN: https://news.ycombinator.com/item?id=24429725

I don't think we'd implement technical solutions like autodead comments of class X on site Y. Those things are too brittle and trying to get around the community with technical tricks is asking for trouble anyhow.

How do you arrange a date with your friend in the city? I usually say "let's meet at 17:00 at the main train station, under the clock".[0]

You have to know your counter-party's accuracy. But if you synchronize on any other metric, surely that could also be inaccurate? So how is time any different?

[0]: Note: in Danish: https://www.berlingske.dk/under-uret-uret-skiftes-ud

That's not desynchronization or synchronization. The synchronization is realizing they are both trying to connect at the same time.

How do you arrange a date with your friend in the city? I usually say "let's meet at 17:00 at the main train station, under the clock".[0]

The conversation is synchronization here. Synchronization is unambiguous communication.

It's certainly possible to communicate in a way that appears to be unambiguous without explicitly referencing time, but the time is always implicit. Practically, all communication is otherwise ambiguous.

Everything happens over time

Synchronization is unambiguous communication with respect to some time coordinates.

No, it could mean ordering but that doesn't mean 'time coordinates'.

It's certainly possible to communicate in a way that appears to be unambiguous without explicitly referencing time

That's because time isn't the most fundamental aspect, reading shared information and ordered writing are.

If cars need to merge on the highway into one lane, the space between them isn't the most important aspect, deciding on the order is. There could be a little or a lot of space in between and it still works.

Programs execute over time, but synchronization is not fundamentally about when things execute, it's about the order that they execute. The order is what is important, not specific time coordinates or specific timing.

Just like cars merging, where they go in absolute space is not the fundamental concern, it is where they are relative to each other so they don't overlap.

Synchronization is about serializing events if they have overlap in the data they affect.

It's only temporal because all execution is temporal.

The fundamentals of what you are trying to achieve aren't when something executes, it is the order.

If try to execute two different things at two different times, your synchronization could go wrong because that isn't really what you want, you want a guarantee of the order.

If you try to synchronize by guaranteeing the order that operations execute it will work regardless of what time something executes.

Ordering is what you are actually trying to achieve, timing in ancillary. That's why the title is 'time is not a synchronization primitive'.

Time is a side effect or consequence of the ordering of events, it does not cause, enforce, or predict the ordering.

One thing depends on some other thing. It does not depend on the passage of any particular amount of time.

I am inspired by Smalltalk but with a twist: it is the pattern and sequence of interactions (such as events, states or messages) between MULTIPLE objects - all interacting - that are interesting.

Golang's and occams select operator is powerful for switching between events from different sources.

My idea: I want to be able to efficiently epoll_wait/uring a sequence of steps and state changes between multiple objects.

The idea is complex event processing for POJOs or objects and states.

Epoll for general purpose objects has usecases in efficient and scalable change detection in microservices, graphical user interfaces dirtychecking, job systems and workflow systems.

If you have a pull or push driven system and you want to adapt it, your hand might be forced into a substandard solution to add reactivity to an event or state or event sequence.

In integration testing, I see this problem. You can fire events in Javascript that your test runner can listen to, this is one solution to this problem but you have to build it yourself. You could hook into Redux state management. I have only created APIs for pages for interrogation for use by Selenium. But what I feel you want is state communication that isn't dependent on timing but efficient notification. (I don't want to use Selenium ExpectedConditions Wait for which is what I generally used to fix tests, there's a far more elegant solution)

You can have a descriptor that represents the combination of all that state.

https://en.wikipedia.org/wiki/Complex_event_processing

I think it would be helpful for microservices and graphical user interfaces for complicated user behaviour.

https://news.ycombinator.com/newsguidelines.html

In fact RCU exists exactly because the previously common method of waiting an arbitrary time before deleting a potentially shared reference and hoping for the best was no longer considered acceptable.

They do say "you generally want to use something that synchronizes that state", but then fail to describe exactly which method they selected in this particular example. A single sentence explaining the particular change would've been sufficient.

This is a little thing but would greatly improve the communicability of the post and practices greater empathy for the reader.

Time is fine for synchronizing when all components are actually reading and acting in accordance with a clock, and agree on using the same one (or a well-characterized facsimile of the same one).

Time is used at the bottom layers of hardware. Certain components generate a signal, and expect the response signal to be settled within a certain time.

Even in situations in which there is robust signaling (like a device can indicate for indefinite periods that it is "not ready" and another device will wait), the underlying signals on which that is based have to meet timing constraints.

You have rules like "when pin X is driven high, the data is expected to be present on lines Y_0 through Y_32 within time T_x." If not, there will be garbage.

All notion order in the machine rests on arbitrary timing at the lowest electronic design level.

Oh, yeah, the simple times before everybody learned that time is a local and observer-dependent measure.

Modern computers do almost none time-based at-distance synchronization. About 2 decades ago, when I was studying asynchronous computation, hacking the chip-design process to make it possible to do time-based synchronization of nearby transistors on the same silicon crystal was a very trendy idea. It was very hard, because everything is extremely noisy, and I imagine the idea faded away, because everything is much more noisy nowadays. Doing that on any large scale is asking for everything to break.

For all of the computer-era, synchronization has being done by signaling. The millenia-old tech of doing it by time is just deprecated.

If it's on the line of "signals only exist because we have time", then yeah, duh; and it's such an outrageous misconstruction of my comment that I imagine you meant something else.

The question is actually a rhetoric implying that "signalling" in the chip level involves time, namely, the clock signal that is firing periodically. It is not so much as a signalling as it is like an assumption baked into the circuit design. The circuit must change its state from one state to another only at the next clock signal and not before that.

   usleep(100000);  // give other thread enough time to do B

The fact that digital circuits operate by different parts all being synced to clock pulses, and some parts even count the pulses such that the 7th bit in a byte is actually picked out strictly by the timing of 7 pulses, is pointless pedantics when the article is operating in an entirely different domain which does not operate that way and there is cause & effect and the definition of a service being ready is when the service is ready, not any particular passage of time. The service might actually be ready before you even asked, or might still not be ready in an hour, or might have come up fully and gone back down all within the sleep.

Today the service takes 100ms to start and you have a 150ms sleep. Tomorrow the same service runs on faster hardware and takes 1ms to start, and now has it's own wrong timing assumption where when it starts, it shuts back down if there are no clients (stupid thing for most services to do, but people do stupid things all the time), it starts up and sleeps 50ms "to give time for the client". The 150ms sleeping client doesn't issue any requests, the service shuts back down.

You have to be writing the firmware to a spinning disk to talk about time being the actual definition of anything.

You seem to be talking about "events". We are not talking about events. We are talking about the clock component of CPU chips that keeps emitting a periodical signal.

> The passage of 150ms of time is not what makes thing B clear to proceed.

It absolutely does. Except that in modern chips the time interval is in nanoseconds. Two decades ago, it used to be in microseconds.

Not trying to be snarky but a genuine question: Do you have any background in chip design and fab? What @kazinator and I are saying is not exactly surprising. These things are taught in undergrad or grad-level integrated circuit courses.

If the circuitry didn't wait those nanoseconds before the next state change, it would be making the state change based on intermediate and possibly garbage values which would make the chip behave in a non-deterministic manner.

Again, we are not talking about events or cause-and-effect. We are talking about something that is more fundamental and is baked into the design of CPU chips at the very lowest level.

See also - https://news.ycombinator.com/item?id=36476621

This is what happens when your connection library starts a call-back after it connects; or blocks until the connection is ready; or use some continuation scheme, like futures to reorder your program and only start the next line of code after the connection is ready.

The alternative is what the article does. AFAIK, you can only do it in Javascript, because it's so obviously bad that no other language supports it.

For my example in circuitry, signal based synchronization uses a clock, or a ready signal. Time based synchronization just uses circuits that are ready to read data exactly when the previous step is done calculating it.

But clock is not a ready signal. The clock is just a periodical signal that keeps firing. If you squint your eyes, it is almost like an event loop performing a time.sleep(ns) before deciding to check the state. In the case of chips, the circuitry performs the next state transition after the very short delay decided by the periodical clock signal. The assumption that every pin would acquire a stable binary state before the next clock tick is a fundamental part of a chip design. Without that we'll have non-deterministic and ill-defined chip behavior.

This is a very odd response. Nothing you write here directly contradicts the parent comment and yet you seem to imply that you have a point that is the opposite of the parent comment.

You seem to be going off on a tangent without addressing the main point that chips do depend on a clock firing periodically and that clock interval determining how long the circuitry waits for its pins to achieve a stable binary state before the next state transition. That was the original point of the parent comment and that point is still valid.

- The general point is correct -- time is not a synchronization mechanism

- Flakey tests are bad for all the reasons listed

- Shoddy synchronization makes for flakey tests

The example chosen -- waiting for a test server to be available is probably a huge source of the issues. However, I don't think the example -- really demonstrates how to do synchronization.

My core objection is that it's really relying on the operating system to take care of the synchronization, and that winds up hiding it entirely:

- the dial will not happen until the operating system has competed the `listen` call

- the operating system is selecting a random port at the time of the `listen` call, and reports the chosen port to the user

This is the correct way to approach this problem, but it doesn't really show a proper synchronization. It shows the operating system synchronizing on behalf of the program. It's also an example that probably only shows up in test -- client and server in the same process space.

In practice, many clients and servers may start at the same time in different processes or on different hosts, meaning that the client has no signal at all that the server has become ready. The best thing that can really be done is quickly polling with low timeouts until the server reports healthy.

I'd have liked to see use of a `sync.Cond`, or channel to signal that the operation has completed, and the waiting goroutine being safe to continue. It wouldn't fit well in this example, because, well, the operating system provides those facilities.

Over, because it does come up a lot with the example used, I'm happy there is a blog posts addresses it. Hopefully I'll see less of the problem in the world. I suppose I hope the author writes a part 2, showing other ways sleeps creep in and how to remove them.

either something systematic or a just a simple stochastic scheduler.

idk how common that is these days, but I saw it all over, used basically blindly to "fix" all manner of mystery reliability problems and never not one of those countless examples was that ever actually the problem or the fix. It was always simply failing to close a file before trying to do something else with it from some other context like a system() command. It was used mystic-dead-chicken style even worse that sticking sleeps everywhere for no actually understood reason.