I was tasked to integrate Fastly into our infrastructure having not done any configuration with Varnish (VCL) before. ~10k req/s
VCL can be challenging for complex flow control (IMO), but it is made easier with Fastly enhanced/custom VCL modules.
Their documentation was good, and the Fastly support team was excellent. Their sales engineers gave us a baseline configuration that suited our needs and were quick to answer any followup questions.
Haven't noticed any downtime or response delays to date.
At its core Fastly uses heavily modified Varnish 2 with customer's having full access to vcl providing an unprecedented level of control of the request processing and cache behavior. It is extremely difficult, if not impossible, to completely remove foot-guns while retaining this ability. They do amazing thing checking validity of the code and aborting broken requests but it is still vcl and in its core varnish 2 does not have multi-tenancy segmentation.
So that looks like a stock Varnish 503. I'm pretty sure the magic behind fastly is a ton of Varnish cache. What's interesting (VCL can be unforgiving), also that it was a big central stuff up. Which makes me think they have layers and layers of caching in depth and some top level config just blacked out everything. I kind of would have thought they'd have lots of small instances.
Good promo for Fastly! Fastly is a great company - effectively they're Varnish-as-a-Service, which is really a non-trivial SaaS. I hope this brings in some business for them.
>At the core of Fastly is Varnish, an open source web accelerator that’s designed for high-performance content delivery. Varnish is the key to being able to accelerate dynamic content, APIs, and logic at the edge.
Varnish and VCL were the hotness at the time Fastly was coming up so it sort of makes sense, but Varnish also doesn’t support WebSockets, can’t proxy GRPC, etc so they’re very limited in functionality vs CloudFlare.
I doubt they’d build it on Varnish today, but it’s a bit late now since they allow custom VCL (which has now proven to be a terrible idea) and will have to support that for eternity.
They can run two or more serving stacks side by side though if it comes to that.
And to add to your point, they also have a separate process that speaks QUIC. It’s an interesting tech stack with a lot of technical debt.
What Fastly has done starting from Varnish 2.x was what they needed to do, to do what they wanted to do: Build a global CDN business.
There are difference between "running Varnish on your website", "running Varnish for your web-hotel customers" and "running Varnish for some of the biggest web-properties in the world" and software and organizations always mirror each other.
For instance Fastly does not run Varnish on Solaris, FreeBSD, NetBSD or ..., they run it on whatever version on Linux they decided on, and what ever brand and model of boxes they find optimal, and therefore they their own code and changes do not need to be portable and can be tuned with laser-like precision to their hardware and kernel they use.
Even if they threw their codebase up on github, I doubt very many organizations could or would use it, because it is chock full with interfaces to Fastlys business systems, encapsulates their network strategy, and God knows what. (Remember when Y! threw Inktomi over the fence ? Not terribly useful if you were not Y!, it took years to generalize it.)
And you will find the same situation, no matter which other major FOSS based organization you look at, Amazon, NetFlix, the pink-bit-pushers, FaceBook, Twitter, Google ...
You simply don't run a huge company on vanilla software, if nothing else because your geeks can not resist the temptation to improve and optimize it to ease their own jobs.
And just to stake the "Fastly vs. Varnish project" thing through the heart at the cross-roads: Fastly is a major sponsor of the Varnish Cache Project.
awesome. I have been using varnish for a while now and it's just awesome. The VCL configuration works like a charm and is very simple.
ESI is really cool and can be easily integrated as a normal include with the common frameworks. For example, cached file including when developing and ESI tags when production and stagging.
Having ESI is very good to cache and distribute in parallel different parts of the page, that can have different CPU usage. Also it's really good to cache stuff even when the user is logged in.
Agreed, varnish is awesome, we use it on a home-brew CDN and it has been a fantastic experience so far.
Also, the guy that wrote it (Poul Henning-Kamp) is very responsive when it comes to answering questions or fixing issues, they have an IRC channel at #varnish on irc.linpro.no.
Definitely a piece of software worth looking in to when your website gets bigger.
Yeah they run a fork of varnish 2 with inline C disabled, and a number of custom extensions (surrogate keys, tables/dictionaries, etc.) -- as well as likely customizations to support their massively multi-tenant deployment and more
Add in the necessity of porting vcl for all of their customers, and the prospect of upgrading varnish is obviously pretty daunting.
I suspect they'd rather focus on their new webassembly-based configuration solution, rather than try to keep up with changes in VCL
We had really bizzare problems with Varnish ~6 years ago even under mild load (hundreds or requests per second). Like it would randomly let all the requests through to our origin servers, bringing them down. We struggled for months, multiple devops/sysadmins looked into it, couldn't figure it out.
I was spec-ing out Varnish for a project recently and was really impressed with what it could do. The stale-while-revalidate approach (immediately serve data past the `ttl` for a `grace` period while asynchronously refreshing it) keeps data flowing. It can `keep` data past the `grace` period and serve that when the origin goes down or use that in conditional requests (If-Modified-Since) to the origin to prevent fetching unchanged content. Coalescing requests mitigates thundering herds that reach stale caches. VCL scripting makes the whole thing very configurable. I think Varnish needs better marketing, but I suppose Fastly does that now.
I'm only familiar with varnish 2.1, but as to that version I think it's a bit of a stretch to say varnish is well written. VCL is very complicated - just check the request flow diagram [1]. Some of the documentation is very poor - try to find out the properties available on beresp for example (you have to grep the source code [2]), or try to understand the precise function and implications of grace mode, saint mode, or hit_for_pass. The best redeeming quality is varnishtest and some of the other tools that are provided.
VCL can be challenging for complex flow control (IMO), but it is made easier with Fastly enhanced/custom VCL modules.
Their documentation was good, and the Fastly support team was excellent. Their sales engineers gave us a baseline configuration that suited our needs and were quick to answer any followup questions.
Haven't noticed any downtime or response delays to date.
reply