I'm currently deciding whether to manage my own Varnish servers or use a hosted service like Fastly. One of the most important decision factors here is efficient tag-based cache invalidation, since I plan to put Varnish in front of our API and we'll need to frequently issue purge requests that invalidate a number of related pages.
Fastly offers Surrogate Keys, and Varnish appears to offer a separate module that goes by a number of names, including Hashtwo, Hashninja, and XKey. These features appear to be the same. Are they in fact the same, or is there some key technical or efficiency difference between the two features that is not clear from the blog posts about them?
Surrogate Keys is Fastly's implementation of this functionality. I wrote our current implementation, but have not used HashTwo/Hashninja/xkey, so I am not an authority on differences between the implementations. Xkey is publicly available as a vmod at https://github.com/varnish/libvmod-xkey.
Surrogate Keys are a standard part of Fastly's service, but as a CDN, we provide it as part of our hosted platform. It is not open source for mostly no good reason; there's been some discussion about doing that, but it's not a huge priority (partially because our Varnish is a fork from 2.1.4).
Individual keys are not allowed to exceed 1kb (because why?) and the entire key list is not allowed to exceed 16kb. We raised the limit to these values (previously it was 1kb total) at customer request a year or so ago. There is no limit on the number of keys as long as they fit within that space (though I realize this does effectively bound the keyspace). The rationale for bounding the length is that a key purge results in some number of linear-time operations, which we prefer to keep bounded. I'd be surprised if there was any practical problem with our current limitations.
I would note that xkey is also limited in length and number of keys in the sense that the key is also specified through a header, and header length is effectively bounded by the available workspace for the thread servicing the connection. This length is tunable if you run your own Varnish, and this may not be a practical limitation for you, but it does exist.
Another minor difference I noted scanning through the code is that the xkey vmod supports multiple xkey headers, while Fastly Surrogate Keys are taken from the first matching header. There are some differences in terms of data structures used to achieve the functionality (partially due to the fact that we run a multi-tenant Varnish), but the functionality otherwise appears to be similar.
Finally, we (at this point in time) have a cluster of several hundred Varnish installations globally. Part of our infrastructure has to do with reliably distributing purges through our network and ensuring that they are applied globally. If you run a cluster of Varnish nodes, you may have to do some additional work to invalidate cache across multiple nodes (though this is unlikely to be a significant problem for a small cluster).