AWS Graviton 3 > Graviton 4 for Vector Similarity Search
If you are doing vector search with a vector library that supports SVE, you should use a Graviton 3 machine. It is cheaper, and it will also deliver more raw performance. A few months ago, we started working on a vertical layout for vector similarity search (PDX). As part of the benchmarks that we were running on different microarchitectures and vector systems like FAISS, Milvus, and Usearch, there was an observation that puzzled us: Graviton3 performed better than Graviton4 in almost all vector search scenarios, not only in queries per dollar (QP$) but also in queries per second (QPS). This was the case across vector libraries and even in our implementations of vector search algorithms. Here is one example of the QPS and QP$ of both microarchitectures on queries to an IVF index: ...