Spark 4.2 Added Native Vector Search: Do You Still Need a Vector Database?

The headline going around is that Spark 4.2 can retire your vector database. That's half true, which is the most dangerous kind of true. Spark did add real vector search, and for some workloads it genuinely removes a whole system from your stack. For others, you'd regret dropping your vector DB. Here's the honest version. The short answer If your vectors already live in your data platform and your searches are batch or analytical, Spark 4.2 can absolutely replace a separate vector database. If you're serving live, low-latency retrieval for a chatbot or search box, you probably still want a dedicated one. It's a "depends on the workload" answer, and the details matter. What Spark 4.2 actually added Spark 4.2 brought vector search into plain SQL. No bolt-on library, no separate engine. The new primitives include vector distance and similarity functions, vector normalization, vector aggregation like sum and average, and the headline one, NEAREST BY, a top-K ranking