For scalability you can look to managed solutions from Elastic Search or Pinecone. If you have a DevOps team then self-managed can be faster and cheaper.

The way I see it is text embeddings are simpler and cheaper to work with than agentic systems, so they are preferable when possible. Of course, AI agents can do things that task-specific models can't, so the details of the use case matter when choosing the right technology.