Amazon Web Services (AWS) has announced a significant update to its Amazon Bedrock platform, introducing a range of features designed to help enterprises streamline the testing of applications before deployment. The new features, unveiled during the ongoing annual re:Invent conference, aim to improve the performance and cost-effectiveness of large language models (LLMs) in various applications.

One of the key updates is the addition of a retrieval augmented generation (RAG) evaluation tool within Bedrock Knowledge Bases. This tool enables enterprises to assess and optimize RAG applications using Amazon Bedrock Knowledge Bases, allowing them to compare different configurations and tune their settings to achieve the desired results for their specific use case. The evaluation process utilizes an LLM to compute the metrics for the evaluation, providing a more accurate and efficient way to test applications.

In addition to the RAG evaluation tool, AWS has also introduced support for custom connectors within Bedrock Knowledge Bases. These custom connectors enable the ingestion of data from a variety of sources, including streaming data, allowing developers to efficiently and cost-effectively ingest, update, or delete data directly using a single API call. This eliminates the need to perform a full sync with the data source periodically or after every change, making the process more streamlined and cost-effective.

Furthermore, AWS has introduced a new Rerank API inside Bedrock Knowledge Bases, designed to offer developers a way to use reranking models to enhance the performance of their RAG-based applications. The reranking models accessed via the new API can help developers overcome limitations of semantic search, which is often used in RAG applications. These limitations include the inability to prioritize the most suitable documents based on user preferences or query context, especially when the user query is complex, ambiguous, or involves nuanced context.

The Rerank API currently supports Amazon Rerank 1.0 and Cohere Rerank 3.5 models, providing developers with more options to improve the relevance and accuracy of responses and reduce costs. The API is designed to help developers overcome the challenges associated with partially relevant document retrieval, which can lead to issues with proper attribution of sources.

In another significant development, AWS has added a new LLM-as-a-judge feature inside Bedrock Model Evaluation, a tool that helps enterprises choose an LLM that fits their use case. The LLM-as-a-judge feature, currently in preview, allows developers to perform tests and evaluate other models with human-like quality at a lower cost compared to human evaluation. This feature is designed to make it easier for enterprises to go into production by providing fast, automated evaluation of AI-powered applications, shortening feedback loops, and speeding up improvements.

The LLM-as-a-judge feature assesses multiple quality dimensions, including correctness, helpfulness, and responsible AI criteria such as answer refusal and harmfulness. This comprehensive evaluation process enables enterprises to make more informed decisions when selecting an LLM for their specific use case.

Overall, the updates to Amazon Bedrock demonstrate AWS's commitment to providing enterprises with the tools and features they need to develop and deploy AI-powered applications efficiently and effectively. As the demand for AI-powered solutions continues to grow, these updates are likely to have a significant impact on the industry, enabling enterprises to build more accurate, relevant, and cost-effective applications.