Harness Amazon Q Business power with Microsoft SharePoint for enterprise search

Introduction

In today’s business landscape, organizations are looking for ways to extract insights from their growing data assets. Organizations rely heavily on their file server infrastructure to store, manage, and share mission-critical data. The volume and complexity of data creates challenges when working to maximize its value. Companies need a new strategy to overcome these obstacles.

Amazon Q Business leverages Generative AI (GenAI) to address these data challenges. This service helps organizations use generative AI to improve decision-making and achieve business goals. Amazon Q Business integrates with existing data sources to reveal valuable insights.

Using GenAI, Amazon Q Business assists you in quickly analyzing your data, identify patterns, and generate personalized recommendations tailored to your unique business needs. Whether you’re looking to improve customer service, optimize operational efficiency, or uncover new revenue opportunities, Amazon Q Business and GenAI will aid in driving innovation and growth.

In this blog post, we’ll demonstrate how Amazon Q Business integrates with Microsoft SharePoint Server to unlock the full potential of your files. You will learn how to query Microsoft SharePoint server data using natural language, find relevant information, extract key points, and derive valuable insights.

Solution overview

Figure 1 – Amazon Q Business data source connection with SharePoint solution architecture.

As referred in Figure 1, ensuring the confidentiality, integrity, and availability of data is of utmost importance. To achieve this, the Amazon Q Business connector for SharePoint employs a robust security framework that honors existing user identities, roles, and permissions. You achieve this through implementing identity crawling and access control lists (ACLs) on the connector, using secure credentials managed by AWS Secrets Manager. To prevent over-exposure of sensitive data, the solution enforces the principle of least privilege, allowing users to access the data for which they have received explicit permissions.

In environments where Microsoft products are used, it is a normal practice to store user and group objects in Microsoft Active Directory. To facilitate permission-based filtering of query responses, Q Business synchronizes user and group information from Active Directory into AWS IAM Identity Center. With this configuration, the solution enforces fine-grained access controls and provides filtered responses that respect the user’s permissions.

The SharePoint Server data connector ingests document content and NTFS and local SharePoint permissions to provide a comprehensive understanding of data access controls. When a user submits a query to Q Business, the solution generates a filtered response that enforces the user’s permissions, ensuring that sensitive data is accessible to authorized individuals.

Note: This solution is for SharePoint Server and not applicable to SharePoint Online.

Prerequisites

The following prerequisites are necessary to test this solution. We assume you have following prerequisites and working SharePoint server deployment in AWS account.

You have a working SharePoint environment deployed on Amazon EC2 in AWS. How to deploy SharePoint server on Amazon EC2.
You have AWS IAM Identity Center configured. Also, an AWS Identity and Access Management (IAM) role and a user with permissions to create and manage resources and components for the Q Business application. IAM Identity Center comes with multi-factor authentication (MFA) turned on by default.
Onboard Amazon Q Business. Amazon Q Business onboarded with AWS IAM Identity Center. If you haven’t set this up yet, see Creating an Amazon Q Business application environment.
Make sure you deployed AWS Directory Service for Microsoft Active Directory domain joined SharePoint Server deployed in your AWS account.
One Amazon Elastic Compute Cloud (EC2) Windows instance with Remote server administrative tools (RSAT). You will use it to manage users with AWS Directory Service for Microsoft Active Directory (AWS Managed AD).
AWS Managed AD as a source of truth for AWS Identity center. This means AD users and groups created in AWS Managed AD will sync to AWS Identity center which you use inside Amazon Q business at the time of application configuration.
Secrets must have access to SharePoint Server. Make sure you add Share Point Server’s user IDs and passwords in AWS Secret Manager manually or, you generate secrets while configuring the SharePoint Server data source connector, which enables direct authentication and integration with Amazon Q.

Note : While this deployment uses AWS Managed AD with SharePoint and IDC integration for blog posts, self-managed AD remains a viable alternative integration option.

Walkthrough

Configure Amazon Q Business application and data source connector for Microsoft SharePoint.

Sign in to the Amazon Q Business console
Select Create application as referenced in Figure 2.

Figure 2 – Amazon Q Business application console
On the Create application page, enter the following information for your Amazon Q Business application.
1. Application name, a name for your Amazon Q Business application environment for dentification.
2. Outcome, select Web experience to create a web experience for your application.
  
  Figure 3 – Amazon Q Business application create wizard
For Access management method, choose IAM Identity Center (recommended).
Note: As illustrated in figure 4,
1. If you have both an IAM Identity Center organization instance and an account instance configured, your instances will be detected automatically.
2. If you’ve connected a pre-configured IAM Identity Center instance that already has users and groups, Amazon Q Business detects those users and groups.
  
  Figure 4 – Amazon Q Business Access management Configuration
For Application details – Amazon Q Business uses the following default configuration settings for your application.
Configure Application service access section as mentioned in figure 5.
1. Select Create and use a new service-linked role (SLR) for your application.
2. Encryption, leave default. Amazon Q Business will create an AWS owned AWS KMS key to encrypt your data.
3. For Web experience service access, choose Create and use new service-link role (SLR)
  
  Figure 5 – Application service access and Web experience setting configuration
To create your application, choose Create and open web experience if you’re creating a web experience as well. It will take a few minutes to complete deployment and then add a few users using manage user access tab.

Connecting to SharePoint

Now you will use the Data sources page to connect to SharePoint server and to add an Index. A data source allows you to combine data from different places into one central index for your Amazon Q Business application. Q Business stores and organizes data in an index. An Index refers to the collection of enterprise data and content that Amazon Q search and reference when answering queries.

Adding an index

From the left navigation menu, choose Data sources.
From the Data sources page, choose Add index.
From the Add index page, choose Create a new index and enter the following information (see figure 5):
1. In Name your index with a unique identifier, for Index name, input a name for your Amazon Q Business application.
2. In Index provisioning, choose between Enterprise and Starter index types based on your use case. We will use Enterprise.
3. For Number of units, choose the number of index units you need, e.g.,50.
  Note: Amazon Q Business charges you based on the document capacity that you choose. Enterprise indexes support up to 50 units. Starter indexes support up to 5 units. Each unit contains 20,000 documents or 200 MB, whichever limit is reached first.
  
  Figure 6 – Create a new index to retrieve responses from data
4. To create your index and retriever, choose Add an index.

Configuring the Amazon Q Business SharePoint Server connector

From the Data sources page, choose Add data source, as illustrated in Figure 7.

Figure 7 – Amazon Q Business data source options for SharePoint
On the Add data source page, from Data sources, find and add the SharePoint data source to your Amazon Q application by selecting the plus button.
On the SharePoint data source page, enter the following information:
1. For Data source name, add your data source’s name for tracking.
2. Description – optional, add an optional description for your data source.
3. In Source, select SharePoint Server option
4. Choose SharePoint Version (Example, SharePoint 2013, 2016,2019, Subscription edition). In this post, we will use SharePoint Subscription edition.
5. Provide the full URL for the SharePoint site you want to include when crawling and indexing. These are URLs specific to your SharePoint repository. The URL must start with the https protocol. (Example, https://sp-genai.demo.com/sites/demo).
6. Domain, enter the fully qualified SharePoint domain name (FQDN).
7. SSL certificate location, upload the public SSL certificate to an Amazon Simple Storage Service (S3) bucket and enter the Amazon S3 path to your SSL certificate file. Example, s3://sharepoint-server-certificate-store/sp-genai.demo.com.pem
  
  Figure 8 – Amazon Q Business Application Source configuration
For Authorization, Amazon Q Business crawls Access Control Lists (ACLs) to generate responses from documents your end users have permission to access.

Figure 9 – Amazon Q Business data source Authorization configuration
For Authentication, choose NTLM authentication (Note: Configuration support SharePoint App-Only, Kerberos authentication as well).

Figure 10 – Amazon Q Business data source Authentication configuration
For AWS Secrets Manager secret, choose an existing secret or create a Secrets Manager secret to store your SharePoint authentication credentials. You will need the LDAP Server Endpoint, LDAP Search Base, LDAP username, LDAP Password information if using Email ID with domain from IDP.

Figure 11 – Amazon Q Business data source authentication Secrets from AWS Secret Manager
Choose IAM role. [Note: Creating a new IAM role for data sources prevents errors, since IAM roles for applications remain separate from data source roles. Select “Create a new role” to ensure proper data source configuration.]

Figure 12 – Amazon Q Business data source IAM role configuration
In Sync scope, for Select entities, choose All (or specify the combination of items to sync). For Sync mode and Sync run schedule, select options based on your needs. For this post, we will use Full Sync mode and Daily Sync Schedule, respectively.
Choose Add data source.
After you create the data source, choose Sync now to start the crawling and indexing. When the sync job finishes, your data source is ready to use as reference in Figure 13.

Figure 13 – Amazon Q Business data source connector deployment status

Amazon Q application Web URL

Users access this URL to interact with Amazon Q Business application through their web browser after their organization created and configures Amazon Q Business application.

Figure 14 – Amazon Q Business Web URL to access Q business application

Test the solution

Example Scenario

The Human Resources administration is currently reviewing a pool of resumes to identify the best candidate for the position of “Cyber Security Strategist”, with a focus on selecting an individual with the most extensive relevant experience.

The organization uses a SharePoint site to host and manage documents for various departments, including Human Resources, Finance, and others. As a best practice, organizations established a user access framework within SharePoint. Organizations assign SharePoint permissions to users based on their roles within a structured access framework. The SharePoint Administrator maintains full access to all departmental documents. Team members access their department’s specific documents – for example, Human Resources staff view HR documents but do not access Finance department files. The HR department stores job description documents and resumes on a SharePoint site, as shown in Figure 15.

Figure 15 – SharePoint demo site HR department documents sample

Use the natural language prompt (NLP) in Amazon Q AI assistant to locate candidates for the Cyber Security Strategist role with relevant experience. To do this, login to the Amazon Q Business application with SharePoint users HR Admin (hradmin) and IT Admin (SP_Gary). Then ask the same question with each user to see the results we get from AI assistant as reference in Figures 16 and 17.

Question: Are any of the resumes we have a good match for the “Senior Cybersecurity Strategist” job listing?

Access Q Business Web URL to launch the application as reference in Figure 14. It will prompt you to enter your user id and password. Login with the HR admin AD account (hradmin) as referenced in figure 16 .

Figure 16 – Amazon Q AI assistant query response using HR admin access

Access Q business Web URL to launch deployed application and login with IT admin account.

Log in as IT admin (SP_Gary) to test his access to retrieve data from the HR department using Amazon Q AI assistance. The desired outcome is that Gary will receive no results, based on the scenario as reference in Figure 17, because of the restricted permissions in place that prevent him from accessing HR department-related information by other department users.

Figure 17 – Amazon Q AI assistant query response using IT admin access

This example illustrates the capability of the HR admin to identify the most suitable candidate for the Cyber Security Strategist position by leveraging a natural language prompt and retrieving information from a SharePoint site document from a designated source location. However, the IT administrator, Gary, didn’t get the desired results because of insufficient permissions.

This is one example of scenarios that you apply to different departments, such as marketing, finance, and IT, by deploying the Q application in a manner tailored to specific use cases.

Note:
Amazon Q Business, by default, incorporates administrative controls and guardrails, ensuring a secure user experience.

The confidentiality and integrity of your data are of utmost importance. A key feature of the Amazon Q business SharePoint data connectors is their adherence to SharePoint’s access control lists (ACLs). Therefore, Amazon Q Business denies users access to data if they lack the required permissions in the source system thereby maintaining the highest standards of data security and confidentiality.

Cleanup

Configuring AWS services from this post will provision resources which incur cost. It is a best practice to delete configurations and resources that you are no longer using so that you do not incur unintended charges.

Delete Q Business application by accessing the Amazon Q Business console, navigate to your application, select the specific web experience you want to delete, and then choose the “Delete” option within the application settings or use the AWS CLI command “delete-web-experience” to achieve the same result. And delete Data source.
Delete AWS Managed AD, EC2 management server, Delete Secrets etc. If deployed for test purpose not production.

Conclusion

In this post, you have learned how to configure the SharePoint connector for Amazon Q Business using the principle of least privilege, with access controls that work with Microsoft SharePoint server which helps employees interact securely with the organization’s knowledge and data stored in SharePoint using natural language, making it effortless to find relevant information, extract key points, and derive valuable insights. This solution help improve productivity, decision-making, and knowledge sharing within your organization. You can Integrate multiple supported data source connectors with Amazon Q Business by applying this concept to your specific use cases.

AWS has significantly more services, and more features within those services, than any other cloud provider, making it faster, easier, and more cost effective to move your existing applications to the cloud and build nearly anything you can imagine. Give your Microsoft applications the infrastructure they need to drive the business outcomes you want. Visit our .NET on AWS and AWS Database blogs for additional guidance and options for your Microsoft workloads. Contact us to start your migration and modernization journey today.