Subsurface data management: How to search for data within OSDU on AWS Sandbox

Introduction

In the previous blog we explained how to authenticate to your OSDU on AWS sandbox. After you have successfully authenticated, you might want to interact with the data stored on your OSDU on AWS sandbox. This is the second blog post in a series that provides support for getting started with your OSDU on AWS environment.

Overview of Solution

This blog explains how to search for data within OSDU on AWS Sandbox. We will use the Node.js application created in the previous blog How to authenticate OSDU on AWS sandbox.

Prerequisites:

1. OSDU R2 installed and configured in an AWS account.
2. TNO and Volve data loaded into the AWS account where you installed OSDU R2.
3. Download and install Node.js.
4. Download and install Visual Studio Code IDE.
5. Completion of blog How to authenticate OSDU on AWS sandbox.

Open the Node.js application that you created for the authentication blog in Visual Studio Code.

1. Open Visual Studio Code Open the folder where you created the Node.js package for the authentication blog.

2. Add Search API code to the index.js file

const osduUrl = “<OSDU API Url>”; 
async function CallSearchApi(token)
{
    const searchData=   {
        "kind": "opendes:osdu:*:0.2.0",
        "query": "data.Data.IndividualTypeProperties.WellboreID:\"srn:master-data/Wellbore:7444:\""
    }
    
    const agent = new https.Agent({
        rejectUnauthorized: true
      });
    const instance = axios.create({
        httpsAgent: agent
      });
    instance.post( (osduUrl + '/api/search/v2/query'), searchData, {
    headers: {
        Authorization: `Bearer ${token}` ,
        'content-type': 'application/json',
        'data-partition-id': 'opendes'
    }
    }).then(function (response) {
        console.log("\nQuery:", searchData); 
        console.log("Search Results: ", JSON.stringify(response.data));
        const searchResultsLength =    response.data.totalCount;
        console.log("Search results Length: ", searchResultsLength);
        for(var i = 0; i < searchResultsLength; i++) {
            var obj = response.data.results[i].data;        
            console.log("Printing Item: ", i + 1, " : ", obj);
        }              
    })
    .catch(function (error) {
        console.log(error);
    });
}

In this code, replace the <OSDU API UrL> with your Amazon API Gateway URL.

3. Call this Search API function just after you receive the token.

Add the following code just after console.log statement for access token.

CallSearchApi(accessToken);

4. Run the index.js on the VS Code terminal

5. The example returns three results that match our query where Wellbore ID (data.Data.IndividualTypeProperties.WellboreID) is 7444 (srn:master-data/Wellbore:7444:). We also mentioned kind as “opendes:osdu:*:0.2.0” that returns any kind within opendes:osdu partition and source name.  You will see the search results as shown.

6. If you wanted to get a specific kind, you could have used a query like the following one. The difference between the previous query and the following one is the * is replaced by welllog-wpc specifically asking for well log work product component.

const searchData=   {
  "kind": "opendes:osdu:welllog-wpc:0.2.0",
  "query": "data.Data.IndividualTypeProperties.WellboreID:\"srn:master-data/Wellbore:7444:\""
}

Code walkthrough

The CallSearchApi function takes the token as a parameter and posts the search data to the OSDU search URL using axios library. The following code specifies what kind of record needs to be searched, and the query. You can use * format for wild-card searches and it can be used on any of the four kind colon separated parts. You can use Apache Lucene syntax for the query.

const searchData=   {
        "kind": "opendes:osdu:*:0.2.0",
        "query": "data.Data.IndividualTypeProperties.WellboreID:\"srn:master-data/Wellbore:7444:\""
    }

For more information on using the query syntax, please see OSDU R2 query syntax

Then, instantiate a Node.js https.agent class which is responsible for managing connection persistence and reuse of HTTP clients. The argument rejectUnauthorized: true is set during instantiation to reject SSL issues.

const agent = new https.Agent({
        rejectUnauthorized: true
      });

Then, post the search request to the OSDU search URL. Using the axios library, create an instance of the axios object that will be used to send the post request to the search API. Add the Authorization, Content-Type and the data-partion-id to the headers of the post request.

const instance = axios.create({
        httpsAgent: agent
      });
    instance.post( (osduUrl + '/api/search/v2/query'), searchData, {
    headers: {
        Authorization: `Bearer ${token}` ,
        'content-type': 'application/json',
        'data-partition-id': 'opendes'
    }

The axios requests are promises. When axios completes the promise the server responds with a success or failure code that can be handled with the then() or catch() functions respectively.

Then, pass the successful response returned by the OSDU Search API to the then() function that prints the search results. The search results response object has a totalCount property that can be used to see how many data components are being returned. The code then goes through each of the data components and prints them on the console. If there are any errors, we print them on the console in the catch() function.

}).then(function (response) {
        console.log("\nQuery:", searchData); 
        console.log("Search Results: ", JSON.stringify(response.data));
        const searchResultsLength =    response.data.totalCount;
        console.log("Search results Length: ", searchResultsLength);
        for(var i = 0; i < searchResultsLength; i++) {
            var obj = response.data.results[i].data;        
            console.log("Printing Item: ", i + 1, " : ", obj);
        }              
    })
    .catch(function (error) {
        console.log(error);
    });

Conclusion

This concludes the blog on searching data within OSDU on AWS Sandbox using JavaScript code. In the next blog, we will demonstrate how to consume data from your OSDU on AWS sandbox.