Posted On: Nov 17, 2023
AWS Glue for Apache Spark now supports native connectivity to Amazon OpenSearch Service which enables users to efficiently read and write data from Amazon OpenSearch Service without the need to install or manage OpenSearch connector libraries. Users can now use data stored in OpenSearch Service indexes as a source or target within AWS Glue Studio's no-code, drag-and-drop visual interface or directly in an AWS Glue ETL job script. When combined with AWS Glue's ETL (Extract, Transform, Load) capabilities, this new connector simplifies the creation of ETL pipelines enabling ETL developers to save time building and maintaining data pipelines.
To get started, create a new Amazon OpenSearch Service connection within AWS Glue Data Catalog and add an OpenSearch Service source or target to your Glue ETL job. When reading from OpenSearch Service, developers can choose an entire OpenSearch Service index as a source or submit a query to select specific index data. When writing to OpenSearch Service, users can enrich, cleanse and transform data in Glue before loading the results into an OpenSearch Service index. These capabilities enable ETL developers to work with OpenSearch Service and AWS Glue across a variety of scenarios.
This feature is available in all commercial AWS Regions where AWS Glue is available. To learn more, visit the AWS Glue documentation.