Listing Thumbnail

    Azure Blob Storage Connector for AWS Glue

     Info
    Easily connect to Azure Blob Storage from AWS Glue.
    Listing Thumbnail

    Azure Blob Storage Connector for AWS Glue

     Info

    Overview

    The Azure Blob Storage Connector for AWS Glue simplifies the process of connecting AWS Glue jobs to extract data from Azure Blob Storage, and also load data into Azure Blob Storage. This connector provides comprehensive access to Azure Blob Storage, facilitating cloud ETL processes for operational reporting, backup and disaster recovery, data governance, and more.

    Highlights

    • Connect to Azure Blob Storage from AWS Glue Jobs
    • Simplify data extracts from Azure Blob Storage
    • Simplify data loads to Azure Blob Storage

    Details

    Delivery method

    Delivery option
    Glue 4.0

    Latest version

    Operating system
    Linux

    Pricing

    Azure Blob Storage Connector for AWS Glue

     Info
    This product is free. Subscriptions have no end date and can be canceled anytime.

    Vendor refund policy

    No refunds.

    Legal

    Vendor terms and conditions

    Upon subscribing to this product, you must acknowledge and agree to the terms and conditions outlined in the vendor's End User License Agreement (EULA) .

    Content disclaimer

    Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.

    Usage information

     Info

    Delivery details

    Glue 4.0

    Supported services: Learn more 
    • Amazon ECS
    • Amazon EKS
    Container image

    Containers are lightweight, portable execution environments that wrap server application software in a filesystem that includes everything it needs to run. Container applications run on supported container runtimes and orchestration services, such as Amazon Elastic Container Service (Amazon ECS) or Amazon Elastic Kubernetes Service (Amazon EKS). Both eliminate the need for you to install and operate your own container orchestration software by managing and scheduling containers on a scalable cluster of virtual machines.

    Version release notes

    Azure Blob Storage Connector for AWS Glue.

    • This version supports AWS Glue 3.0 and AWS Glue 4.0.
    • This version supports both read from and write to Azure Blob Storage.
    • This version supports 5 file formats, csv/parquet/json/orc/text.

    Additional details

    Usage instructions

    Please subscribe to the product from AWS Marketplace and Activate the Glue connector from AWS Glue Studio .

    Pre-requisite

    • An account for Azure Blob Storage
    • AWS Secrets Manager secret

    Create a new secret for Azure Blob Storage in AWS Secrets Manager

    We create a secret in AWS Secrets Manager to store the Azure Blob Storage credentials.

    1. Please prepare the Azure Blob Storage connection info including container, account name and account Key in advance. Follow the instructions to manage account keys:https://docs.microsoft.com/en-us/azure/storage/common/storage-configure-connection-string?toc=/azure/storage/blobs/toc.json#configure-a-connection-string-for-azurite 
    2. On the Secrets Manager console, choose Store a new secret.
    3. For Secret type, select Other type of secret.
    4. Enter your key as accountName, accountKey, container and the value respectively, for example,"devstoreaccount1", "Eby8vdM02xNOcqFlqUwJ***", "containername".
    5. Leave the rest of the options at their default.
    6. Choose Next.
    7. Give a name to the secret azureblobstorage_credentials.
    8. Follow through the rest of the steps to store the secret.

    Create a custom connection

    Select the created Secret name, azureblobstorage_credentials, in connection edit page. Then save it.

    Create a Glue job and set connection options

    Create a Glue Job and specify details:

    1. Create a job of data source or target from this connector, select custom connection. Then input options and values.
    2. File format connection options, we support 5 different formats, csv/parquet/json/orc/text. The basic options are path, the Azure Blob storage URI, e.g. /loadingtest-input/10g. fileFormat, input or output file format, e.g. csv/parquet/json/orc/text
    3. For each format, there are different connection options supported. CSV: Option name, header, delimiter, compression, Option Value corresponding,true/false, any delimiter char, none/uncompressed/snappy/gzip/lzo/lz4/brotli/zstd, default value is false, , none. PARQUET, ORC, TEXT and JSON: Option name, compression, Option Value, none/uncompressed/snappy/gzip/lzo/lz4/brotli/zstd, default value is none.
    4. Remember to set the Glue version to be Glue 4.0 on job detail tab.

    Support

    Vendor support

    Please allow 24 hours http://aws.amazon.com/  .

    AWS infrastructure support

    AWS Support is a one-on-one, fast-response support channel that is staffed 24x7x365 with experienced and technical support engineers. The service helps customers of all sizes and technical abilities to successfully utilize the products and features provided by Amazon Web Services.

    Similar products

    Customer reviews

    Ratings and reviews

     Info
    0 ratings
    5 star
    4 star
    3 star
    2 star
    1 star
    0%
    0%
    0%
    0%
    0%
    0 AWS reviews
    No customer reviews yet
    Be the first to write a review for this product.