DataOps Solution from APN Partner Delphix Boosts Cloud Data Sync and Migration

By Binoy Das, Partner Solution Architect at AWS
By Doug Smith, Principal Solutions Engineer at Delphix

Over the last decade, DevOps practices have revolutionized enterprise IT through automation and repeatability the same way assembly lines modernized manufacturing in the past.

Until recently, however, DevOps practices did not extend to data organization. Data-intensive applications have become major sources of strategic advantage for enterprises, and these organizations need increased agility in collaboration between developers, architects, data scientists, data engineers, and operations.

DataOps platforms achieve this by delivering a unified data fabric that pulls together legacy, structured, and unstructured data. These platforms centralize access and governance policies of data at various stages while optimizing for data locality and availability. They also provide data-as-a-service to all stakeholders through a virtualized layer.

Delphix Dynamic Data Platform unlocks the data residing in silos within an enterprise IT landscape and provides secured delivery of up-to-date data to every stakeholder. This enables rapid development and delivery of data-driven applications.

Delphix is an AWS Partner Network (APN) Advanced Technology Partner.

How Delphix Dynamic Data Platform Works

By utilizing Amazon Web Services (AWS) to maintain and manage the unified data fabric, Delphix Dynamic Data Platform accelerates cloud adoption and is a cornerstone during an organization’s migration to cloud-native and hybrid cloud deployments.

Liberating data from enterprise silos poses an uphill challenge for most IT organizations. The process usually involves backup of production databases, lift-and-shift migration to a cloud environment, provisioning of cloud resources, sanity checks and ingestion of migrated data, and validation of imported data.

There are also manual requests, approvals, and handoffs that can cause delays, not to mention the presence of multiple data sources that often complicates the issues.

Figure 1 – Delphix engine and pods for continual data sync and refresh.

Delphix On-Premises

Delphix address these challenges using two components that work in tandem: Delphix On-Premises and Delphix Cloud Engine.

Delphix On-Premises stays close to the data sources and collects, compresses, and masks data at scheduled intervals. Since the On-Premises component always monitors the data sources, it can easily keep track of the changes and replicate the delta rather than full copies of source data after the initial load.

More specifically, Delphix is enterprise software deployed as a virtual machine appliance using a supported hypervisor on-premises. It connects to your source systems, including any file system, such as application and database binaries, configuration files, and even engineered data sources, and the underlying database such as Oracle, SQL Server, and DB2.

Delphix then creates a one-time compressed copy of each data source and remains constantly in sync with the sources via log shipping. This means you essentially have a time machine of every change to your sources.

Consequently, Delphix lets you provision or refresh virtual copies instead of physical copies. This process is called data virtualization. Virtual copies look and act like physical copies to consumers, but there are a few key differences.

First, virtual copies share common data blocks, making them incredibly space efficient. If you had a 10 TB database, for example, a virtual copy may occupy just gigabytes of actual space. Many companies can provision 10, 50, and sometimes even 100 virtual databases in the space of one physical database.

The other difference is that virtual copies can be automatically provisioned, refreshed, or restored to any point in time in minutes. This is the case whether by an administrator via user interface, by a developer via self-service, or through APIs for full automation. Once the data has been captured by the on-premises “engine” you can create a Replication Profile to move data to the Delphix Cloud Engine.

Additionally, the Delphix Dynamic Data Platform includes integrated data masking. Delphix masking technology enables quick data profiling to identify the locations and types of sensitive data and creates an inventory. Masking algorithms are automatically applied based on the profiling outputs, enabling fast creation and execution of securing your sensitive data. Combined with data virtualization, the Delphix Platform enables efficient and secure data movement from your on-premises environments to AWS.

Delphix Cloud Engine

The Delphix On-Premises component funnels data to a Delphix Cloud Engine over an encrypted channel. Delphix Cloud Engine manages up-to-date copies of data as pods that can be handed off to stakeholders who can then consume the published pods in self-service mode. This removes dependencies on owners of the data, and limits the risk and overhead associated in replicating data in a high-velocity environment.

Figure 2 – How Delphix accelerates cloud migration.

The Delphix Cloud Engine is easily deployed into your Virtual Private Cloud (VPC) using an Amazon Machine Image (AMI) available through AWS Marketplace. Launched on a compatible Amazon Elastic Compute Cloud (Amazon EC2) instance, the Delphix Cloud Engine can receive replicated data from your on-premises engine.

Simply creating a Replication Profile and selecting the dataset you want to migrate quickly enables the movement of data. The replication can run either in scheduled or ad-hoc mode, resulting in the latest information being made available on your Delphix Cloud Engine.

Once the replication is complete, the same rich time flow that exists on-premises is now available on AWS. You can then provision virtual datasets to desired targets or create new physical environments using Delphix V2P capabilities.

Conclusion

AWS customers wanting to migrate on-premises data to the AWS Cloud find a lower resistance path to migration using the Delphix engines. Among the benefits over traditional lift-and-shift migration are the abilities to create a fully-replicated pod before the actual cutover happens, and to create test environments to tweak production performance in the cloud.

With a two-part system, Delphix offers many benefits when it comes to managing vast amounts of data with agility. The Delphix On-Premises component enables synchronization between environments in a transparent way. Having a Delphix Cloud Engine with replication ensures data consumed by downstream applications always stays fresh. Replication also provides opportunity to mask certain data elements, ensuring sensitive data stays safe.

Delphix data pods also effectively create a repository of data, which can be consumed by teams in part or as a whole to provision environments for various project needs. These environments can be either on-premises or on the cloud, and can be bookmarked and configured to accelerate various testing and pilot programs.

Finally, by keeping the virtual-to-physical data in sync, Delphix Cloud Engine helps migration of data to the cloud with speed and confidence.

Companies in retail, hospitality, health care, and financial management are using Delphix to move slow data to AWS, either to operate in a hybrid cloud environment or achieve a complete migration to highly-secure and always-available environments on AWS.

Next Steps

If you want to be data agile in your organization, subscribe to Delphix Dynamic Data Platform on AWS Marketplace >>

If you’re interested in a deeper dive, check out the fantastic resources on the Delphix Blog. Also, check out this whitepaper to learn how Delphix solutions can help you become GDPR compliant.

.

Delphix – APN Partner Spotlight

Delphix is an APN Advanced Technology Partner. Their solutions allow you to connect all of your data, at the office and in the cloud, with the people who need it most. Move fast, get secure, and stay in control with Delphix.

Contact Delphix | Practice Overview | Customer Success | Spec Sheet | Buy on Marketplace

*Already worked with Delphix? Rate this Partner

*To review an APN Partner, you must be an AWS customer that has worked with them directly on a project.