Amazon MSK Connect MM2 Plugin

See user guide: https://cornerstone-offerings-public.s3.amazonaws.com/msk-mm2-plugin/Cornerstone_MSK_MM2_Plugin_1.0_SetupGuide.pdf 

• Amazon Marketplace will lead you to the deployment of EC2 instance through CloudFormation stack in the region you choose. EC2 will come with a profile given the least privilege necessary to deploy the replication plugin. • You will be asked to provide a Subnet ID, Security Group ID, and the name of the key you will be using to SSH into your instance.
• Once the stack creation is completed navigate to your Amazon EC2 console and verify that you're in the region you have chosen when spinning up the instance. • Choose the Instance you just launched and connect to it. • Navigate to the /home/ec2-user/Amazon-MSK-Connect-MM2-replication-plugin-by-CCG-1.0 directory. • Confirm that the "ccg-mm2", "MSKConnect-Connectors-Monitoring-DestinationMSKClusterIAMAuthentication_CF.json", "MSKConnect-Connectors-Monitoring_CF.json", and the "Cornerstone_MSK_MM2_Plugin_1.0_SetupGuide.pdf" are there. • Download and follow the Cornerstone_MSK_MM2_Plugin_1.0_SetupGuide.pdf setup guide. • If the CloudFormation stacks fail to deploy, proof check the values of the parameters you provided. Alternatively check the IAM permissions of the EC2 role you are deploying the stack with and add any further permissions necessary. • The EC2 in use is a temporary means for the deployment of the plugin to your AWS Account. Once the infrastructure is set up and the data replication is confirmed, you can clean up the EC2 resource. • User guide contains detailed instruction on how to test and validates the plugin once the setup has completed. • If you have set up the plugin but are not able to observe the data replication happening, or you are not able to connect to the clusters, you can refer to the CloudWatch Alarms and CloudWatch log groups that come with plugin deployment. • The plugin comes with 5 different CloudWatch alarm metrics and 5 CloudWatch alarms which monitor the Kafka/Amazon MSK source cluster health, Destination Amazon MSK cluster health and the occurrence of the data replication between the cluster. The alarm names and observations are:

• source-cluster-health-timeoutexception - source cluster gives a timeout exception more frequently than is to be considered normal
• source-cluster-health-DNSnotresolved - source cluster’s DNS address cannot be resolved
• destination-cluster-health-disconnected - destination cluster is disconnected
• worker-health-noheartbeat - if worker/s is/are not sending heartbeat logs they are not replicating data
• source-cluster-health-brokerunavailable - source cluster is not available

• In order to receive the alarm notifications, subscribe to the "msk-mm2-health" or "mskIAM-mm2-health" SNS topic, dependent on the CloudFormation template you have created the resources with. • If you see any of the alarms in the “Alarm” state, firstly check the availability of your clusters. If the issue persists you should check the logs of the connectors in the CloudWatch Log groups which will have names either starting with "msk/mm2_Source_Checkpoint_Connector_" or "msk/mm2_Heartbeat_Connector_" and take action corresponding to the logged errors. • If your clusters are up and running, cluster configurations are compatible, and the Kafka version you are using is supported by the Amazon MSK Connect you should not be getting any errors.

NOTE: No sensitive information saved by customers No encryption configuration No requirement for rotating programmatic system credentials and cryptographic keys No requirement to decrypt data