Amazon ElastiCache is a web service that makes it easy to deploy, operate, and scale an in-memory cache in the cloud. Amazon ElastiCache improves the performance of web applications by allowing you to retrieve information from a fast, managed, in-memory caching system, instead of relying entirely on slower disk-based databases. The service simplifies and offloads the management, monitoring and operation of in-memory cache environments, enabling your engineering resources to focus on developing applications. Using Amazon ElastiCache, you can not only improve load and response times to user actions and queries, but also reduce the cost associated with scaling web applications.
Amazon ElastiCache automates common administrative tasks required to operate a distributed cache environment. Using Amazon ElastiCache, you can add a caching layer to your application architecture in a matter of minutes via a few clicks of the AWS Management Console. Once a cache cluster is provisioned, Amazon ElastiCache automatically detects and replaces failed cache nodes, providing a resilient system that mitigates the risk of overloaded databases, which slow website and application load times. Through integration with Amazon CloudWatch monitoring, Amazon ElastiCache provides enhanced visibility into key performance metrics associated with your cache nodes. Amazon ElastiCache is protocol-compliant with Memcached, a widely adopted memory object caching system, so code, applications, and popular tools that you use today with your existing Memcached environments will work seamlessly with the service. As with all Amazon Web Services, there are no up-front investments required, and you pay only for the resources you use.
The in-memory caching provided by Amazon ElastiCache can be used to significantly improve latency and throughput for many read-heavy application workloads (such as social networking, gaming, media sharing and Q&A portals) or compute-intensive workloads (such as a recommendation engine). In-memory caching improves application performance by storing critical pieces of data in memory for low-latency access. Cached information may include the results of I/O-intensive database queries or the results of computationally-intensive calculations.
A Cache Node is the smallest building block of an Amazon ElastiCache deployment. It is a fixed-size chunk of secure, network-attached RAM. Each Cache Node runs an instance of the Memcached software and has its own DNS name and port. Multiple types of Cache Nodes are supported, each with varying amount of associated memory.
A Cache Cluster is a collection of one more Cache Nodes, each running an instance of the Memcached service. Most of your operations will be performed at the Cache Cluster level. All Cache Nodes within a Cache Cluster are designed to be of the same Node Type, have the same parameter and security group settings.
If you are not already signed up for Amazon ElastiCache, you can click the "Sign Up Now" button on the Amazon ElastiCache detail page and complete the sign-up process. You must have an Amazon Web Services account; if you do not already have one, you will be prompted to create one when you begin the Amazon ElastiCache sign-up process. After you are signed up for ElastiCache, please refer to the Amazon ElastiCache documentation, which includes our Getting Started Guide.
Once you have familiarized yourself with Amazon ElastiCache, you can launch a Cache Cluster within minutes by using the AWS Management Console or Amazon ElastiCache APIs.
Once your Cache Cluster is available, you can retrieve your Cache Node endpoints using the following steps on the AWS Management Console:
Alternatively, you can use the DescribeCacheClusters API to retrieve the Endpoint list.
You can then configure the Memcached config file used by your favorite Memcached clients with this endpoint list and use your favorite programming language to add or delete data from your ElastiCache Nodes. In order to allow network requests to your Cache Nodes, you will need to authorize access. For a detailed explanation to get started, please refer to our Getting Started Guide.
Amazon ElastiCache is protocol-compliant with Memcached. Therefore, you can use standard Memcached operations like get, set, incr and decr in exactly the same way as you would in your existing Memcached deployments. Amazon ElastiCache supports both the text and binary protocols. It also supports most of the standard stats results, which can also be viewed as graphs via CloudWatch. As a result, you can switch to using Amazon ElastiCache without recompiling or re-linking your applications - the libraries you use will continue to work. To configure the cache servers your application accesses, all you will need to do is to update your application's Memcached config file to include the endpoints of the servers (Cache Nodes) we provision for you. You can simply use the "Copy Node Endpoints" option on the AWS Management Console or the "DescribeCacheClusters" API to get a list of the endpoints. As with any migration process, we recommend thorough testing of your new Amazon ElastiCache deployment before completing the cut over from your current solution.
Please note that Amazon ElastiCache currently allows access only from the Amazon EC2 network, so in order to use the service, you should have your application servers in Amazon EC2.
Amazon ElastiCache uses DNS entries to allow client applications to locate cache servers (Cache Nodes). The DNS name for a Cache Node remains constant, but the IP address of a Cache Node can change over time, for example, when Cache Nodes are auto replaced after a failure. See this FAQ for recommendations to deal with Cache Node failures.
You can think of the Amazon ElastiCache maintenance window as an opportunity to control when software patching occurs, in the event either are requested or required. If a "maintenance" event is scheduled for a given week, it will be initiated and completed at some point during the 60 minute maintenance window you identify.
Your Cache Nodes could incur some downtime during your maintenance window if software patching is scheduled. Please refer to Cache Engine Version Management for more details. Patching can be user requested - for example cache software upgrade, or determined as required (if we identify any security vulnerabilities in the system or caching software). Software patching occurs infrequently (typically once every few months) and should seldom require more than a fraction of your maintenance window. If you do not specify a preferred weekly maintenance window when creating your Cache Cluster, a 60 minute default value is assigned. If you wish to modify when maintenance is performed on your behalf, you can do so by modifying your DB Instance in the AWS Management Console or by using the ModifyCacheCluster API. Each of your Cache Clusters can have different preferred maintenance windows, if you so choose.You pay only for what you use and there is no minimum fee. Pricing is per Cache Node-hour consumed for each Node Type. Partial Node-hours consumed are billed as full hours. There is no charge for data transfer between Amazon EC2 and Amazon ElastiCache within the same Availability Zone. While standard Amazon EC2 Regional Data Transfer charges apply when transferring data between an Amazon EC2 instance and an Amazon ElastiCache Node in different Availability Zones of the same Region, you are only charged for the Data Transfer in or out of the Amazon EC2 instance. There is no Amazon ElastiCache Data Transfer charge for traffic in or out of the Amazon ElastiCache Node itself. For more information, please visit the pricing page.
Billing commences for a Cache Node as soon as the Cache Node is available. Billing continues until the Cache Node is terminated, which would occur upon deletion.
Node hours are billed for any time your Cache Nodes are running in an "Available" state. If you no longer wish to be charged for your Cache Node, you must terminate it to avoid being billed for additional Node hours.
Except as otherwise noted, our prices are exclusive of applicable taxes and duties, including VAT and applicable sales tax.
With Reserved Cache Nodes, you can now make a one-time, up-front payment to create a one or three year reservation to run your Cache Node in a specific Region and receive a significant discount off of the ongoing hourly usage charge. There are three Reserved Cache Node types (Light, Medium, and Heavy Utilization Reserved Cache Nodes) that enable you to balance the amount you pay upfront with your effective hourly price.
Functionally, Reserved Cache Nodes and On-Demand Cache Nodes are exactly the same. The only difference is how your Cache Node(s) are billed; with Reserved Cache Nodes, you make a one-time up-front payment and receive a lower ongoing hourly usage rate (compared with On-Demand Cache Nodes) for the duration of the term.
You can use the "Purchase Reserved Cache Nodes" option in the AWS Management Console. Alternatively, you can use the API tools to list the reservations available for purchase with the DescribeReservedCacheNodesOfferings API method and then purchase a cache node reservation by calling the PurchaseReservedCacheNodesOffering method.
Creating a Reserved Cache Node is no different than launching an On-Demand Cache Node. You simply specify the Cache Node class and Region for which you made the reservation. So long as your reservation purchase was successful, Amazon ElastiCache will apply the reduced hourly rate for which you are eligible to the new Cache Node.
Pricing changes associated with a Reserved Cache Node are activated once your request is received while the payment authorization is processed. You can follow the status of your reservation on the AWS Account Activity page or by using the DescribeReservedCacheNodes API. If the one-time payment cannot be successfully authorized by the next billing period, the discounted price will not take effect.
When your reservation term expires, your Reserved Cache Node will revert to the appropriate On-Demand hourly usage rate for your Cache Node class and Region.
Amazon ElastiCache supports Cache Nodes of the following memory types:
Each Node Type above lists the memory available to Memcached after taking Amazon ElastiCache System Software overhead into account. The total amount of memory in a Cache Cluster is an integer multiple of the memory available for the Cache Node Type selected. For example, a Cache Cluster consisting of ten Cache Nodes of 6 GB each will provide 60 GB of total memory.
Though there is no precise answer for this question, with Amazon ElastiCache, you don't need to worry about getting the number of Cache Nodes exactly right, as you can very easily add or remove Nodes later.
The following two inter-related aspects could be considered for the choice of your initial configuration:
The amount of memory required is dependent upon the size of your data set and the access patterns of your application. To improve fault tolerance, once you have a rough idea of the total memory required, divide that memory into enough Cache Nodes such that your application can survive the loss of one or two Cache Nodes. For example, if your memory requirement is 14GB, you may want to use two cache.m1.large nodes instead of using one cache.m1.xlarge node. It is important that other systems such as databases will not be overloaded if the cache-hit rate is temporarily reduced during failure recovery of one or more of Cache Nodes. Please refer to the the Amazon ElastiCache User Guide for more details.
The service will detect the Cache Node failure and react with the following automatic steps:
When not using VPC, Amazon ElastiCache allows you to control access to your Cache Clusters through Cache Security Groups. A Cache Security Group acts like a firewall, controlling network access to your Cache Cluster. By default, network access is turned off to your Cache Clusters. If you want your applications to access your Cache Cluster, you must explicitly enable access from hosts in specific EC2 security groups. This process is called ingress.
To allow network access to your Cache Cluster, create a Cache Security Group and link the desired EC2 security groups (which in turn specify the EC2 instances allowed) to it. The Cache Security Group can be associated with your Cache Cluster at the time of creation, or using the "Modify" option on the AWS Management Console.
Please note that IP-range based access control is currently not enabled for Cache Clusters. All clients to a Cache Cluster must be within the EC2 network, and authorized via security groups as described above.
When using VPC, please see here for more information.
A Cache Security Group acts like a firewall, controlling network access to your Amazon ElastiCache Cluster when not deployed in an Amazon Virtual Private Cloud (VPC). For more details, see here.
No. Currently, all clients to an ElastiCache Cluster must be within the Amazon EC2 network, and authorized via security groups as described here.
Yes, EC2 instances in a VPC can access Amazon ElastiCache if the ElastiCache cluster was created within the VPC. Details on how to create an Amazon ElastiCache cluster within a VPC are given here.
Amazon VPC lets you create a virtual networking environment in a private, isolated section of the Amazon Web Services (AWS) cloud, where you can exercise complete control over aspects such as private IP address ranges, subnets, routing tables and network gateways. With Amazon VPC, you can define a virtual network topology and customize the network configuration to closely resemble a traditional IP network that you might operate in your own datacenter.
One of the scenarios where you may want to use Amazon ElastiCache in a VPC is if you want to run a public-facing web application, while still maintaining non-publicly accessible backend servers in a private subnet. You can create a public-facing subnet for your webservers that has access to the Internet, and place your backend infrastructure in a private-facing subnet with no Internet access. Your backend infrastructure could include RDS DB Instances and an Amazon ElastiCache Cluster providing the caching layer. For more information about Amazon VPC, refer to the Amazon Virtual Private Cloud User Guide.
For a walk through example of creating an Amazon ElastiCache Cluster in VPC, refer to the Amazon ElastiCache User Guide.
Following are the pre-requisites necessary to create a Cache Cluster within a VPC:
Creating an Amazon ElastiCache Cluster in an existing VPC is the same as that for a newly created VPC. Please see this for more details.
Amazon ElastiCache Nodes, deployed within a VPC, can be accessed by EC2 Instances deployed in the same VPC. If these EC2 Instances are deployed in a public subnet with associated Elastic IPs, you can access the EC2 Instances via the internet.
Amazon ElastiCache Nodes, deployed within a VPC, can never be accessed from the Internet or from EC2 Instances outside the VPC.
We strongly recommend you use the DNS Name to connect to your ElastiCache Node as the underlying IP address can change (e.g., after a cache node replacement).
A Cache Subnet Group is a collection of subnets that you must designate for your Amazon ElastiCache Cluster in a VPC. A Cache Subnet Group is created using the Amazon ElastiCache Console. Each Cache Subnet Group should have at least one subnet. Amazon ElastiCache uses the Cache Subnet Group to select a subnet. The IP Addresses from the selected subnet are then associated with the Cache Node Endpoints. Furthermore, Amazon ElastiCache creates and associates Elastic Network Interfaces to cache nodes with the previously mentioned IP addresses.
Please note that, we strongly recommend you use the DNS Names to connect to your cache nodes as the underlying IP addresses can change (e.g., after cache node replacement).
An existing Cache Subnet Group can be updated to add more subnets either for existing Availability Zones are for new Availability Zones added since the creation of the ElastiCache Cluster. However, changing the Cache Subnet Group of a deployed Cache Cluster is not currently allowed.
The basic functionality of Amazon ElastiCache remains the same whether VPC is used or not. Amazon ElastiCache manages automatic failure detection, recovery, scaling, auto discovery, and software patching whether your ElastiCache Cluster is inside or outside a VPC.
Similarly, an Amazon ElastiCache Cluster, inside or outside a VPC, is never allowed to be accessed from the Internet. Within a VPC, nodes of an ElastiCache cluster only have a private IP address (within a subnet that you define). Outside of a VPC, the access to the ElastiCache cluster can be controlled using Cache Security Groups as described here.
No, you cannot move an existing Amazon ElastiCache Cluster from outside VPC into a VPC. You will need to create a new Amazon ElastiCache Cluster inside the VPC.
Currently, direct migration of ElastiCache Cluster from inside to outside VPC is not supported. You will need to create a new Amazon ElastiCache Cluster outside VPC.
Amazon ElastiCache allows you to control access to your Cache Cluster and therefore the Cache Nodes using Cache Security Groups in non-VPC deployments. A Cache Security Group acts like a firewall controlling network access to your Cache Node. By default, network access is turned off to your Cache Nodes. If you want your applications to access your Cache Node, you can set your Cache Security Group to allow access from EC2 Instances with specific EC2 Security Group membership or IP ranges. This process is called ingress. Once ingress is configured for a Cache Security Group, the same rules apply to all Cache Nodes associated with that Cache Security Group. Cache Security Groups can be configured with the “Cache Security Groups” section of the Amazon ElastiCache Console or using the Amazon ElastiCache APIs.
In VPC deployments, access to your cache nodes is controlled using the VPC Security Group and the Cache Subnet Group. The VPC Security Group is the VPC equivalent of the Cache Security Group.
You are responsible for modifying routing tables and networking ACLs in your VPC to ensure that your ElastiCache Nodes are reachable from your client instances in the VPC. To learn more see the Amazon ElastiCache Documentation.
No, Cache Security Groups are not used when operating in a VPC. Instead they are used in the non VPC settings. When creating a cache cluster in a VPC you will need to use VPC Security Groups.
No, you can only associate VPC security groups that are part of the same VPC as your Cache Cluster.
Yes, Cache Nodes of an Amazon ElastiCache cluster can span multiple subnets as long as the subnets are part of the same Cache Subnet Group that was associated with the ElastiCache Cluster at creation time. Furthermore, all subnets must be within the same AZ.
A Cache Parameter Group acts as a "container" for engine configuration values that can be applied to one or more Cache Clusters. If you create Cache Cluster without specifying a Cache Parameter Group, a default Cache Parameter Group is used. This default group contains engine defaults and Amazon ElastiCache system defaults optimized for the Cache Cluster you are running. However, if you want your Cache Cluster to run with your custom-specified engine configuration values, you can simply create a new Cache Parameter Group, modify the desired parameters, and modify the Cache Cluster to use the new Cache Parameter Group. Once associated, all Cache Clusters that use a particular Cache Parameter Group get all the parameter updates to that Cache Parameter Group. For more information on configuring Cache Parameter Groups, please refer to the Amazon ElastiCache User Guide.
Amazon ElastiCache by default chooses the optimal configuration parameters for your Cache Cluster taking into account the Node Type's memory/compute resource capacity. However, if you want to change them, you can do so using our configuration management APIs. Please note that changing configuration parameters from recommended values can have unintended effects, ranging from degraded performance to system crashes, and should only be attempted by advanced users who wish to assume these risks. For more information on changing parameters, please refer to the Amazon ElastiCache User Guide.
You can use the AWS Management Console, Amazon ElastiCache APIs, or Command Line Tools to see information about your Cache Parameter Groups and their corresponding parameter settings.
Amazon ElastiCache allows you to control if and when the Memcached protocol-compliant software powering your Cache Cluster is upgraded to new versions supported by Amazon ElastiCache. This provides you with the flexibility to maintain compatibility with specific Memcached versions, test new versions with your application before deploying in production, and perform version upgrades on your own terms and timelines.
Unless you specify otherwise, your Cache Cluster will automatically be upgraded to new Memcached minor versions as they are supported by Amazon ElastiCache. This patching will occur during your scheduled maintenance window, and will be announced on the Amazon ElastiCache Forum in advance. If you wish to turn off automatic version upgrades, you can do so by selecting the "Auto Minor Version Upgrade" for your Cache Cluster to "No". Since major version upgrades involve some compatibility risk, they will not occur automatically and must be initiated by you.
This approach to cache software patching puts you in the driver's seat of version upgrades, but still offloads the work of patch application to Amazon ElastiCache. You can learn more about version management by reading the FAQs that follow. Alternatively, you can refer to the Amazon ElastiCache User Guide.
While Cache Engine Version Management functionality is intended to give you as much control as possible over how patching occurs, we may patch your Cache Cluster on your behalf if we determine there is any security vulnerability in the system or cache software.
You can specify any currently supported version (minor and/or major) when creating a new Cache Cluster. If you have opted out of automatically scheduled upgrades by selecting the "Auto Minor Version Upgrade" option to "No" but wish to manually initiate an upgrade to a supported minor version or major version release, you can do so using the "Modify" option for your Cache Cluster. Simply specify the version you wish to upgrade to via the "Cache Engine Version" field. The upgrade will then be applied on your behalf either immediately (if the "Applied Immediately" option is checked) or during the next scheduled maintenance window for your Cache Cluster.
In the context of Memached, version numbers are organized as follows:
Memcached version = X.Y.Z
X = Major version, Y = Release level, Z = Version number within release series.
From the Amazon ElastiCache standpoint, a version change would be considered major if either major version or release level is being changed. Example: going from 1.4.X -> 1.5.X. A version change would be considered minor if the version number within the release is being changed. Example: going from 1.4.5 -> 1.4.6.
As of today, Amazon ElastiCache supports the Memached major version 1.4. We plan to support additional Memcached major versions in the future.
Over time, we plan to support additional Memcached versions for Amazon ElastiCache, both minor and major. The number of new version releases supported in a given year will vary based on the frequency and content of the Memcached version releases and the outcome of a thorough vetting of the release by our engineering team. However, as a general guidance, we aim to support new Memcached versions within 3-5 months of their General Availability release.
In terms of deprecation policy:
Amazon ElastiCache supports the Memcached text and binary protocol as of version 1.4.5 of Memcached.
Amazon ElastiCache is ideally suited as a front-end for Amazon Web Services like Amazon RDS and Amazon SimpleDB, providing extremely low latency for high performance applications and offloading some of the request volume while these services provide long lasting data durability. The service can also be used to improve application performance in conjunction with Amazon EC2 and EMR.
Memcached client libraries are available for many, if not all of the popular programming languages. See the list of available clients at the Memcached project page here. If you encounter any issues with specific Memcached clients when using Amazon ElastiCache, please engage us via the Amazon ElastiCache community forum.
Amazon ElastiCache does not require specific client libraries and works with existing Memcached client libraries without recompilation or application re-linking (Memcached 1.4.5 and later); examples include libMemcached (C) and libraries based on it (e.g. PHP, Perl, Python), spyMemcached (Java) and fauna (Ruby).
Auto Discovery is a feature that saves developers time and effort, while reducing complexity of their applications. Auto Discovery enables automatic discovery of cache nodes by clients when they are added to or removed from an Amazon ElastiCache cluster. Until now to handle cluster membership changes, developers must update the list of cache node endpoints manually. Depending on how the client application is architected, typically a client initialization, by shutting down the application and restarting it, is needed resulting in downtime. Through Auto Discovery we are eliminating this complexity.
With Auto Discovery, in addition to being backwards compatible with the Memcached protocol, Amazon ElastiCache provides clients with information on cache cluster membership. A client capable of processing the additional information reconfigures itself, without any initialization, to use the most current nodes of an Amazon ElastiCache cluster.
An Amazon ElastiCache cluster can be created with nodes that are addressable via named endpoints. With Auto Discovery the Amazon ElastiCache cluster is also given a unique Configuration Endpoint which is a DNS Record that is valid for the lifetime of the cluster. This DNS Record contains the DNS Names of the nodes that belong to the cluster. Amazon ElastiCache will ensure that the Configuration Endpoint always points to at least one such “target” node. A query to the target node then returns endpoints for all the nodes of the cluster in question. After this, you can connect to the cluster nodes just as before and use the Memcached protocol commands such as get, set, incr and decr. For more details, see here.
To use Auto Discovery, you will need an Auto Discovery capable client. Auto Discovery clients for Java and PHP are available for download from the Amazon ElastiCache console. Upon initialization, the client will automatically determine the current members of the Amazon ElastiCache cluster using the Configuration Endpoint.
When you make changes to your cache cluster by adding or removing nodes or if a node is replaced upon failure, the Auto Discovery client automatically determines the changes and you do not need to initialize your clients manually.
To get started, download the Amazon ElastiCache Cluster Client by clicking the “Download ElastiCache Cluster Client” link on the Amazon ElastiCache console. Before you can download, you must have an Amazon ElastiCache account; if you do not already have one, you can sign up from the Amazon ElastiCache detail page. After you download the client, you can begin setting up and activating your Amazon ElastiCache cluster by visiting the Amazon ElastiCache console. More details can be found here.
No, you will not get the Auto Discovery feature with the existing Memcached clients. To use the Auto Discovery feature a client must be able to use a Configuration Endpoint and determine the cluster node endpoints. You may either use the Amazon ElasitCache Cluster Client or extend your existing Memcached client to include the Auto Discovery command set.
To take advantage of Auto Discovery, an Auto Discovery capable client must be used to connect to an Amazon ElastiCache Cluster.
Amazon ElastiCache currently supports Auto Discovery capable clients for both Java and PHP. These can be downloaded from the Amazon ElastiCache console. Our customers can create clients for any other language by building upon the popular Memcached clients available.
You can take any Memcached Client Library and add support for Auto Discovery. If you would like to add or modify your own client to enable Auto Discovery, please refer to the Auto Discovery command set documentation.
Yes, we are looking at Ruby next and may add more languages after that.
Yes, Amazon ElastiCache is still Memcached protocol compliant and does not require you to change your clients. However, for taking advantage of auto-discovery feature, we had to enhance the Memcached client capabilities. If you choose to not use the Amazon ElastiCache Cluster Client, you can continue to use your own clients or modify your own client library to understand the auto-discovery command set.
Yes, the same Amazon ElastiCache cluster can be connected through an Auto Discovery capable Client and the traditional Memcached client at the same time. Amazon ElastiCache remains 100% Memcached compliant.
Yes, you can stop using Auto Discovery anytime. You can disable Auto Discovery by specifying the mode of operation during the Amazon ElastiCache Cluster client initialization. Also, since Amazon ElastiCache continues to support Memcached 100% you may use any Memcached compatible client as before.