AWS Database Blog
2019: The year in review for Amazon Neptune
Amazon Neptune celebrated its 18-month birthday last month. It has been a humbling experience to learn the innovative ways customers want to use graph. NBCUniversal is using Amazon Neptune to manage a graph for serving curated, personalized content. Thomson Reuters is using graph to understand complex regulatory models. Netflix has improved data infrastructure reliability by using a graph-based system for data lineage. Zeta Global has built a customer intelligence platform that uses a graph based identity resolution to relate multiple devices and users. There are many more customers using Amazon Neptune in production.
Listening to our customers, we’ve also learned how to make Neptune better. We have made 12 engine releases in the span of 12 months to enhance developer experience, performance, enterprise features, compliance, and region expansion. The following 2019 releases are organized by category and then by release dates, with the most recent release at the top of each category. You can use this post to refer to features and fixes that are available in the product as well as comment on ones you would like to see in upcoming releases. As always, we would like to hear your input. Please drop a comment or reach out on @awscloud for any questions or feedback.
Developer experience
Amazon Neptune provides capabilities that make developing graph applications easy and fast.
- October 16: Amazon Neptune now supports Streams to capture graph data changes
Amazon Neptune supports Streams, an easy way to capture changes in your graph. When enabled, Neptune Streams logs changes to your graph (change-log data) as they happen. Neptune Streams are useful when you want to notify processes (e.g., trigger a lambda) as changes occur in your graph. Streams can also be useful to maintain a current version of your graph in a different region or service such as the Amazon Elasticsearch Service, Amazon ElastiCache, or Amazon Simple Storage Service (S3).
- October 16: Amazon Neptune now supports SPARQL 1.1 Federated Query
Using SPARQL 1.1 Federated Query, customers can execute a portion of their query across different SPARQL endpoints within their Virtual Private Cloud (VPC), combine the result, and return it to the user. Customers can distribute their data across multiple Neptune clusters and using a single query to access that data across those clusters. Customers can also use federation to combine data in Neptune with data from external SPARQL endpoints.
- November 26: Amazon Neptune offers full-text search integration with Elasticsearch clusters
Customers asked for an easy way to run full-text search on graph data. While graph query languages support basic text search, customers can access Elasticsearch’s built-in text indexing and query capabilities from Neptune. Customers can use an existing Elasticsearch cluster or create a new one to use with full-text search queries.
- December 04: Amazon Neptune Workbench provides in-console experience to query your graph
Amazon Neptune offers a workbench, which provides an in-console experience to query your graph. The workbench lets you quickly and easily query your Neptune databases with Jupyter notebooks – a fully managed, interactive development environment with live code and narrative text.
- December 19: Amazon Neptune supports Cross-region Snapshot Copying
Amazon Neptune supports copying snapshots (created either automatically or manually) across regions. AWS customers are interested in building applications that run in more than one public AWS Region. Customers can use cross-region snapshot copies with Neptune to simplify and streamline the data manipulation operations associated with building and running global graph applications.
Enterprise capability
Amazon Neptune’s decoupled storage and compute architecture offers capabilities to run graph databases in production.
- February 28: Resource Groups Tagging API Supports Additional AWS Services
You can use the Resource Groups Tagging API to centrally manage tags and search resources in Amazon Neptune. The Resource Groups Tagging API is a single central point reaching across 57 AWS services to programmatically perform operations including tag resources, untag resources, list unique tag keys, list tag values, and search for resources with specific tags.
- March 15: Amazon Neptune Announces 99.9% Service Level Agreement
AWS has published a service level agreement (SLA) for Amazon Neptune. We will use commercially reasonable efforts to make Multi-AZ clusters available with a Monthly Uptime Percentage of at least 99.9% during any monthly billing cycle. In the event Neptune does not meet the Service Commitment, you will be eligible to receive a Service Credit as described in the SLA.
- May 09: Amazon Neptune Now Supports Per Second Billing
Starting today, Amazon Neptune instances are billed in one-second increments. With on-demand pricing and no up-front commitment required, Neptune’s per-second billing provides additional granularity, so you only pay for capacity you use. Partial instance hours are billed in one-second increments, with a 10-minute minimum charge following a billable status change such as creating, modifying, or deleting an instance.
- June 17: You can now publish Amazon Neptune Audit Logs to CloudWatch
You can configure Amazon Neptune to publish audit logs to Amazon CloudWatch Logs. Publishing these logs to CloudWatch Logs allows you to maintain continuous visibility into database activity in your Amazon Neptune databases. For example, you can set up CloudWatch Alarms for events recorded audit logs to alert on unwanted changes made to your databases.
- July 26: Database cloning in Neptune
Using database cloning, you can quickly and cost-effectively create clones of all your databases in Amazon Neptune. The clone databases require only minimal additional space when they are first created. Database cloning uses a copy-on-write protocol.
Instances
Amazon Neptune launched with support for R4 instances, and shortly after, the service added support for R5 instances in all available AWS Regions. R5 instances can provide better performance over R4 instance for the same instance cost. As always, testing with your queries and data will give you the most accurate information about the improvements you’ll see for your workloads. With R5 instances, you can scale to db.r5.12xlarge instances that have 48 vCPU, 384 GiB of memory, and 10 Gbps network performance.
- May 09: Amazon Neptune Now Offers Improved Performance at Lower Cost with R5 Instances
You can launch R5 instance types when using Amazon Neptune. Amazon EC2 R5 instances, based on the Amazon EC2 Nitro System, are the next generation of the Amazon EC2 Memory Optimized instances offering superior performance at lower cost compared to previous-generation instances.
Performance and query improvements
We continue to improve performance for both loading as well as querying data.
- May 01: Analyzing Neptune query execution using SPARQL Explain
Amazon Neptune supports new explain feature to SPARQL queries to help you visualize the query plan and take steps to optimize it if necessary.
- July 26: Query hint for SPARQL query timeouts specifies a timeout for the SPARQL statement. It can appear in the
WHERE
clause of the main query or a subquery. In a SPARQL update, it can be set in theINSERT
,DELETE
, orWHERE
clause. - July 26: Added support for Apache TinkerPop 3.4.x including Gremlin text predicates.
- July 26: Added the ability to retrieve the status of a Gremlin query, and to cancel a Gremlin query
- July 26: Improved performance for Gremlin step ordering
.repeat.dedup
,valueMap()
andpath().by()
traversals.
- August 13: Added an
OVERSUBSCRIBE
option to the parallelism parameter of the Neptune Loader Command, which causes the Neptune bulk loader to use all available threads and resources.
- August 13: Improved performance of SPARQL filters containing simple logical OR expressions.
. - August 13: Improved Gremlin performance in handling conjunctive predicates.
- October 15: Improved Gremlin subgraph operator performance, Gremlin response serialization and the Gremlin Union step.
- November 22: Added the option using lab mode to create a fourth index (an OSGP Index) for large numbers of predicates.
- November 22: Added a detailsmode to SPARQL Explain. See Using SPARQL Explain and Details Mode Output for details.
- November 22: Added optimizations for Gremlin
.project()
and.identity()
steps, Gremlin.union()
cases. Added native support for Gremlin.path().by()
and.coalesce()
.
Compliance
Amazon Neptune is designed to meet the highest security standards and to make it easy for you to verify our security and meet your own regulatory and compliance obligations. Amazon Neptune has been assessed to comply with PCI DSS, ISO 9001, 27001, 27017, and 27018, and SOC 1, 2, and 3 in addition to being HIPAA eligible.
- May 17: Amazon Neptune Is Now SOC Compliant
Amazon Neptune is SOC 1, 2, and 3 compliant, allowing you to get deep insight into the security processes and controls that protect customer data. These reports are often leveraged by diverse industries, such as technology, healthcare, banking and financial services, and used for Sarbanes-Oxley (SOX) efforts.
Regions
Amazon Neptune launched last year with support for the US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Ireland), Europe (London), Europe (Frankfurt), and Asia Pacific (Singapore) Regions. Over the course of the year, we have rolled out Neptune in 16 Regions worldwide. To find the most up-to-date list, see the AWS Services Region Table.
- January 09: Amazon Neptune is Now Available in Asia Pacific (Sydney)
- January 23: Amazon Neptune is Now Available in the Asia Pacific (Tokyo) Region
- March 13: Amazon Neptune Now Available in Asia Pacific (Mumbai)
- May 01: Amazon Neptune is Now Available in Asia Pacific (Seoul)
- June 27: Amazon Neptune is Now Available in Europe (Stockholm)
- August 14: Amazon Neptune is Now Available in the AWS GovCloud (US-West) Region
- August 21: Amazon Neptune is Now Available in the AWS GovCloud (US-East) Region
- October 30: Amazon Neptune is Now Available in the Canada (Central) Region
- October 30: Amazon Neptune is Now Available in the Middle East (Bahrain) Region
It is still Day 1 for the Amazon Neptune team, and we look forward to delivering more capabilities for you in 2020 and beyond.
About the Author
Karthik Bharathy leads product for Amazon Neptune.