AWS Database Blog

2019: The year in review for Amazon Neptune

Amazon Neptune celebrated its 18-month birthday last month. It has been a humbling experience to learn the innovative ways customers want to use graph. NBCUniversal is using Amazon Neptune to manage a graph for serving curated, personalized content. Thomson Reuters is using graph to understand complex regulatory models. Netflix has improved data infrastructure reliability by using a graph-based system for data lineage. Zeta Global has built a customer intelligence platform that uses a graph based identity resolution to relate multiple devices and users. There are many more customers using Amazon Neptune in production.

Listening to our customers, we’ve also learned how to make Neptune better. We have made 12 engine releases in the span of 12 months to enhance developer experience, performance, enterprise features, compliance, and region expansion. The following 2019 releases are organized by category and then by release dates, with the most recent release at the top of each category. You can use this post to refer to features and fixes that are available in the product as well as comment on ones you would like to see in upcoming releases. As always, we would like to hear your input. Please drop a comment or reach out on @awscloud for any questions or feedback.

Developer experience

Amazon Neptune provides capabilities that make developing graph applications easy and fast.

  • October 16: Amazon Neptune now supports Streams to capture graph data changes
    Amazon Neptune supports Streams, an easy way to capture changes in your graph. When enabled, Neptune Streams logs changes to your graph (change-log data) as they happen. Neptune Streams are useful when you want to notify processes (e.g., trigger a lambda) as changes occur in your graph. Streams can also be useful to maintain a current version of your graph in a different region or service such as the Amazon Elasticsearch Service, Amazon ElastiCache, or Amazon Simple Storage Service (S3).
  • October 16: Amazon Neptune now supports SPARQL 1.1 Federated Query
    Using SPARQL 1.1 Federated Query, customers can execute a portion of their query across different SPARQL endpoints within their Virtual Private Cloud (VPC), combine the result, and return it to the user. Customers can distribute their data across multiple Neptune clusters and using a single query to access that data across those clusters. Customers can also use federation to combine data in Neptune with data from external SPARQL endpoints.
  • November 26: Amazon Neptune offers full-text search integration with Elasticsearch clusters
    Customers asked for an easy way to run full-text search on graph data. While graph query languages support basic text search, customers can access Elasticsearch’s built-in text indexing and query capabilities from Neptune. Customers can use an existing Elasticsearch cluster or create a new one to use with full-text search queries.
  • December 19: Amazon Neptune supports Cross-region Snapshot Copying
    Amazon Neptune supports copying snapshots (created either automatically or manually) across regions. AWS customers are interested in building applications that run in more than one public AWS Region. Customers can use cross-region snapshot copies with Neptune to simplify and streamline the data manipulation operations associated with building and running global graph applications.

Enterprise capability

Amazon Neptune’s decoupled storage and compute architecture offers capabilities to run graph databases in production.

  • February 28: Resource Groups Tagging API Supports Additional AWS Services
    You can use the Resource Groups Tagging API to centrally manage tags and search resources in Amazon Neptune. The Resource Groups Tagging API is a single central point reaching across 57 AWS services to programmatically perform operations including tag resources, untag resources, list unique tag keys, list tag values, and search for resources with specific tags.
  • March 15: Amazon Neptune Announces 99.9% Service Level Agreement
    AWS has published a service level agreement (SLA) for Amazon Neptune. We will use commercially reasonable efforts to make Multi-AZ clusters available with a Monthly Uptime Percentage of at least 99.9% during any monthly billing cycle. In the event Neptune does not meet the Service Commitment, you will be eligible to receive a Service Credit as described in the SLA.
  • May 09: Amazon Neptune Now Supports Per Second Billing
    Starting today, Amazon Neptune instances are billed in one-second increments. With on-demand pricing and no up-front commitment required, Neptune’s per-second billing provides additional granularity, so you only pay for capacity you use. Partial instance hours are billed in one-second increments, with a 10-minute minimum charge following a billable status change such as creating, modifying, or deleting an instance.
  • June 17: You can now publish Amazon Neptune Audit Logs to CloudWatch
    You can configure Amazon Neptune to publish audit logs to Amazon CloudWatch Logs. Publishing these logs to CloudWatch Logs allows you to maintain continuous visibility into database activity in your Amazon Neptune databases. For example, you can set up CloudWatch Alarms for events recorded audit logs to alert on unwanted changes made to your databases.
  • July 26: Database cloning in Neptune
    Using database cloning, you can quickly and cost-effectively create clones of all your databases in Amazon Neptune. The clone databases require only minimal additional space when they are first created. Database cloning uses a copy-on-write protocol.

Instances

Amazon Neptune launched with support for R4 instances, and shortly after, the service added support for R5 instances in all available AWS Regions. R5 instances can provide better performance over R4 instance for the same instance cost. As always, testing with your queries and data will give you the most accurate information about the improvements you’ll see for your workloads. With R5 instances, you can scale to db.r5.12xlarge instances that have 48 vCPU, 384 GiB of memory, and 10 Gbps network performance.

Performance and query improvements

We continue to improve performance for both loading as well as querying data.

  • July 26: Query hint for SPARQL query timeouts specifies a timeout for the SPARQL statement. It can appear in the WHERE clause of the main query or a subquery. In a SPARQL update, it can be set in the INSERT, DELETE, or WHERE clause.
  • July 26: Added support for Apache TinkerPop 3.4.x including Gremlin text predicates.
  • July 26: Improved performance for Gremlin step ordering .repeat.dedup, valueMap() and path().by() traversals.
  • August 13: Added an OVERSUBSCRIBE option to the parallelism parameter of the Neptune Loader Command, which causes the Neptune bulk loader to use all available threads and resources.
  • August 13: Improved performance of SPARQL filters containing simple logical OR expressions.
    .
  • August 13: Improved Gremlin performance in handling conjunctive predicates.
  • October 15: Improved Gremlin subgraph operator performance, Gremlin response serialization and the Gremlin Union step.
  • November 22: Added the option using lab mode to create a fourth index (an OSGP Index) for large numbers of predicates.
  • November 22: Added optimizations for Gremlin .project() and .identity() steps, Gremlin .union() cases. Added native support for Gremlin .path().by() and .coalesce().

Compliance

Amazon Neptune is designed to meet the highest security standards and to make it easy for you to verify our security and meet your own regulatory and compliance obligations. Amazon Neptune has been assessed to comply with PCI DSS, ISO 9001, 27001, 27017, and 27018, and SOC 1, 2, and 3 in addition to being HIPAA eligible.

  • May 17: Amazon Neptune Is Now SOC Compliant
    Amazon Neptune is SOC 1, 2, and 3 compliant, allowing you to get deep insight into the security processes and controls that protect customer data. These reports are often leveraged by diverse industries, such as technology, healthcare, banking and financial services, and used for Sarbanes-Oxley (SOX) efforts.

Regions

Amazon Neptune launched last year with support for the US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Ireland), Europe (London), Europe (Frankfurt), and Asia Pacific (Singapore) Regions. Over the course of the year, we have rolled out Neptune in 16 Regions worldwide. To find the most up-to-date list, see the AWS Services Region Table.

It is still Day 1 for the Amazon Neptune team, and we look forward to delivering more capabilities for you in 2020 and beyond.

 

About the Author

 

Karthik Bharathy leads product for Amazon Neptune.

 

 

 

View Comments