Category: Amazon SDB


Upcoming Webinars

There are a number of webinars scheduled over the next several weeks, and the purpose of this post is to make certain everyone is aware of the various options and opportunities:

  • Thursday, April 16 at 9:00am PST
    Cloud for Large Enterprise — Where to Start
    Hear Capgemini put AWS and cloud computing in context for large enterprises during this live webinar now taking place on April 16. You’ll learn key steps for creating a cloud strategy that works with your business and discover ways that cloud computing can lower costs while accelerating development. Speakers will include Amazon Web Service’s Terry Wise, who is a Director of Business Devleopment, Simon Plant, who is a Chief Enterprise Architect at Capgemini, and Andrew Gough, also with Capgemini. Register here. (Business level discussion)
     
  • Tuesday, April 21 at 9:00am PST
    ERP in the Cloud
    The AWS community includes a number of innovative ISVs. Compiere is one great example of the innovation, and has released ERP software running on Amazon Web Services. This one-hour non-technical session will include Compieres CEO Don Klaiss, a Compiere customer, and a few comments by me. Register here. (Business level discussion with a bit of light technical content)
     
  • Wednesday, April 22 at 9:00am PST
    Amazon SimpleDB Brown Bag As previously blogged, the development team will host this webinar. Several topics will be included; and I am excited that I will be able to present a proxy class that allows Visual Studio developers to integrate Amazon SimpleDB into Visual Studio 2008. Registration details are in the blog post, or here. (Developer-focused technical content)
     
  • Thursday, April 23 at 9:00am PST
    Introduction to Amazon Web Services for IT Professionals
    Attend this April 23 webinar for a live demonstration of how to get started using Amazon Web Services. AWS technical evangelist Mike Culver will present an IT-oriented overview of all AWS products, including an in-depth discussion on using Amazon S3 for cloud storage and Amazon EC2 for cloud computing. Register here.(Both business and technical content, erring on the side of “technical”)
     

Mike

Webinar: Amazon SimpleDB Developer Brown Bag

Join the Amazon SimpleDB team for the latest in our ongoing Developer Brown Bag series.

Topics will include:

  • Recap of recently released features
  • A step-by-step SimpleDB tutorial by AWS Evangelist Mike Culver
  • Your questions

We will take live questions via the webinar console, but to allow for a more thorough response, attendees are encouraged to pre-submit questions to simpledb-developer-brownbag@amazon.com.  If you’re struggling with a specific coding issue or are experiencing unexpected SimpleDB results and are willing to share details with the audience, the SimpleDB team will be happy to address these as well.

Title:
Amazon SimpleDB Developer Brown Bag
Date: Wednesday, April 22, 2009
Time: 9:00 AM – 10:00 AM PDT

Space is limited.
Reserve your Webinar seat now at:
https://www2.gotomeeting.com/register/266160655

– Jinesh

Amazon SimpleDB Batch Put, More Attributes Per Domain

We’ve just released an important new feature for Amazon SimpleDB. The new BatchPutAttributes function allows you to create or update up to 25 SimpleDB items at a time, in transactional fashion.

This new call is more efficient than a series of individual PutAttribute calls since one call incurs far less connection latency than a series of 25 calls. This new feature is available today and you can read about it here.

Update: Now that we’ve had a chance to benchmark the new batch put function in real-world operating conditions, we’ve found that it results in a doubling of the overall write throughput!

Also, based on feedback from the SimpleDB user community, we have increased the limit on the number of items per domain from 250 million to 1 billion.

— Jeff;

Amazon SimpleDB – New Count Function, Long Running Queries, Etc.

We’ve introduced two improvements to SimpleDB‘s Select API. You can now count the number of items which match a condition, and you can handle queries which take more than 5 seconds to run.

The new count function is used like this:

select count(*) from MyFriends where EyeColor = ‘Blue’

Queries which take more than 5 seconds to execute used to time out and returned no useful data. Now, queries will return all of the results accumulated prior to the 5 second time limit, along with a NextToken which will allow a subsequent query to pick up where the first one left off.

You can learn more about both of these features in the SimpleDB Release Notes.

By the way, the basic SimpleDB space allocation is 1 Terabyte (100 domains of 10 Gigabytes each). If you need more domains, fill in this form, send it our way, and and we’ll get you going within a day or two.

Other interesting SimpleDB resources include:

— Jeff;

SimpleDB Developer’s Brown Bag

You are invited to join the Amazon SimpleDB team on Tuesday, January 20, 2009 at 9am PST for the first session of our new Developers Brown Bag. During these once monthly webinar sessions, developers will hear from the technical experts behind SimpleDB, and have the opportunity to engage in live Q&A.

Interested developers may register by emailing simpledb-developer-brownbag@amazon.com. Please include name & AWS Account ID. Here’s where to find your AWS Account ID (that’s not my real ID so don’t try anything funny):

In addition, developers are encouraged to pre-submit any questions they may have, to allow for a more thorough response during the live webinar. For those struggling with the development of a new application, sample code and a description of the intended application may also be submitted for review and discussion.

The team is looking forward to speaking with developers on the 20th.

— Jeff;

Amazon SimpleDB – Now With Select

There’s now a new and somewhat easier way to write SimpleDB queries.

In addition to SimpleDB’s existing query language, you can now use select statements which look very similar to standard SQL (Structured Query Language). We made some small changes and additions to the language in order to accomodate SimpleDB’s unique multi-valued attribute model.

Here are some valid select statements:

select * from mydomain where city = ‘Seattle’
select * from mydomain where city = ‘Seattle’ or city = ‘Portland’
select * from mydomain where author not like ‘Henry%’

Things get even more interesting once multi-valued atttibutes are used. This query returns the items where the only attribute value for keyword is ‘Book’:

select * from mydomain where every(keyword) = ‘Book’

The following query returns items where the only value for keyword is ‘Book’ or Paperback’:

select * from mydomain where every(keyword) in (‘Book’, ‘Paperback’)

And the following query returns all the items which have the values ‘Book’ and ‘Hardcover’ in keyword:

select * from mydomain where keyword = ‘Book’ intersection keyword = ‘Hardcover’

You can also sort the results on any of the attributes that was used in the expression:

select * from mydomain where Year = ‘2007’ intersection Author is not null order by Author desc

The new SimpleDB Select function accepts queries in this new syntax. The existing Query and QueryWithAttributes functions are still usable, of course. There’s full information in the new version of the Developer Guide.

–Jeff;

Bizo SDB Tool

Bizo_simpledb_tool

Bizo is a B2B advertising network running entirely on top of AWS. They needed a GUI built on top of SimpleDB and decided to extend the Firefox plugin suite with the addition of SDB Tool.

The new Bizo SDB Tool provides a nice visual interface to Amazon SimpleDB in the form of a Firefox plugin. After installing the tool and entering your AWS account credentials, you can fetch the list of your SimpleDB domains, create and delete domains, and run queries. Query results are shown in outline form, and you can also add new items.

Source code for SDB Tool is kept in Github and can be found here. Read more on the Bizo blog.

— Jeff;

SimpleDB Domain Metadata Spec – Your Input Needed

The team behind Amazon SimpleDB is in the process of designing and implementing a new API call. The new call will return information about a particular SimpleDB domain. They have posted the design spec to the SimpleDB forum and would be very interested in receiving some more feedback.

I’ve said it before and I’ll say it again — much of what we do at Amazon is driven by real feedback from real customers. If you are using or are thinking about using SimpleDB, please take a look at the spec and let us know (via a post to the forum) what you think.

— Jeff;

Amazon SimpleDB Releases QueryWithAttributes

Amazon SimpleDB just released an update that includes a new feature called QueryWithAttributes. With this update, developers will be now able to retrieve all the information associated with items returned as a response to a particular query. The feature provides additional flexibility because it enables you to retrieve anywhere between one and all attributes for each item. This highly requested feature simplifies application development process for all clients of Amazon SimpleDB.  Instead of issuing a Query request followed by a series of GetAttributes requests, application designers can now use a single API call to retrieve all information about items stored in Amazon SimpleDB.

I am very excited about this new feature because it simplifies my application code. This is useful for developers who are not used to parallel programming or who utilize programming languages that do not support parallel programming.

The updated API documentation is here. I also highly recommend reading the Query 101, Query 102, and the best practices articles from our resource center. Amazon SimpleDB is still in limited beta. However if you sign up for the service at aws.amazon.com/SimpleDB youll be the first to know when additional applications are accepted.

If you dont list specific attributes in your query, then all attributes are returnedwhich is the default behavior of this new API method.

Theres a FAQ below; however I believe that examples always help developers understand what the changes mean in terms of writing code:

Current Query Syntax:
Domain = ZipCode
[State = WI]

Calling the Query method returns

 <QueryResult>
  <ItemName>24103</ItemName>
  <ItemName>23724</ItemName>
  Etc
</QueryResult>

New QueryWithAttributes Syntax
The new query is identical in every way except that the API has a new method. Calledof courseQueryWithAttributes:

Domain = ZipCode
[State = WI]

Calling the QueryWithAttributes method returns

<QueryWithAttributesResult>
  <Item>
    <Name>24103</Name>
    <Attribute><Name>state</Name><Value>WI</Value></Attribute>
    <Attribute><Name>zipcode</Name><Value>54936</Value></Attribute>
    <Attribute><Name>city</Name><Value>FOND DU LAC</Value></Attribute>
    <Attribute><Name>county</Name><Value>FOND DU LAC</Value></Attribute>
  </Item> 

<Item>
    <Name>23724</Name>
    <Attribute><Name>state</Name><Value>WI</Value></Attribute>
                  <Attribute><Name>zipcode</Name><Value>53703</Value></Attribute>
                  <Attribute><Name>city</Name><Value>MADISON</Value></Attribute>
                  <Attribute><Name>county</Name><Value>DANE</Value></Attribute>
  </Item>
Etc
</QueryWithAttributesResult>

FAQ
Q: Can I use the same query language?
Yes, the query language is exactly the same as for the regular Query API call.

Q: Will I get back the same set of items?
Yes, the overall set of items that will match a given query expression is exactly the same as that for the regular Query API call.

Q: How many attributes can I retrieve for each item?
You can retrieve anywhere between one and all attributes for each item.  The default behavior is to return all attributes, but you can specify a list of specific attributes to return.

Q: Will my result set be paginated?
Yes, Amazon SimpleDB paginates the result set if it exceeds specified maximum number of items or a total overall response size of 1 MB.

Q: How many items can I retrieve in one page of results?
You can indicate the maximum number of items to return per page that can be between 1 and 250 (default 100).  Amazon SimpleDB will attempt to return as many items as possible per page without exceeding the maximum byte size limit (1 MB) and the maximum number of items specified.

Q: What happens to page size if my attributes are very large?
Your page size will likely be smaller than the maximum number of items specified, since the overall size of the response object will approach the limit of 1 MB.

Q: Will I ever get one item split across multiple pages of results?
No, an item will never get split across multiple pages of results.  All specified attributes for a given item will be returned within the same page of results.

Q: Does the query timeout apply to my queries?
Yes, the same query timeout applies to long running queries.

Q: How much does each call cost?
The cost of each call is proportional to the amount of system resources that it consumes.  You can monitor the cost through the BoxUsage parameter, which is returned with every response.

Q. Is the Amazon SimpleDB Beta open to all comers now?
Amazon SimpleDB is still in limited beta. However if you sign up for the service at http://aws.amazon.com/SimpleDB youll be the first to know when additional applications are accepted.

— Mike

New features come to Amazon SimpleDB

Amazon SimpleDB released a new version last week. With this new version, developers will be now able sort the results and use a new does-not-start-with operator in their queries – the two most frequently asked feature requests.

I am very excited about the new sort feature because now all the processing will happen in-the-cloud and I will be able to execute scenarios like:

  1. Top ten scenario return top tem items based on some criteria (price, rating, etc)
  2. Most recent scenario – get X most recently updated items (sorted by LastUpdatedDate)
  3. Contact List scenario return items sorted alphabetically on Last Name

I would also like to highlight if you are storing your data on Amazon S3 and indexing your corresponding metadata on Amazon SimpleDB, this feature will be highly useful. Use this excellent library to index all your Amazon S3 object metadata in Amazon SimpleDB.

I also highly recommend reading the Query 101, Query 102 and the best practices articles from our resource center.

— Jinesh