DateTime serialization changes in the AWS SDK for .NET and AWSPowerShell modules

Summary: This blog post describes best practices for using DateTime values in distributed .NET programming. It also highlights some of the recent, related changes in the AWS SDK for .NET and the AWSPowerShell modules.

Today, it’s increasingly common to write code that interacts with external systems, and it’s very likely that these systems won’t share the same time zone configuration as our local machine.

The established best practice is to always store universal timestamps and, if necessary, store the related time zone information in a separate variable. Local times should be avoided everywhere except for UI components. All timestamps communicated to external systems, and AWS is no exception, should be in Universal Time.

For a .NET programmer, dealing with different time zones can be particularly error prone because of the idiosyncrasies of the DateTime type.

• A DateTime value, depending on its DateTimeKind, can represent either a Local time, a Utc time or even be Unspecified. For this reason, an API accepting a DateTime parameter cannot enforce a compile-time check requiring its value to be in Universal Time.
• DateTime operators (and also the Equals method) don’t consider the DateTimeKind value. Forgetting to verify if operands are of the same kind frequently results in unexpected behaviors

DateTime utc1 = new DateTime(2018, 11, 4, 8, 30, 0, DateTimeKind.Utc);
    //2018-11-04 08:30 UTC
DateTime utc2 = new DateTime(2018, 11, 4, 9, 30, 0, DateTimeKind.Utc);
    //2018-11-04 09:30 UTC

DateTime local1 = utc1.ToLocalTime(); //2018-11-04 01:30 Local (Daylight Saving)
DateTime local2 = utc2.ToLocalTime(); //2018-11-04 01:30 Local

DateTime local3 = new DateTime(2018, 11, 4, 8, 30, 0, DateTimeKind.Local);

utc1 == utc2; //false
local1 == local2; //true !!
utc1 == local3; //true !!

utc2 - utc1; //01:00
local2 - local1; //00:00 !!
local1 - utc1; //07:00 !!

• DateTime parsing methods convert strings to local timestamps by default, even when the original string was explicitly marked as Universal Time.

DateTime.Parse("2018-11-04T08:30:00Z", CultureInfo.InvariantCulture);
    //returns 2018-11-04 01:30 Local

We reviewed the AWS SDK for .NET libraries to address some inconsistencies in how non-Universal DateTime values are handled when they are communicated to AWS services. We identified that the DateTimeKind value of some DateTime properties was not honored when communicated to AWS services. This results in the values always being treated as Utc, even when the user provides them as Local or Unspecified. Starting with version 3.3.390.0 of the AWS SDK for .NET, such properties are marked as obsolete, and we have introduced replacement properties that honor the DateTimeKind value. The new properties are marked with the “Utc” suffix. The obsolete properties maintain the old behavior for backward compatibility.

Developers should review any usage of the obsolete properties and update their code to use the suggested replacements.

The following is a sample implementation for one of the new properties and its obsolete, backward-compatible counterpart. The corresponding serialization logic has been updated to use the new “Utc” property and will always invoke .ToUniversalTime() before serializing the value to be communicated to the AWS service.

private DateTime? _startTimestampUtc;
private DateTime? _startTimestamp;

public DateTime StartTimestampUtc
{
    get { return this._startTimestampUtc.GetValueOrDefault(); }
    set { this._startTimestamp = this._startTimestampUtc = value; }
}

[Obsolete("Setting this property results in non-UTC DateTimes not being marshalled correctly. " +
    "Use StartTimestampUtc instead. Setting either StartTimestamp or StartTimestampUtc results in both " +
    "StartTimestamp and StartTimestampUtc being assigned; the latest assignment to either of the " +
    "two properties is reflected in the value of both. StartTimestamp is provided for backward compatibility " +
    "only and assigning a non-Utc DateTime to it results in the wrong timestamp being passed to the service.", false)]
public DateTime StartTimestamp
{
    get { return this._startTimestamp.GetValueOrDefault(); }
    set
    {
        this._startTimestamp = value;
        this._startTimestampUtc = new DateTime(value.Ticks, DateTimeKind.Utc);
    }
}

We have also standardized the behavior of DateTime properties across the AWS SDK for .NET to treat DateTime values of kind Unspecified as Local.

Finally, we have extended these changes to AWS Tools for PowerShell, which internally relies on the AWS SDK for .NET. Starting with version 3.3.390.0, we have marked cmdlet parameters that don’t honor the DateTimeKind as obsolete and have introduced replacement parameters, marked with the “Utc” prefix.

We advise developers to review their scripts and watch out for deprecation warnings. This is especially important because PowerShell automatically converts string values to local DateTime values when passing them to cmdlets. As a result, you may be using a local timestamp even when specifying a universal string like “2018-11-04T08:30:00Z”.

The described changes apply to the following NuGet packages, starting with the specified version.