Front-End Web & Mobile

Building a Synchronization Framework for Android

This is the sixth and final part in a series on synchronizing data within an Android mobile app to the AWS Cloud.  Check out the full series:

So far in this series, we have shown how to build a notes app.  You built a local store (called a content provider) for the data, and integrated Amazon Cognito into the standard Android Accounts page.  Now it’s time to consider synchronization of data.  Synchronization is considered in two parts: a backend component and a frontend component.  In the previous post, you developed the backend component.  Today, you synchronize data after you develop the frontend component.

Alternatively, you can build an in-app service object that does the synchronization for you.  This alleviates much of the boilerplate code that you developed during this series.  However, writing a SyncAdapter that uses the official Android API has several distinct advantages. First, it interfaces with the ContentProvider, which is a standardized method of accessing data that can be shared with other components, such as with widgets.  Second, since the SyncManager schedules synchronizations to run with other synchronization tasks and enables the user to set up rules for synchronization, it is much more battery efficient.  Finally, it deals with some of the complexity of synchronization (such as incremental back-offs when things don’t work) for you.

Building a synchronization framework for your data is a five step process:

  1. Build a content provider to handle your data locally on the device.
  2. Build an account provider to handle backend authentication.
  3. Build a backend service on AWS to provide the cloud data to your mobile app.
  4. Build a sync adapter that includes the actual synchronization logic for your data.
  5. Build a sync service to run the synchronization process.

The previous posts in this series already described how to do the first three steps.  The next step is to create a sync adapter.  This is a specific implementation of the AbstractThreadedSyncAdapter.  It includes just one method – the onPerformSync() method.  This is called when a synchronization is requested.  It contains the logic necessary to do the synchronization.  Let’s take a look:

public class NotesSyncAdapter extends AbstractThreadedSyncAdapter {
    private final AccountManager mAccountManager;
    private static final String _TAG = NotesSyncAdapter.class.getSimpleName();

    public NotesSyncAdapter(Context context, boolean autoInitialize) {
        super(context, autoInitialize);
        mAccountManager = AccountManager.get(context);
    }

    @Override
    public void onPerformSync(Account account, Bundle extras, String authority, ContentProviderClient provider, SyncResult syncResult) {
        Log.d(_TAG, String.format("onPerformSync for %s", account.name));

        try {
            String authToken = mAccountManager.blockingGetAuthToken(account, AuthTokenType.DEFAULT_AUTHTOKENTYPE, true);
            Log.d(_TAG, String.format("Authentication Token = %s", authToken));
            
            // Submit dirty records one-by-one, clearing the isDirty record if there is a 200
            
            // Read the lastSynced record from a backing store
            
            // Retrieve the updated records since the lastSynced record
            
            // Store the lastSynced record in a backing store

        } catch (OperationCanceledException cancelerr) {
            Log.d(_TAG, String.format("Sync operation was cancelled: %s", cancelerr));
        } catch (Exception err) {
             Log.e(_TAG, String.format("%s: %s", err.getClass().getSimpleName(), err.getMessage()));
    }
}

Note that the code includes comments about the synchronization.  First, you send any dirty records to the backend, clearing the isDirty flag as you go.   The code uses a “last write wins” policy to bypass conflict resolution requirements.  This is probably not appropriate in most applications; you should manage conflicts between the server side and client side using a checksum capability.

After the dirty records are stored on the backend, you read any changed records from the cloud back down to the client. You store the changed records, overwriting the original version if needed.  To submit dirty records, first create a Cursor that finds all the dirty records. If there are dirty records, iterate through each record, submitting the JSON version of the record to the backend.  The response from the server is either in the 200 series or the 400 series.  If it is in the 200 series, then clear the isDirty record.  The query and manipulations with the ContentProvider were described earlier in this series.

That leaves the API submission.  When you created the API definition within Cloud Logic, it also generated a Cloud Logic SDK that is specific to the API definition that was generated.  Download the Cloud Logic SDK:

  1. Sign in to the AWS Mobile Hub console.
  2. Select your project.
  3. Click Integrate in the left hand menu.
  4. Click Download > Android under NoSQL / Cloud Logic Additional Resources.

The SDK is generated and packaged as a ZIP file, which you can unzip.   Copy the directories under src/main/java/com/amazonaws into the same place in your project (under app/src/main/java/com/amazonaws).  Be careful not to overwrite existing files.

Download and update the awsconfiguration.json file within your project.  After the merge, your project should look like this in Android Studio:

You can (and should) delete the models.nosql package because you do not directly access DynamoDB in this project.

Continue by adding the Amazon API Gateway SDK to your app/build.gradle file:

compile 'com.amazonaws:aws-android-sdk-apigateway-core:2.6.3'

You can now write the code to send each dirty record to the server:

final SyncMobileHubClient apiClient = new ApiClientFactory()
        .credentialsProvider(credentialsProvider)
        .build(SyncMobileHubClient.class);
final String apiClientPath = "/notes";

// Submit dirty records one-by-one, clearing the isDirty record if success is returned
Cursor dirtyRecords = provider.query(
        NotesContentContract.Notes.CONTENT_URI,
        NotesContentContract.Notes.PROJECTION_ALL,
        "isDirty = 1", null,
        null);
if (dirtyRecords != null && dirtyRecords.getCount() > 0) {
    for (dirtyRecords.moveToFirst(); dirtyRecords.isAfterLast(); dirtyRecords.moveToNext()) {
        final Note dirtyRecord = NotesConverter.fromCursor(dirtyRecords);

        // Store the updated Note to the backend
        final String body = dirtyRecord.toString(); // This returns JSON content
        final byte[] content = body.getBytes(StringUtils.UTF8);
        final ApiRequest request = new ApiRequest(apiClient.getClass().getSimpleName())
                .withPath(apiClientPath)
                .withHttpMethod(HttpMethodName.POST)
                .addHeader("Content-Type", "application/json")
                .addHeader("Content-Length", String.valueOf(content.length))
                .withBody(content);
        final ApiResponse response = apiClient.execute(request);
        if (response.getStatusCode() == 200) {
            // Clear the isDirty flag for the Note
            dirtyRecord.setIsDirty(false);
            provider.update(
                    NotesContentContract.Notes.uriBuilder(dirtyRecord),
                    NotesConverter.toContentValues(dirtyRecord),
                    null, null);
        } else {
            throw new RuntimeException(String.format("API-Error: %s", response.getStatusText()));
        }
    }
}

The Note.toString() method returns the JSON representation of the note as a string. You generate an ApiRequest that describes the request; in this case, it’s an HTTP POST with the updated note in the body. That produces an ApiResponse from the server.

You can do a similar thing when you pull new records from the server:

// Read the lastSynced record from a backing store
long lastUpdated = readLastSyncRecord();

// Retrieve the updated records since the lastSynced record
final ApiRequest pullRequest = new ApiRequest(apiClient.getClass().getSimpleName())
        .withPath(apiClientPath)
        .withHttpMethod(HttpMethodName.GET)
        .addHeader("Accept", "application/json")
        .withParameter("lastUpdated", String.format("%ld", lastUpdated));
final ApiResponse pullResponse = apiClient.execute(pullRequest);
if (pullResponse.getStatusCode() == 200) {
    // Retrieve the response and de-serialize into individual notes
    final InputStream pullResponseStream = pullResponse.getContent();
    final String json = IOUtils.toString(pullResponseStream);
    Type listType = new TypeToken<ArrayList<Note>>(){}.getType();
    List<Note> pullResults = new Gson().fromJson(json, listType);

    // Store each note in the database
    Iterator<Note> iterator = pullResults.iterator();
    while (iterator.hasNext()) {
        final Note updatedNote = iterator.next();
        Uri noteUri = NotesContentContract.Notes.uriBuilder(updatedNote);
        Cursor cursor = provider.query(noteUri,
                NotesContentContract.Notes.PROJECTION_ALL,
                null, null, null);
        if (cursor.getCount() > 0) {
            provider.update(noteUri, NotesConverter.toContentValues(updatedNote), null, null);
        } else {
            provider.insert(NotesContentContract.Notes.CONTENT_URI, NotesConverter.toContentValues(updatedNote));
        }
        // Update the last updated if needed
        if (updatedNote.getUpdated() > lastUpdated) {
            lastUpdated = updatedNote.getUpdated();
        }
    }
}

// Write the lastSynced record from a backing store
writeLastSyncRecord(lastUpdated);

In this case, all the new and updated records are pulled from the server.  Each one is handled in the mobile app. The app determines if each record needs to be stored in the database.  You store the last time you synchronized so that you don’t re-download records needlessly.

You need to register the sync adapter.  This is done in two parts.  First, there is an XML file called syncadapter.xml within the res/xml folder.  It has the following contents:

<?xml version="1.0" encoding="utf-8"?>

<sync-adapter xmlns:android="http://schemas.android.com/apk/res/android"
    android:contentAuthority="com.amazonaws.mobile.samples.notetaker.provider"
    android:accountType="com.amazonaws.mobile.samples.cognito"
    android:userVisible="true"
    android:allowParallelSyncs="false"
    android:isAlwaysSyncable="false"
    android:supportsUploading="true"/>

The contentAuthority is from the content provider and the accountType is from the authenticator.  These both need to match the rest of your app.  The rest of the content of this file is boilerplate.

Finally, add a service to run the synchronization framework.  This is also mostly boilerplate.  The following example shows this code in NotesSyncService.java:

package com.amazonaws.mobile.samples.notes.sync;

import android.app.Service;
import android.content.Intent;
import android.os.IBinder;

public class NotesSyncService extends Service {
    private static final Object syncAdapterLock = new Object();
    private static NotesSyncAdapter syncAdapter = null;

    @Override
    public void onCreate)() {
        synchronized (syncAdapterLock) {
            if (syncAdapter == null) {
                syncAdapter = new NotesSyncAdapter(getApplicationContext(), true);
            }
        }
    }

    @Override
    public IBinder onBind(Intent intent) {
        return syncAdapter.getSyncAdapterBinder();
    }
}

Replace the NotesSyncAdapter with the class name of your sync adapter.  Declare the service in the AndroidManifest.xml file:

<service android:name=".sync.NotesSyncService" android:exported="true">
    <intent-filter>
        <action android:name="android.content.SyncAdapter"/>
    </intent-filter>
    <meta-data android:name="android.content.SyncAdapter"
        android:resource="@xml/syncadapter" />
</service>

This is defined in the same way as the Authenticator service.  The resource field must point to the XML file you created earlier to define the sync adapter.  While you are updating AndroidManifest.xml, also add the following permissions:

<uses-permission android:name="android.permission.WRITE_SYNC_SETTINGS" />
<uses-permission android:name="android.permission.READ_SYNC_SETTINGS" />

WRITE_SYNC_SETTINGS gives you permission to add a sync adapter to the system.  This permission is required for the synchronization to work.  READ_SYNC_SETTINGS allows you to read the state of the sync adapter (such as if it is idle or currently syncing).  You can read the state of the sync adapter by registering a SyncStatusObserver within your application.

After you make these changes, run the code, exit the app, and go to Settings > Accounts in the Android emulator.  You should see the following screens after registering or adding an account:

Wrap up

There are a couple of problems with this method that are easily corrected.  First, if there are many records waiting to be pulled down, you should page through the records so that you can handle cases when the connection drops or if you have a memory limitation.  Second, as written, there is no conflict management.  Conflict management can be done automatically according to rules or manually through the UI.

This series has covered the Android Sync Framework from beginning to end.  In most cases, an in-app synchronization is perfectly adequate for synchronization, especially when the data is not shared across apps or widgets.  When such sharing is required, the infrastructure is available to handle the job effectively.