How to Sync

There are many articles on the internet that talk about how to design a sync engine or the steps of sync algorithms, but not much about the actual implementations. This is a simplified version of how Gibberish’s sync system works. I’m posting it here so I don’t have to remember all this again in the future.

To implement sync, we first need to add four fields to our models:

1. id: Usually, your models already have an id, which is an incrementing integer. If we change the id to a UUID, we can create new items even when we’re not online.

2. soft_deleted: When users delete an item, don’t erase it from the database. If you do, there will be nothing left to sync. Instead, set soft_deleted to true. Also, update your code so it only fetches items where soft_deleted = false.

3. server_updated_at: You only need this if you’re changing the updated_at that Rails gives you in the app (like if you want to order items by updated_at). This field should be updated automatically whenever anything changes on an item on the server. The app should never change it.

4. pending_sync: Add this to the models in your app. Add code to change pending_sync to true for your items whenever something changes and needs to be synced.

The app starts the syncing process by sending a request to the server. This request includes all items marked with pending_sync = true and the timestamp of the most recently updated item (server_updated_at).

When the server gets the request, it looks at each item. It checks if an item with the same id already exists. If it doesn’t, the server creates a new item. If it does, the server compares the updated_at timestamps. If the app’s item is more recent, the server updates its item with the information from the app’s item.

Next, the server sends a response back to the app. This response contains all items that have a server_updated_at timestamp later than the one sent in the request. This includes the items that the server just received, which shows that they have been synced.

Finally, when the app receives this response, it processes each item, either creating a new item or updating an existing one. The app also sets pending_sync to false for all items.