Federated social identity backup and restore

Version: 0.1.0 (released 7th Feb 2016)

This version:

https://the-federation.info/specs/backup-restore/0.1.0.html

Latest published version:

https://the-federation.info/specs/backup-restore

Editors:

Jason Robinson, mail@jasonrobinson.me

Contributors:

Jonne Haß, Senya

Repository:

Status of this document

This document is published as a working draft after preliminary discussion in the diaspora* user migration issue and the relevant Loomio discussion. The original spec draft was published in the diaspora* wiki.

Working on the specification

Work on this specification should be done via issues and pull requests in the git repository. Comments can also be given via other ways, for example The Federation mailing list or the above linked earlier discussions. Issues and pull requests are the recommended way to participate however.

Versioning

This specification document should follow Semantic Versioning with a 1.0 released on the acceptance of the first version by a both the editor and possible implementing platform.

Overview

Specification to deal with two common problems with decentralized social sites:

Dying servers (ie losing your identity due to an existing server disappearing without warning)
Server migration (ie not being able to move from one server to another without creating a whole new identity)

These two problems create lack of identity security and lack of continuity for users of these social networks.

The purpose of this specification is to provide means to protect the identity of users, not actual content. As such, content like posts, comments, likes, photos, or any other content type objects are not in scope of this specification.

NOTE! This specification assumes the servers implement public and private keys to verify authorship of content. If not, a platform implementing this specification should use these methods within this specification.

High level concept

Users can be backed up automatically to servers who opt-in to receive backup data
Users can download their backup manually, for importing to another server
The restore process migrates the user to the server he/she is restoring on
Backup data is encrypted to protect the private key contained
Backup is automatic and after the user chooses a passphrase, the user doesn’t have to do anything else, except store a piece of information

Terms and concepts

Term	Explanation
User/identity	An object that is backed up or restored
Server	A server that is home to the user/identity. Called ‘pods’ for example in diaspora*.
Handle	A network wide unique identifier for the identity. For example `user@domain.tld` or `https://domain.tld/user`. This could also be a GUID but to allow a user friendly restore process a human friendly identifier should be preferred.

Use cases and flow of actions

The following use cases are related to and describe the flow of actions in this specification.

User creation

User arrives at user creation.
User is asked to give a passphrase for backups. User can opt-out, which ends this use case.
Backup server is chosen from available servers automatically and stored in user data. In the case of no available servers available to backup to, exit here.
Jump to ‘Initializing a backup’.

Initializing a backup

User is sent an email containing information regarding: Where to restore in case of problems? What information to store that is needed for restoration (the passphrase).
Schedule initial sync immediately.

Manually choosing a backup pod

User goes to settings.
User chooses from a list a new backup server.
User chooses a passphrase.
Jump to ‘Initializing a backup’.

Manually downloading a backup archive

User goes to settings.
User chooses ‘Request my profile data’.

User requests restore (automatically backed up archive)

User follows an earlier saved link to restore page on backup server.
User gives the handle they want to restore. If handle is not found in backups, exit story.
Encrypted backup archive is verified against the public key for the existing remote profile. If this verification fails, exit this case.
User gives passphrase to open encrypted backup archive.
User is sent an email with a link provided to continue the restore process.

User requests restore (manually downloaded backup archive)

User follows a link to restore page on backup server.
User uploads their backup archive.
Server checks that user is known. If not the server should fetch the remote profile. If it cannot be fetched, restore should not be allowed.
Backup archive private key is verified against the public key for the existing remote profile. If this verification fails, exit this case.
User is sent an email with a link provided to continue the restore process.

User verifies data restore via email

User is asked to provide a password and username for their new local account.
A local account is created on the backup server and linked to the existing remote profile.
A new public/private key pair is generated for the user on the backup server, which becomes now the users home server (called just Server from now on).
Handle ‘Sending out a moved message’

Sending out a moved message

Server sends out “I’ve moved” federation messages to all other known servers, signed with the users old private key. The moved message should contain the users new public key.
If the moved message receives a non-2xx status code, schedule a resend. Exit this case.

Receiving a moved message (not users old home server)

Server verifies message against users current known public key. If unsuccessful, exit case.
Server stores new public key of user which is contained in the moved message.
Server fetches new profile as per moved message.

Receiving a moved message (old home server)

Handle use case ‘Receiving a moved message (not users old home server)’.
Server removes old local user account, making the user now a remote profile only.

Backup server receiving a backup

If this server doesn’t currently support backups at all, return a 403 status code. Exit case here.
If this server doesn’t currently support new backed up users and the backup is not for an existing backed up user, return a 403 status code. Exit case here.
Fetch remote user profile, if it doesn’t exist yet.
Verify backup message as normally against public key.
Store backup, replacing any previous version, if any.
If first backup for this user, send email to user that backups are being received and instructions on how to proceed if restore is wanted.

Becoming a backup server

Server admin enables setting to advertise backup readyness to other servers.

Sending a backup to a backup server

Server creates backup message for user and sends it to the designated backup server
On success, exit this case.
On receiving a 403 forbidden status back, choose a new backup server from available servers. Jump to ‘Initializing a backup’.
On other failure, increase failed backup delivery count in user data.
If failed count is over threshold, choose a new backup server from available servers. Jump to ‘Initializing a backup’.

Update information of servers wanting to receive backups.
If no backup servers previously existed, add backup server to all existing users who didn’t opt out, send them a notification and schedule initial sync (see ‘User creation’ -case).
If a server that was previously backup ready becomes not backup ready, choose a new backup server to all existing users who have that server currently, send them a notification and schedule initial sync (see ‘User creation’ -case).

Backup server discovery

Servers which support this specification should publicize a JSON endpoint at .well-known/x-acc-backup-restore. This endpoint should contain the following schema:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://the-federation.info/specs/backup-restore#backup_server_discovery",
  "type": "object",
  "properties": {
    "allow_backups": {
      "type": "boolean"
    },
    "allow_new_backups": {
      "type": "boolean"
    }
  },
  "required": [
    "allow_backups"
  ]
}

allow_backups - This is the main true/false setting whether a server will allow any backups, new or old.
allow_new_backups - This setting can be set to false (optionally, this is not required as a setting at all) if the server wishes to indicate that they wish not to receive new backup users, but are fine continuing to receive backups for existing backed up users.

Servers should query other known servers frequently (1/week minimum) to refresh this information.

TODO: To avoid an extra endpoint, should we use a version of NodeInfo instead?

Data specifications

Backup archive

The archive should, depending on the features offered by the server, contain the following data:

Profile
Contacts (or whatever the term that is used for a contact)
Contact lists (or whatever the term is used to group contacts)
Followed hashtags
Public and private key pair

Archive format

The archive should be in JSON format.

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://the-federation.info/specs/backup-restore#archive_format",
  "type": "object",
  "properties": {
    "email": {
      "type": "string"
    },
    "content": {
      "type": "string"
    }
  },
  "required": [
    "email",
    "content"
  ]
}

TODO: Define full schema of content or place content keys in the first level.

Archive encryption

The archive should be strongly encrypted using a passphrase given by the user.

TODO: Define encryption method.

Servers sending backups

Servers which send out backups should store the following extra information for users:

column	type	example
Backups opted out	boolean	`false`
Backup server	string	`sub.domain.tld`
Backup server receive route	string	`/receive/backups`
Backup server fail count	integer	0

Servers receiving backups

A dedicated table or other storage should be available for storing backups information.

column	type	example
Backed up handle (unique key)	string	`user@domain.tld`
Backup content	large text	(encrypted text content)

Settings should be available to allow advertising backup readyness to other servers.

Servers that have received backups should always allow restoring them, even if they stop allowing new backups to be received.

Delivery of backups

To protect from arbitrary storage of data and to validate backup ownership, the backup delivery needs to be signed with the user private key. A receiving server should check the signature against user public key before storing the backup.

Actual implementation on how to verify the delivered packages can be left to individual implementation. However, cross-platform compatibility would improve from using identical methods.

Platforms are free to restrict what platforms they deliver backups to, for example to ensure users are able to restore their identity to a place with a similar set of features available.

TODO: Give example of signing method.
TODO: Should signing method be specificied in the delivery schema?
TODO: Should backup servers advertise what signing methods they support? (if above)

Scheduling delivery

Servers should aim to backup the identities of users at minimum once per week.

Delivery package schema

The delivery JSON message needs to contain the following schema:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://the-federation.info/specs/backup-restore#delivery_package",
  "type": "object",
  "properties": {
    "handle": {
      "type": "string"
    },
    "backup": {
      "type": "string"
    }
  },
  "required": [
    "handle",
    "backup"
  ]
}

backup is signed using the user private key. This is done to validate the content of what is inside. Additionally, before signing, it is encrypted using a user chosen passphrase.

Status codes

Delivery of backup archive

A successful delivery of a backup should expect to receive 200, 201 or 202 status code. These should all be counted as successful delivery of backup.

Refusal to accept this backup should be indicated with a 403 status code.

Any other error code should be understood as temporary problems with receiving the backup.

Moved messages

On a successful restore of user identity, a moved message should be sent out to all known servers.

The moved message is signed with the users old private key and should contain the users new public key.

In the case of the moved message failing to be understood by the recipient, due to lack of support for this specification (non-2xx status code response), a retry should be scheduled. Since the users old private key is not kept, the scheduled moved message should contain the fully prepared moved message to send. A server should retry moved messages for a minimum of 6 months. The time between sending out retries can be lenghtened over time.

Moved message schema

The moved message JSON needs to contain the following schema:

{
  "$schema": "http://json-schema.org/draft-04/schema#",
  "id": "https://the-federation.info/specs/backup-restore#moved_message",
  "type": "object",
  "properties": {
    "old_handle": {
      "type": "string"
    },
    "new_handle": {
      "type": "string"
    },
    "new_public_key": {
      "type": "string"
    }
  },
  "required": [
    "old_handle",
    "new_handle",
    "new_public_key"
  ]
}

Receiving a moved message

A server receiving a moved message for an identity that exists locally either as a local user or a remote profile, should do the necessary internal changes to map the user to the new location, discarding then the old public key and old handle.

The server receiving a moved message should ensure all local and remote content stored now points to the new handle.

Security considerations

A user backup must be encrypted strongly so it can be safely sent for storage to other servers.

A user backups must not be allowed to be restored without email confirmation. The dual protection of passphrase AND email confirmation is to avoid identity theft in the case that the user passphrase leaks out (from sending server database leak for example).

All signed backup deliveries must be verified against a recent public key of the sending profile.

Servers should not allow restoring any uploaded backup archives unless the user to be restored can be found either existing as a remote profile or by fetching the to be restored remote profile. This is to protect against uploading of faked identity data for identities that have disappeared off the network.

User discovery and public key

This specification assumes any to be backed up users have their public key available via common known discovery routes.

TODO: Should the spec take opinion on where users can be discovered from?

License

This work is licensed under a Creative Commons Attribution 4.0 International License.