We have two fairly distinct products: the old app (sometimes called “live app” during redesign development, or production) and the new app (called the “redesign” usually). Both follow roughly the same stack/arch, but are significantly different and their parts are almost completely incompatible.
Both the “live app” and the redesign are React projects and share a common origin, but the redesigned frontend is by and large a complete re-write. As such, the live app should not be used to poach code or get technical inspiration, and may only be occasionally useful to understand the intention behind some specific features, or e.g. how integrations are supposed to work. The headline requirements for the redesign are 1) a PWA that is stable, secure, and intuitive; that 2) follows the designs in Sketch; and 3) can import data from users profiles in the live app in a more-or-less sensible and predictable way. The features of the redesign are different than in the live app, so the data migration will never be 1-to-1.
There are numerous inherited quirks in the redesign frontend, and many bad ideas we are still trying to banish. useServerProfile comes to mind - ideally we are aiming to maintain (in a context provider) a centralised React state of user data that matches the shape of backend data as exactly as possible. From there, CRUD operations can be abstracted to sync React state with backend/db state with the same operations, and maybe one sunny day we can use something like Redux + middleware to resiliently sync state automagically.
The backend (Node + Serverless) is more of an evolution of the “live app” backend than a complete redesign, but it is significantly different in some areas. Again, it would be well advised to ignore the old backend and focus on building a more RESTful, sensible new backend.
The crux of the migration. New or old, all user data is saved in DynamoDB and is therefore just one JSON blob per user (primary key: PassportId also acts as a foreign key for the cognito pool (production pool is for some reason called “devtwo” and will remain the same)).
Previously, whatever data was wanted in the frontend could be added to this blob, and schema changes were done without even realising we were doing them. In the new backend there are two schemata in the JSONschema format: version: 0.1 == v1.0.0 == ”optless” for old data; and version: 2 == v2.0.0 == ”optin” for new data. For migration, custom code exists in the ‘optin’ handler to migrate between these shapes. Ajv is used to validate user data against both schemata to confirm the migration. After that, validation is done against v2.0.0 in both the frontend and backend. Currently frontend and backend schemata are synced by copy and paste, but a git submodule probably makes the most sense in the future.
The code has been partially set up to allow further on-the-fly evolution of the schema. Schema version is kept in the data itself, and v1 can migrate to v3 by writing functions that migrate v1 → v2 and v2→v3 (the first of these already exists). In the future, when the backend reads from the db it should check the schema version and migrate it to latest before passing to the FE or applying patches, etc.
Because of reasons, we keep “optless” v1.0.0 data in a separate table (theoretically the old production table but for safety a copy should be made on release night). When a user first logs into the redesign frontend, it calls GET isoptlessprofile which tells us if this user has an old profile, but not a new one, and therefore whether they need to migrate. At this point the user has the choice to either begin fresh with a blank v2.0.0 “optin” profile, or, to opt in and migrate their data. If they choose this, we POST /optin which runs the Node migration code and saves the migrated data to the table prefixed with optin- . This endpoint also responds with the to-be-saved migrated data so that the front end does not need to wait for backend db propagation to proceed to the next step.
If the user has “optin” data already (or if we successfully requested an /optin ) then we proceed to the “professional profile migration” stage.
Almost every JSON object under the sun in Creative Passport land is referred to as a “profile” at least somewhere, in code or in user-facing phraseology. To be most clear we have some disambiguating terms:
The persona structure contains a boolean field isProfessionalProfile. If the frontend detects that none of the personae in a passport are “professional profiles,” we initiate a flow in the frontend called “professional profile migration.” The user is prompted to select which existing profile represents their human self for which e.g. a government ID could be verified. If none are selected, a new one can be created. It is mandatory for the working of the app that a user have exactly one professional profile. After selection, the necessary changes are made to the professional persona and those changes are PATCHed back to the db.