Overview
PrecisionLender is constantly improving and updating the data and information architecture which leads to evolving datasets. Our Data Studio datasets go through changes in structure, leading to version changes and deprecation of older versions. As client integrations have dependencies on these datasets, PrecisionLender maintains a dataset version and deprecation process.
In this Article
- Dataset Deprecation Timeline
- Deprecation Communication
- Dataset Move to "Deprecated" Path
- Completing Deprecation
Dataset Deprecation Timeline
When a dataset is marked for deprecation, the steps and timeline are as follows:
- 2 months out - email notification stating the deprecation is pending and scheduled.
- 5 weeks out - email reminder regarding deprecation plan.
- 1 month out - dataset is renamed and prefixed with "Deprecated-" and remains in the same dataset stage.
- 1 week out - final email reminder regarding deprecation.
- 0 days out - The job producing the deprecated dataset is disabled, leaving the deprecated dataset in a static form (under a "Deprecated" name) for continued reference without any updates.
- e.g. /{root}/{dataset_group}/Deprecated-{dataset_name}
Deprecation Communication
The deprecation email to affected clients will be sent with a request for confirmation and will contain the following:
- Statement explaining why the deprecation is taking place and whether there is replacement data
- If there is replacement data, instructions for accessing the replacement data will be provided
- Timeline for deprecation
- Effect on the clients' systems at each stage of the deprecation process
Dataset Move to “Deprecated” Path
We will move the deprecated dataset to the “Deprecated” stage. The client will still be able to access the dataset but under a different path. In order to access the dataset, the client will simply have to update the paths in their jobs to point to the new location.
The purpose of moving the data to the Deprecated path is to make clear and visible to the client the effect of our data change on their processes and systems.
The dataset move will be conducted via a standard “move” job by the Data Engineering team. At the same time the move is conducted, the batch job producing the dataset in question will be repointed to write into the new location.
Completing Deprecation
On the final day of deprecation, the job generating the dataset will be turned off. The deprecated dataset will no longer continue to update.
