Editing Chronic disease

==Background==
Chronic Disease is a contract managed by the Health team. This article describes a custom process to manage data for these clients during the transition phase between [[Aiga]] and [[Indici]].

==Rationale==
As at June 2024 the health team are recording Chronic disease contract activity in Indici.

Because Aiga is still in use - and still acts as the key data source for referral and allocation data - it is necessary to merge and reconcile the two data sets.

The general workflow for this pipeline is to:
# Source a set of current Chronic disease clients from Aiga
# Push data set to S3<ref>Bucket <code>kpa-aiga</code>.</ref>
# Write data set to RDS<ref>Schema <code>aiga_staging</code>.</ref>
# Build reconciled patient cohort, and count activities for reporting.

==Technical information==
===Sourcing cohort data from Aiga===
A custom batch job using python is scheduled to run on the Windows server every day at 4pm.<ref>Need to check timing of job and adjust if necessary</ref>

This executes a SQL query to find current clients allocated under the Chronic disease contract. The result set is then saved locally as parquet with a date key.

===Push data to s3===
The script then pushes the latest parquet file to <code>s3://kpa-aiga</code>.<ref>A service role <code>sql-backup-user</code> is utilised which has minimum necessary permissions.</ref>

A lifecycle policy expires files after 30 days.

===Load to RDS===
PUT into S3 bucket triggers lambda function <code>aigaLoadPatient</code>. This function truncates target table <code>aiga_staging.patient</code> then inserts every record in new parquet data. A history is not required, so no redundancy is built into this process.

===Reconcile data===
Finally, database function <code>rpt.fn_merge_patient_aiga_indici()</code> runs on a cron schedule and performs the following actions:
* Gets patients flagged in Indici as Chronic disease patients.<ref>Via finding the 'chronic disease' string in field <code>PracticeRemarks</code></ref>
* Gets the Aiga records from <code>aiga_staging.patient</code> that already have an NHI.
* Gets the remaining Aiga records that don't have an NHI available.
* These 3 sets are then reconciled and matched against each other to try and find an NHI identifier for all patients. Result is written to <code>rpt.patient_aiga_indici</code>.
* This reporting dataset can now be used as a Chronic Disease 'cohort', and identifiers used to look up consults/interactions for them in Indici.
* Sweet!

==References==