Secure and scalable deduplication of horizontally partitioned health data for privacy-preserving distributed statistical computation

BMC Medical Informatics and Decision Making

Technical Advance

Kassaye Yitbarek Yigzaw¹  Antonis Michalas²  Johan Gustav Bellika³ 

[1] Department of Computer Science, UiT The Arctic University of Norway, 9037, Tromsø, Norway;Norwegian Centre for E-health Research, University Hospital of North Norway, 9019, Tromsø, Norway;Department of Computer Science, University of Westminster, 115 New Cavendish Street, W1W 6UW, London, UK;Norwegian Centre for E-health Research, University Hospital of North Norway, 9019, Tromsø, Norway;Department of Clinical Medicine, UiT The Arctic University of Norway, 9037, Tromsø, Norway;

关键词: Bloom Filter;    Data Reuse;    Deduplication;    Distributed Statistical Computation;    Data Linkage;    Duplicate Record;    Electronic Health Record;    Privacy;    Record Linkage;    Set Intersection;   

DOI  :  10.1186/s12911-016-0389-x

 received in 2016-07-27, accepted in 2016-11-10,  发布年份 2017

来源: Springer

【 摘 要 】

BackgroundTechniques have been developed to compute statistics on distributed datasets without revealing private information except the statistical results. However, duplicate records in a distributed dataset may lead to incorrect statistical results. Therefore, to increase the accuracy of the statistical analysis of a distributed dataset, secure deduplication is an important preprocessing step.MethodsWe designed a secure protocol for the deduplication of horizontally partitioned datasets with deterministic record linkage algorithms. We provided a formal security analysis of the protocol in the presence of semi-honest adversaries. The protocol was implemented and deployed across three microbiology laboratories located in Norway, and we ran experiments on the datasets in which the number of records for each laboratory varied. Experiments were also performed on simulated microbiology datasets and data custodians connected through a local area network.ResultsThe security analysis demonstrated that the protocol protects the privacy of individuals and data custodians under a semi-honest adversarial model. More precisely, the protocol remains secure with the collusion of up to N − 2 corrupt data custodians. The total runtime for the protocol scales linearly with the addition of data custodians and records. One million simulated records distributed across 20 data custodians were deduplicated within 45 s. The experimental results showed that the protocol is more efficient and scalable than previous protocols for the same problem.ConclusionsThe proposed deduplication protocol is efficient and scalable for practical uses while protecting the privacy of patients and data custodians.

【 授权许可】

CC BY   
© The Author(s). 2016

【 图 表 】

12864_2017_4186_Article_IEq26.gif

12864_2017_4004_Article_IEq7.gif

12864_2017_4186_Article_IEq25.gif

12864_2017_4186_Article_IEq23.gif

12864_2017_4186_Article_IEq21.gif

12864_2017_4186_Article_IEq34.gif

12864_2016_3477_Article_IEq11.gif

12864_2016_2796_Article_IEq14.gif

12864_2016_2821_Article_IEq47.gif

12864_2016_3477_Article_IEq10.gif

12864_2015_2055_Article_IEq94.gif

12864_2015_2055_Article_IEq92.gif

12864_2016_2796_Article_IEq9.gif

12864_2017_3600_Article_IEq1.gif

12864_2017_4274_Article_IEq10.gif

12864_2016_3098_Article_IEq57.gif

12888_2017_1504_Article_IEq1.gif

12864_2017_3670_Article_IEq1.gif

12864_2017_3527_Article_IEq1.gif

12864_2017_3910_Article_IEq1.gif

12864_2016_3477_Article_IEq2.gif

12864_2016_2821_Article_IEq35.gif

12864_2016_2912_Article_IEq3.gif

12864_2015_2296_Article_IEq159.gif

12864_2015_2296_Article_IEq158.gif

12864_2016_2463_Article_IEq4.gif

12864_2017_4020_Article_IEq37.gif

12864_2017_4020_Article_IEq34.gif

12864_2016_2443_Article_IEq14.gif

12864_2016_2443_Article_IEq13.gif

12893_2017_312_Article_IEq1.gif

12864_2017_3604_Article_IEq5.gif

12864_2016_3263_Article_IEq16.gif

12864_2016_2789_Article_IEq52.gif

12864_2016_3263_Article_IEq13.gif

12864_2016_3263_Article_IEq11.gif

12864_2016_2798_Article_IEq6.gif