Membership and Memorization in LLM Knowledge Distillation

Pytorch implementation for the paper:
Membership and Memorization in LLM Knowledge Distillation
Ziqi Zhang, Ali Shahin Shamsabadi, Hanxiao Lu, Yifeng Cai, Hamed Haddadi

Scripts

The scripts for membership inference on distilled models are in scripts/model_family_6k. The scripts of memorization is in mia.

Environment Configuration

Please see README_distiLLM.md.

Acknowledgement

The code framework is based on DistiLLM and MiniLLM