Membership and Memorization in LLM Knowledge Distillation

Pytorch implementation for the paper:
Membership and Memorization in LLM Knowledge Distillation
Ziqi Zhang, Ali Shahin Shamsabadi, Hanxiao Lu, Yifeng Cai, Hamed Haddadi

Scripts

The scripts for membership inference on distilled models are in scripts/model_family_6k. The scripts of memorization is in mia.

Environment Configuration

Please see README_distiLLM.md.

Acknowledgement

The code framework is based on DistiLLM and MiniLLM

Name		Name	Last commit message	Last commit date
Latest commit History 2 Commits
configs		configs
data_utils		data_utils
distillm		distillm
imgs		imgs
mia		mia
minillm		minillm
scripts/model_family_6k		scripts/model_family_6k
tools		tools
transformers		transformers
.gitattributes		.gitattributes
.gitignore		.gitignore
README.md		README.md
README_distiLLM.md		README_distiLLM.md
arguments.py		arguments.py
evaluate.py		evaluate.py
evaluate_main.py		evaluate_main.py
evaluate_memorization.py		evaluate_memorization.py
evaluate_memorization_w_teacher.py		evaluate_memorization_w_teacher.py
evaluate_mi.py		evaluate_mi.py
evaluate_mi_blimp.py		evaluate_mi_blimp.py
evaluate_mi_main.py		evaluate_mi_main.py
evaluate_mi_mope.py		evaluate_mi_mope.py
finetune.py		finetune.py
generate.py		generate.py
install.sh		install.sh
mi_utils.py		mi_utils.py
rouge_metric.py		rouge_metric.py
train_minillm.py		train_minillm.py
utils.py		utils.py

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

Membership and Memorization in LLM Knowledge Distillation

Scripts

Environment Configuration

Acknowledgement

About

Uh oh!

Releases

Packages

Languages

ziqi-zhang/LLM_Distillation_Privacy

Folders and files

Latest commit

History

Repository files navigation

Membership and Memorization in LLM Knowledge Distillation

Scripts

Environment Configuration

Acknowledgement

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages