I used to use WD-40 heavily for cleaning just about everything like this. These days I use it very rarely for anything at all.
I use two products for collet systems. Clean everything with flushing solvent, fast evaporating no residue, everything meaning holder, collet, collet nut, and tool. Then I spray aerosol silicon oil inside the nut and in the collet seat and on the threads of the holder. No oil on the collet or the tool. This means that every interface is lubricated except between the collet and tool, which has no oil at all.
I have two rules for collets in the shop that everyone follows - assembling them this way, and never using the collapse range of the collet except for drills. This way holders and collets stay in good condition and we haven't had a tool move in a collet in several years.