Are you sure you have the threaded aluminum retainer plate oriented correctly? Flat side should face up towards the mag plate….
Up and down movement is controlled by the fit between the mag plate/brass ring retainer/aluminum threaded retainer.
Side to side movement is controlled by the fit between the mag plate brass bushing and the crankcase boss it rides on…
The threaded aluminum retainer has a “step” machined into it that helps control the up/down movement, the brass retainer ring is sandwiched between these two pieces. So, if the aluminum threaded retainer is damaged/worn or perhaps the threads/screws are messed up, there will be up/down movement. The brass retainer ring could be worn down, or perhaps the surface of the mag plate that rides on the brass retainer is worn….
Tell us more about the history of this engine, is it new to you? If not, why is it that you are working on the ignition? Perhaps posting some pictures will help.
“Dimpling” the brass retainer is a common “fix” for a sloppy mag plate, but you must be very careful to avoid the situation you described from occurring….