Try this ... you need a keyboard and screen attached
- Shut down the Mac Mini
- Locate the following keys on the keyboard: Command, Option, P, and R. You will need to hold these keys down simultaneously in step 4.
- Turn on the Mac Mini.
- Press and hold the Command-Option-P-R keys. You must press this key combination before the gray screen appears.
- Hold the keys down until the Mac Mini restarts and you hear the startup sound for the second time.
- Release the keys.
- Rest the clock settings if necessary, as it may reset to the default.
I have found this to cure a problem with one troublesome CD drive I had ... you only need to do it once. However, I told this to somebody else and he said it didn't work for him so you are not the only one who has had this problem with the Apple USB SD and if you have it under warranty you may be able to return it and get a Samsung drive, which does not have this problem.
Apple is aware of this problem so may now have a cure. Surely they can't be happy their own drive prefers a PC to a Mac.