Might I suggest that you install a washer between the windshield washer bottle and the frame then re-install the fastener. It moves the bottle neck far enough away from the frame where it won't "rattle" permanently. It took me about an hour because I had to remove the inner fender well to get access to the washer mounting bolts.
Rattles really piss me off, so I'm glad you found out what it was.