It could be both...I had the swing arm nut loose one time and had both wheel nuts loose causing same symptoms. Luckliy for me both case only required tightning things back up.
I have a brand new set of swing arm bearings if you are interested.
(they come free with the putchase of a set of ITP wheels and tires..ha ha..just kidding

)