The belt is directional. If it started doing that all at once the first thing I would check is for asphalt or rocks or junk in the belt or back sprocket, that would push the belt over till it rubbed sometimes.
Also the belt should run close to the left (outside) but not rubbing, that is the way it is designed. Did you just change a tire, how is the rubber dampers seating where the sprocket fits in the hub (is the sprocket running true).
Shouldn't be any play in the swingarm, I would fix that right away. That could cause wobble. Have to pull the header pipes off to get the swingarm bolt out, so order some gaskets. The rest is basic stuff that you take off anyways to change a tire. Good time to change the belt if it needs it, it might be the problem like was said above.
Good luck and keep us posted,