Depends on why you want the bagger so bad.
If you're evolving into a total, long distance rider or always hitting the road 2 up, you might not ride your other bike much.
Otherwise, it seems like most folks who own two different types of bikes find the time to get some riding in on each one.