Is your cat stressed? Is that stressing you out?
Cats just don't like change. They may seem spontaneous, but no. They like a known schedule and no surprises (perhaps with the exception of when hunting). Even changes in your routine can be stressful (think weekend get-aways).