Yours.
If you can’t do things on your own terms due to work commitments, whose job is it? Who really owns the job? Are you occupying the space because you need to, or because they need someone to?
They say it’s your job. But is it really? If it was truly your job, you would have more control over what goes on.
Once you have a financial buffer, it’s fine to take back time for yourself, even more so it should be encouraged. It’ll make you better at your job. That’s whose job it is.