The question, am I better off now than four years ago? assumes that what has made me better or worse off are things that happened within the last four years and can clearly be attributed to the sitting president. Neither are true. It’s not a bad thing to be thinking about, what has made me better or worse off?, but that inquiry will range over years and will involve not just presidents but congresses and businesses and a lot else besides.
My point is, you should always be paying attention to all of those things. Don’t just check in every four years and think the presidential election is where you go to express your happiness or frustration. It doesn’t work like that.