The 25-24 loss in Melbourne left the Highlanders with two wins from three pre-season games, after previously beating the Crusaders and Brumbies away from home.
Although the new season kicks off next weekend, the Highlanders have a first-round bye and will not play their first game until February 21, against the Crusaders at Forsyth Barr Stadium.
With the team's three official pre-season games in the bag, Joseph said the team needed to work on consistency.
''We have played in bits and pieces, really,'' he told the Otago Daily Times from Melbourne.
''Sometimes we are looking very classy and sometimes we are looking average. So, I guess consistency would be something we need to concentrate on going forward into the season.
''It's a marathon, this competition - it's not about one or two games. [Thursday night] showed you can drop your concentration for five minutes and games can be snatched away from you.
''Those are all mental things we need to consider in the next few days and work on that.''
The Highlanders led the Rebels 24-13 with a few minutes to go before Rebels centre Mitch Inman and winger Dom Shipperly crashed over to snatch victory.
Joseph said it was ''disappointing'' to ''give the game back to the opposition'' after coming back from a 13-8 halftime deficit to lead by 11 points in the final minutes.
''But you have to give a bit of credit to the Rebels, because they came firing back at us. But we were good enough to win that game and we just didn't - so that's something we can work on,'' he said.
Joseph tried a string of different combinations during the final pre-season game and, minus a few ''dings and dongs'', the squad is healthy two weeks out from its first game.
Tasman first-five eighth Marty Banks, who missed the first two games with injury, got his first run in a Highlanders jersey and kicked two penalties and a conversion.
While the former Hurricanes pivot is ''obviously not at peak form'', Joseph liked what he saw.
''Marty showed glimpses and he's going to be a good player in the Highlanders jersey. We just need to give him some more opportunities,'' he said.
The side arrived back in Dunedin yesterday and has been given the weekend off.
Training will resume on Monday and the players will have a game among themselves on Friday.
Joseph is still a ''wee bit away'' from deciding on a preferred starting line-up for the season-opener. Competition for starting spots was ''really competitive'', particularly in the forwards, he said.
Two weeks out from hosting the Crusaders in front of what should be a bumper crowd at Forsyth Barr Stadium, Joseph is not yet getting too excited.
''We can't get too excited because it's two weeks away. We have got another week of training before we are playing a game of rugby. But if you're talking this time next week, there will be a lot more excitement in the camp.''
