But he was proud of the performance of the players and said he backed the way they kept trying to attack.
Jason Emery was sent from the field after he collided with Sharks fullback Willie Le Roux early in the first half.
"It was a massive impact on the game. How do you expect those 14 guys to play against some big men for all that time?'' Joseph said.
"What I was surprised about was the advice from the TMO was that it was not deliberate. They pondered over it for two or three times and then they made that decision.
"The TMO ruled it was not deliberate. It was not deliberate, you could clearly see that, but it was dangerous. So, if you go to the law book, then I guess he ruled it that way.''
Emery's absence forced the side to defend with real vigour and plenty of penalties came from both sides.
Joseph said referee Ben O'Keefe had a tough game to handle.
"I'm really proud of the team ... a lot of teams could have thrown the towel in but we could have won the game at the end. I guess that fatigue at the end, the skill level was not there.''
He said kicking for the corners and soaking up some pressure from the Sharks was not as easy as it sounded, especially with only 14 men to call on.
"You can have a defensive mindset but you are asking your players to defend 14 on 15 and that was not the way we wanted to play. But it put the players under a lot of pressure.''
Joseph had not spoken to Emery after the game.
Hooker Liam Coltman had taken a knock to his foot and was on crutches after the game.
Lock Joe Wheeler said the red card made it tough for the side.
"We had to stick to some really basic plays and could not do what we wanted to do. But what I think it showed was the immense character in this team and the guys worked ... for each other,'' Wheeler said.
Wheeler thought he might have scored in the first half but a Sharks defender got under him just as he went to dot down.
"They obviously knew the way we were going to play and they came hard at us. They are a pretty formidable pack.''
He said after the red card the side looked to each other for motivation.
"We said we have to just do it for each other. We can't control those sort of things. We just had to carry on, had to adapt to the plays we wanted to run. We just had to focus on next job and not dwell on these sort of things.''
Sharks head coach Gary Gold said he was happy with the result but not the performance.
The two yellow cards picked up by his side were disappointing. His side had the opportunities but could not finish off.
He had no doubt about the red card and said Emery had to be sent from the field. He said felled fullback Willie Le Roux still had a sore neck.
Gold felt Highlanders first five-eighth Lima Sopoaga was lucky not to be yellow-carded for a tackle with no shoulders in the first five minutes.
Halfback Cobus Reinach injured his knee from the incident and is out of the tour after the incident with Sopoaga.
No 8 Philip van der Walt is also off home with a sore neck.