So you’ve admitted to asking someone during class about how their weekend was, which is completely irrelevant to the lesson and then you’re whining you got punished for it?
Nothing frustrates me more as a teacher than students like you who identify that they’ve done something incorrect but then refuse to accept responsibility and the consequences for it.
If you’re a repeat offender, which it sounds like you are, then you can’t expect this teacher to just deal with you by having a quiet word about it, since clearly that is having no effect on you. You’re wasting time he could be spending with students who want to be there and want some help. He would be doing a disservice to them by keeping you in there!