I disagree. Take 5 people, each with a capacity for freedom between 0 and 100. One of those people is equipped to enslave the other 4. Total freedom is going to be less than if the other 4 enslave / subdue the one.
you suggested some acts (like enslavement) rob the freedom of others, the alternative is robbing the man of his freedom to enslave---there is a 1 to 1 ratio with the act pertaining to freedom.
so if you have 5 people, one person can enslave each of the four, or he can have the freedom to enslave any of the four removed so that each of them can have their freedom to life as free people. Think about it, a jailer hasn't the right to imprison anyone, he only has a 1:1 ratio, he has the freedom to restrict the physical freedom of one person who is deemed a criminal, he isn't given 'freedom to imprision' which he can then use on 1 billion people meaning he gains 1 freedom point rather than the world having 1 billion freedom points and thus the 'sum' being different or some such distorted sum. to say 'you have the freedom to take the freedom of this person, but not that person' means either you've lost a freedom, or someone else has lost a freedom to you, there is no accumulation
edit:
and to anticipate your objection
to get down to details you might say, no, he gets one freedom to own someone, but if he didn't have that freedom, they'd have... the freedom to do as they want, eat what they want, work where they want, die if they want, live where they want, and you could name 100 'freedoms' which
look like a sum greater than the linguistic 'one freedom' he (The slave owner) has if he rather than his potential slave is given the freedom.
the 1:1 though does continue, and I think that is blatent in modern society under government. In the scenario above, he would have the freedom to say what they eat/work/etc. rather than them having each one of those freedoms, it isn't one package for him but counted as single things for the mathematics... look at society, you have a freedom to work/eat/fuck/etc. but you dont' have the freedom of not paying tax/of smoking whatever drugs you want, having sex with any aged human you want, etc... you lose individual freedoms to what they gain (The freedom to restrict that one act), so in a sense we are that slave, and clearly it isn't that the government has 1 freedom point for itself, where as if America had no government there would be 300,000 points, which means the total sum of freedom is far greater, no, it's just that a number of the freedoms are on the governments side, giving them to one person, and the next person, one freedom of one act to each person doesn't increase the number, they withheld those freedoms from each individual, so the total sum remains the same, so as I suggested, it's only the distribution which has changed, and a distribution where between individuals there is little variation is what people seem to aim for in our sort of society (none of us have the right to smoke weed or crack, if some of us did, people wouldn't be happy about that, its individuals each having the same rights or lack there of which is strived for, not a gain of a 'sum total' as if you would be pleased at some 'increase in sum total of freedom' if women were allowed to sell drugs but men were not given that freedom.)