Manipulability stats for (some) poll methods
Here are voter manipulability stats for some of the poll methods, using
James Green-Armytage's spatial model with 4 dimensions, 4 candidates and
99 voters. Each method is tested on 500k elections, with 32k attempts to
strategize per election.
The manipulability value is the fraction of elections in this model
where the method elected a unique winner, and voters who preferred
somebody else to the current winner could get that somebody elected by
changing their ballots. Note that it does not check strategic nomination.
I've prefixed entries that aren't actually part of the poll with an
asterisk. I'll explain later why I've included them. Entries prefixed
with a number sign are from JGA as my simulator doesn't support them.[1]
The simulator uses full ballots, so Smith//DAC is the same as
Smith//DSC. If truncation would make the method more resistant, that's
not reflected here.
0.698 *Borda
0.668 #Approval (from JGA)
0.545 Condorcet//Borda (Black)
0.480 Copeland//Borda (Ranked Robin)
0.417 Plurality
0.417 Smith//DAC
0.412 *BTR-IRV
0.350 Baldwin
0.333 Raynaud (Gross Loser Elimination)
0.333 Schulze(wv)
0.332 Minmax(wv)
0.321 Ranked Pairs(wv)
0.075 Woodall, Schwartz-Woodall
0.074 RCIPE
0.074 IRV
0.074 Benham
I've included Borda to show that my results are similar to James Green
Armytage's. (Compare also the results minmax results.) In addition, I've
included BTR-IRV to see how well it would do. Too bad it didn't do
better, though...
My simulator show higher manipulability for IRV and the Condorcet-IRV
hybrids than JGA's simulator did. I think this comes down to that my
simulator is more thorough and thus is able to uncover more
nonmonotonicity- and pushover-related strategies.
Finally, I'm working on the cardinal methods, but the devil's in the
details so it's taking a lot longer than expected. More about that when
I've solved it. But my preliminary tests put Smith-Range around 0.5 and
STAR as worse than this. Unnormalized (fixed scale) Range, though not a
poll method, even does worse than Borda. And if the preliminary tests
give some indication, all the Range/Score-based methods do worse than
Plurality.
-km
[1] Green-Armytage, James (2011). "Four Condorcet-Hare hybrid methods
for single-winner elections". Voting matters (29): p. 7;
https://www.votingmatters.org.uk/ISSUE29/I29P1.pdf
Kristofer,
How did Approval interpret these fully ranked ballots?
Chris
On 27/04/2024 9:06 pm, Kristofer Munsterhjelm wrote:
Here are voter manipulability stats for some of the poll methods,
using James Green-Armytage's spatial model with 4 dimensions, 4
candidates and 99 voters. Each method is tested on 500k elections,
with 32k attempts to strategize per election.
The manipulability value is the fraction of elections in this model
where the method elected a unique winner, and voters who preferred
somebody else to the current winner could get that somebody elected by
changing their ballots. Note that it does not check strategic
nomination.
I've prefixed entries that aren't actually part of the poll with an
asterisk. I'll explain later why I've included them. Entries prefixed
with a number sign are from JGA as my simulator doesn't support them.[1]
The simulator uses full ballots, so Smith//DAC is the same as
Smith//DSC. If truncation would make the method more resistant, that's
not reflected here.
0.698 *Borda
0.668 #Approval (from JGA)
0.545 Condorcet//Borda (Black)
0.480 Copeland//Borda (Ranked Robin)
0.417 Plurality
0.417 Smith//DAC
0.412 *BTR-IRV
0.350 Baldwin
0.333 Raynaud (Gross Loser Elimination)
0.333 Schulze(wv)
0.332 Minmax(wv)
0.321 Ranked Pairs(wv)
0.075 Woodall, Schwartz-Woodall
0.074 RCIPE
0.074 IRV
0.074 Benham
I've included Borda to show that my results are similar to James Green
Armytage's. (Compare also the results minmax results.) In addition,
I've included BTR-IRV to see how well it would do. Too bad it didn't
do better, though...
My simulator show higher manipulability for IRV and the Condorcet-IRV
hybrids than JGA's simulator did. I think this comes down to that my
simulator is more thorough and thus is able to uncover more
nonmonotonicity- and pushover-related strategies.
Finally, I'm working on the cardinal methods, but the devil's in the
details so it's taking a lot longer than expected. More about that
when I've solved it. But my preliminary tests put Smith-Range around
0.5 and STAR as worse than this. Unnormalized (fixed scale) Range,
though not a poll method, even does worse than Borda. And if the
preliminary tests give some indication, all the Range/Score-based
methods do worse than Plurality.
-km
[1] Green-Armytage, James (2011). "Four Condorcet-Hare hybrid methods
for single-winner elections". Voting matters (29): p. 7;
https://www.votingmatters.org.uk/ISSUE29/I29P1.pdf
Election-Methods mailing list - see https://electorama.com/em for list
info
Kristofer,
The simulator uses full ballots, so Smith//DAC is the same as
Smith//DSC. If truncation would make the method more resistant, that's
not reflected here.
As the nominator of Smith//DAC I strongly request that in that case you
list it as Smith//DSC and accept that the Smith//DAC poll candidate
hasn't been simulated. It has a strong truncation incentive, and
forcing full strict ranking destroys most of its point.
BTW there is no justification for Smith//DSC versus Smith,DSC.
Also with the voters fully strict ranking, the three versions of Raynaud
become the same thing and those three "winning votes" Condorcet methods
become the same thing as Margins.
But nonetheless some of this is interesting, thanks.
Chris
On 27/04/2024 9:06 pm, Kristofer Munsterhjelm wrote:
Here are voter manipulability stats for some of the poll methods,
using James Green-Armytage's spatial model with 4 dimensions, 4
candidates and 99 voters. Each method is tested on 500k elections,
with 32k attempts to strategize per election.
The manipulability value is the fraction of elections in this model
where the method elected a unique winner, and voters who preferred
somebody else to the current winner could get that somebody elected by
changing their ballots. Note that it does not check strategic
nomination.
I've prefixed entries that aren't actually part of the poll with an
asterisk. I'll explain later why I've included them. Entries prefixed
with a number sign are from JGA as my simulator doesn't support them.[1]
The simulator uses full ballots, so Smith//DAC is the same as
Smith//DSC. If truncation would make the method more resistant, that's
not reflected here.
0.698 *Borda
0.668 #Approval (from JGA)
0.545 Condorcet//Borda (Black)
0.480 Copeland//Borda (Ranked Robin)
0.417 Plurality
0.417 Smith//DAC
0.412 *BTR-IRV
0.350 Baldwin
0.333 Raynaud (Gross Loser Elimination)
0.333 Schulze(wv)
0.332 Minmax(wv)
0.321 Ranked Pairs(wv)
0.075 Woodall, Schwartz-Woodall
0.074 RCIPE
0.074 IRV
0.074 Benham
I've included Borda to show that my results are similar to James Green
Armytage's. (Compare also the results minmax results.) In addition,
I've included BTR-IRV to see how well it would do. Too bad it didn't
do better, though...
My simulator show higher manipulability for IRV and the Condorcet-IRV
hybrids than JGA's simulator did. I think this comes down to that my
simulator is more thorough and thus is able to uncover more
nonmonotonicity- and pushover-related strategies.
Finally, I'm working on the cardinal methods, but the devil's in the
details so it's taking a lot longer than expected. More about that
when I've solved it. But my preliminary tests put Smith-Range around
0.5 and STAR as worse than this. Unnormalized (fixed scale) Range,
though not a poll method, even does worse than Borda. And if the
preliminary tests give some indication, all the Range/Score-based
methods do worse than Plurality.
-km
[1] Green-Armytage, James (2011). "Four Condorcet-Hare hybrid methods
for single-winner elections". Voting matters (29): p. 7;
https://www.votingmatters.org.uk/ISSUE29/I29P1.pdf
Election-Methods mailing list - see https://electorama.com/em for list
info
On 2024-04-27 14:10, Chris Benham wrote:
Kristofer,
How did Approval interpret these fully ranked ballots?
That's part of why I'm just referring to JGA. To do the simulation
myself, I would have to implicitly code a guideline that says where the
cutoff should be placed, given candidate-voter distances (which stand in
for absolute utilities). (The other part is that my simulator doesn't
support forcing the strategic ballots to be approval-style either yet.)
Unfortunately, James doesn't say just how he did it, so I'm CCing this
post to him. How were the approval ballots generated in "Four
Condorcet-Hare hybrid methods"?
In the absence of any information, I'd guess he used above-mean utility
thresholding. The strategic ballots (used to try to flip the winner)
don't have to care about utility at all: that process just tries
approval ballots at random until something works.
-km
On 2024-04-27 14:44, Chris Benham wrote:
Kristofer,
The simulator uses full ballots, so Smith//DAC is the same as
Smith//DSC. If truncation would make the method more resistant, that's
not reflected here.
As the nominator of Smith//DAC I strongly request that in that case you
list it as Smith//DSC and accept that the Smith//DAC poll candidate
hasn't been simulated. It has a strong truncation incentive, and
forcing full strict ranking destroys most of its point.
My thought for adding it was like this:
Suppose that the honest voters have opinions about every candidate.
Then, if they're being honest, they could rank them all. If the honest
voters have to do defensive strategy to protect themselves in such a
case, that's a limitation of the method.
On the other hand, if the truncation incentive is an offensive strategy
that can be used once the honest ballots are submitted, then truncation
strategy will only increase the method's manipulability.
So the manipulability should still give a lower bound on its
manipulability in the case that honest voters just rank everybody.
That said, I could try to implement a quick and dirty truncation setup,
something like truncating a random number of ranks down, or truncating
at mean utility. Would that solve the problem, in your opinion?
Also with the voters fully strict ranking, the three versions of Raynaud
become the same thing and those three "winning votes" Condorcet methods
become the same thing as Margins.
I know. I'm using the names of the methods as indicated in the poll to
make it clear which ones I'm referring to, even though some of their
description is redundant.
-km
On Sat, Apr 27, 2024 at 06:10 Kristofer Munsterhjelm km_elmet@t-online.de
wrote:
On 2024-04-27 14:44, Chris Benham wrote:
Also with the voters fully strict ranking, the three versions of Raynaud
become the same thing and those three "winning votes" Condorcet methods
become the same thing as Margins.
What? How is wv the same as margins if people strictly rank all the
candidates?
Election-Methods mailing list - see https://electorama.com/em for list
info
Oh okay, full strict ranking. Didn’t occur to me because not everyone
will vote like that.
On Sat, Apr 27, 2024 at 06:31 Michael Ossipoff email9648742@gmail.com
wrote:
On Sat, Apr 27, 2024 at 06:10 Kristofer Munsterhjelm km_elmet@t-online.de
wrote:
On 2024-04-27 14:44, Chris Benham wrote:
Also with the voters fully strict ranking, the three versions of Raynaud
become the same thing and those three "winning votes" Condorcet methods
become the same thing as Margins.
What? How is wv the same as margins if people strictly rank all the
candidates?
Election-Methods mailing list - see https://electorama.com/em for list
info
Hi Kris, thanks for the results! They're definitely interesting.
That said, I'm not sure how useful a metric raw probabilities provide; I
don't think they provide a very strong measure of how severely each
system is affected by strategy. Missing are:
- How much do voters have to distort their ballots? Is it just truncation,
compression (as with tied-at-the-top), or full decapitation? - How hard is it to think of the strategy? Counterintuitive strategies
(e.g. randomized strategies or pushover) require large, organized parties
to educate their supporters about how to pull it off. This could be good or
bad depending on if you like institutionalized parties. Good: sometimes
people can't pull it off. Bad: this creates an incentive for strong parties
and partisanship. See the Alaska 2022 Senate race, where Democrats pulled
off a favorite-betrayal in support of Murkowski to avoid a center-squeeze. - Is a counterstrategy available?
- How feasible is the strategy (does it involve many or few voters)?
- How bad would the effects of the strategy be? Borda is bad not just
because it's often susceptible to strategy, but because it gives turkeys a
solid chance of winning.
IRV is a good example of this. It's usually not susceptible to strategy
(in the IAC model), but I think of it as one of the most strategy-afflicted
methods on this list. It's vulnerable to some particularly-egregious
strategies (decapitation), ones that are complex or difficult to explain
(pushover), and many strategies don't have a simple defensive
counterstrategy available (like truncation).
A low-probability but occasionally high-impact strategy might be the worst
of both worlds; voters get lulled into a false sense of security by a few
elections where strategy doesn't matter, then suddenly find a candidate
they dislike elected because they failed to execute the appropriate
defensive strategy.
Limelike,
Can you please define and explain the "decapitation" strategy? I
haven't heard of it.
And can you elaborate a bit on this? :
IRV is a good example of this. It's /usually/ not susceptible to
strategy (in the IAC model), but I think of it as one of the most
strategy-afflicted methods on this list. It's vulnerable to some
particularly-egregious strategies (decapitation), ones that are
complex or difficult to explain (pushover), and many strategies
[that?] don't have a simple defensive counterstrategy available (like
truncation).
Chris B.
On 29/04/2024 2:31 am, Closed Limelike Curves wrote:
Hi Kris, thanks for the results! They're definitely interesting.
That said, I'm not sure how useful a metric raw probabilities provide;
I don't think they provide a very strong measure of how
/severely/ each system is affected by strategy. Missing are:
- How much do voters have to distort their ballots? Is it just
truncation, compression (as with tied-at-the-top), or full decapitation? - How hard is it to think of the strategy? Counterintuitive
strategies (e.g. randomized strategies or pushover) require large,
organized parties to educate their supporters about how to pull it
off. This could be good or bad depending on if you like
institutionalized parties. Good: sometimes people can't pull it off.
Bad: this creates an incentive for strong parties and partisanship.
See the Alaska 2022 Senate race, where Democrats pulled off a
favorite-betrayal in support of Murkowski to avoid a center-squeeze. - Is a counterstrategy available?
- How feasible is the strategy (does it involve many or few voters)?
- How bad would the effects of the strategy be? Borda is bad not just
because it's often susceptible to strategy, but because it gives
turkeys a solid chance of winning.
IRV is a good example of this. It's /usually/ not susceptible to
strategy (in the IAC model), but I think of it as one of the most
strategy-afflicted methods on this list. It's vulnerable to some
particularly-egregious strategies (decapitation), ones that are
complex or difficult to explain (pushover), and many strategies don't
have a simple defensive counterstrategy available (like truncation).
A low-probability but occasionally high-impact strategy might be the
worst of both worlds; voters get lulled into a false sense of security
by a few elections where strategy doesn't matter, then suddenly find a
candidate they dislike elected because they failed to execute the
appropriate defensive strategy.
Election-Methods mailing list - seehttps://electorama.com/em for list info
On 2024-04-28 20:15, Chris Benham wrote:
Limelike,
Can you please define and explain the "decapitation" strategy? I
haven't heard of it.
That would be what's called favorite betrayal on the list.
-km