What Reproducibility Costs in a Parallel Program
A genetic algorithm is made almost entirely of random choices. The initial population, which parents get selected, where crossover cuts, which genes mutate, whether to restart after stagnation, which random immigrants get injected, where local search starts. A seed parameter promises that all of it repeats.
Delivering on that promise in a single-threaded program is bookkeeping. Delivering it once the islands run in parallel is an architectural constraint, and it is worth being precise about why.
The single-threaded version is just discipline
One seeded generator, threaded through every decision, and you are done:
result_a = minimize(f, genome_length=10, bounds=(-5, 5), seed=42)
result_b = minimize(f, genome_length=10, bounds=(-5, 5), seed=42)
islands=1: fitness identical: True, genome identical: True
islands=4: fitness identical: True, genome identical: True
islands=8: fitness identical: True, genome identical: True
Identical to twelve decimal places, on the genome and not just the fitness. Different seeds give different answers, five for five, and unseeded runs give four distinct results out of four, which is the other half of what a seed parameter has to mean.
The discipline part is that every source has to descend from the seed. Miss one, and runs mostly reproduce, which is worse than obviously not reproducing because the failures are rare and look like something else.
Then you go parallel
Now eight islands are evolving at once, each making its own random draws. There is one seed and eight workers, and the seed has to determine everything.
There are three ways to arrange that, and two of them are wrong.
Share one generator across the islands. Every draw now touches shared mutable state from multiple threads, so you need a lock, and the whole point of islands was to run them concurrently. Take the lock away and the order of draws depends on thread scheduling, so the run is no longer a function of the seed. It is a function of the seed and whatever the operating system did that afternoon.
That is the lock-or-a-lie choice, and neither branch is acceptable: one gives up the parallelism, the other gives up the reproducibility.
Give every island the same seed. Reproducible, cheap, and useless. Eight islands running identical searches is one search with an eight-times electricity bill.
Derive each island’s seed from the master seed, structurally. Island i gets a seed that is a deterministic function of the master seed and i. No shared state, no locking, no scheduling dependence, and eight genuinely different searches.
Checking which one you have
The test is short. Turn migration off so the islands cannot influence each other, run one seeded island model, and look at what each island found:
m = IslandModel(..., num_islands=6, migration_interval=1000, seed=42)
res = m.run()
res.island_best_fitness()
per-island bests: [-4.745305, -7.245344, -13.157005, -10.953963, -15.940581, -8.765622]
distinct values: 6 of 6
Six distinct results from one master seed. If the islands shared a seed, those six numbers would be identical and the parallelism would be decorative. If they shared a generator without a lock, re-running would produce different numbers.
Both failure modes are visible in one assertion, and neither shows up in a normal benchmark, because a benchmark reports the best island and the best of eight clones is still a valid answer. It is just eight times more expensive than it needed to be.
Why it is an architecture problem
The reason this cannot be bolted on afterwards is that “derive child seeds structurally” is a statement about the shape of the whole program.
Every component that makes a random choice has to accept a generator rather than reach for a global one. That is a signature change on selection, crossover, mutation, restart, immigrants and local search, and any one of them that calls a module-level random() silently reintroduces shared state. In Rust the compiler makes that hard to get wrong, since sharing a mutable generator across threads does not typecheck without explicit synchronization. In Python it is entirely possible to write the parallel version, have it appear to work, and never learn that your seed stopped meaning anything.
The general rule is worth stating without the GA context: reproducibility is a property of how state is arranged, not a feature you add. If randomness is reachable from anywhere, no amount of seeding at the entry point will fix it, and the resulting program will reproduce often enough to look correct.
What it buys
The obvious thing is debugging: a run that produced a strange answer can be re-run and inspected.
The less obvious one is that it makes comparison possible at all. Every measurement in the island model post is a median over seeds, comparing configurations on the same seeds. That is only meaningful because seed 3 means the same starting population in every configuration, so the difference between two columns is the configuration rather than the luck. Without per-run reproducibility you can still benchmark, but you need far more runs to see the same effect, because you are measuring through noise you could have eliminated.
Trying it
import parga
a = parga.minimize(f, genome_length=10, bounds=(-5, 5), islands=8, seed=42)
b = parga.minimize(f, genome_length=10, bounds=(-5, 5), islands=8, seed=42)
assert a.best_fitness == b.best_fitness
pip install parga.
Stay in the loop
Get notified when I publish new posts. No spam, unsubscribe anytime.