Ant
Content
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
Swarm Intelligence,
Ant Algorithms and
Ant Colony Optimization
Marco Dorigo
IRIDIA
Université Libre de Bruxelles
Belgio
[email protected]
1
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
2
Ants
Leaf cutter,
fungus growing ants
Breeding ants
Weaver ants
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
3
Ants
• Fungus growers
• Breeding ants
• Weaver ants
• Harvesting ants
• Army ants
• Slavemaker ants
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
4
Insects, Social Insects, and Ants
• 10
18
living insects (rough estimate)
• ~2% of all insects are social
• Social insects are:
– All ants
– All termites
– Some bees
– Some wasps
• 50% of all social insects are ants
• Avg weight of one ant between 1 and 5 mg
• Tot weight ants ~ Tot weight humans
• Ants have colonized Earth for 100 million years,
Homo sapiens sapiens for 50,000 years
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
5
Ant Colony Societies
• Ant colony size: from as few as 30
to millions of workers
• Work division:
– Reproduction → queen
– Defense → soldiers
– Food collection → specialized workers
– Brood care → specialized workers
– Nest brooming → specialized workers
– Nest building & maintenance → specialized
workers
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
6
How Do Ants and Social Insects
Coordinate their Activities?
• Self-organization:
– Set of dynamical mechanisms whereby
structure appears at the global level as the result
of interactions among lower-level components
– The rules specifying the interactions among the
system's constituent units are executed on the basis
of purely local information,
without reference to the global pattern,
which is an emergent property of the system
rather than a property imposed upon the system by
an external ordering influence
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
7
Self-organization
Four basic ingredients:
• Randomness
• Positive feedback
– E.g., recruitment and reinforcement
• Negative feedback
– E.g., limited number of available foragers
• Multiple interactions
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
8
Characteristics of a
Self-organized System
• Creation of spatio-temporal structures
– E.g., foraging trails, nest architectures, social organization
• Multistability
(i.e., possible coexistence of several stable states)
– E.g., ants exploit only one of two identical food sources
• Existence of bifurcations when some parameters
change
– E.g., termites move from a non-coordinated to a coordinated
phase only if their density is higher than a threshold value
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
9
How Do Social Insects Achieve
Self-organization?
• Communication is necessary
• Two types of communication:
– Direct: antennation, trophallaxis (food or liquid exchange),
mandibular contact, visual contact, chemical contact, etc.
– Indirect: two individuals interact indirectly when one of
them modifies the environment and the other responds to
the new environment at a later time
This is called stigmergy
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
10
Stigmergy
Grassé P. P., 1959
• “La coordination des taches, la regulation des constructions
ne dependent pas directement des oeuvriers, mais des
constructions elles-memes. L’ouvrier ne dirige pas son
travail, il est guidé par lui. C’est à cette stimulation d’un type
particulier que nous donnons le nom du STIGMERGIE
(stigma, piqure; ergon, travail, oeuvre = oeuvre stimulante).”
Grassé P. P., 1959
• [“The coordination of tasks and the regulation of constructions
does not depend directly on the workers, but on the
constructions themselves. The worker does not direct his
work, but is guided by it. It is to this special form of
stimulation that we give the name STIGMERGY (stigma,
sting; ergon, work, product of labour = stimulating product of
labour).”]
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
11
Stigmergy
Stimulation of workers
by the performance
they have achieved
Grassé P. P., 1959
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
12
Stigmergy
Example: Termites Building their Nest
1 h 18’
after start
3 h 23’
after start
5 h 15’
after start
8 h 13’
after start
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
13
Termites’ Nest
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
14
Sign-based Stigmergy
Example: Trail Following and
Ants Foraging Behavior
While walking, ants and termites:
• May deposit a pheromone on the ground
• Follow with high probability pheromone
trails they sense on the ground
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
15
Sign-based Stigmergy
Example: Trail Following and Ants
Foraging Behavior
Movie by Marco Dorigo
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
16
Pheromone Trail Following
Ants and termites follow pheromone trails
Movie by Jean-Louis Deneubourg
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
17
Ants Foraging Behavior
Example: The Double Bridge Experiment
15 cm
Nest Food
A
B
0
25
50
75
100
0 5 10 15 20 25 30
Time (minutes)
% Passages A % Passages B
Simple bridge % of ant passages on
the two branches
Goss et al., 1989, Deneubourg et al., 1990
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
18
Double Bridge Experiment
Goss et al., 1989, Deneubourg et al., 1990
Movie by Jean-Louis Deneubourg
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
19
Model for
Symmetric Bridge Experiment
Deneubourg et al., 1990
number of ant passages
%
o
f
p
a
s
s
a
g
e
s
o
n
w
i
n
n
i
n
g
b
r
a
n
c
h
100
80
60
40
20
0
1000 1500 2000 500
Let A
i
= number of ants which chose A
after i ants have made a choice
A
i
+ B
i
= i
δ is randomly uniform in [0,1]
P
c A
c A c B
P
A
i
i i
B
=
+ ( )
+ ( ) + + ( )
= −
2
2 2
1
A
A P
A P
i
i A
i A
+
=
+ ≤
>
1
1 if
if
δ
δ
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
20
Quantitative vs. Qualitative Stigmergy
Termites Wasps
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
21
Wasps
Building a Nest
Movies by Guy Theraulaz
a)
b)
(c)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
22
Wasps’ Nests
Photos by Guy Theraulaz
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
23
Nest Building Simulations (1)
Theraulaz & Bonabeau, 1995
z
z+1
z-1
Sensory field
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
24
Nest Building Simulations (2)
Theraulaz & Bonabeau, 1995
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
25
Nest Building Simulations (3)
Theraulaz & Bonabeau, 1995
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
26
More Simulations Results
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
27
Types of Stigmergy
• Sematectonics
E.g., termites nest building
• Sign-based
E.g., ants trail following behavior
• Quantitative
E.g., ants trail following behavior and
termites nest building
• Qualitative
E.g., social wasps nest building
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
28
“Artificial” Stigmergy
Indirect communication mediated by
modifications of environmental states
which are only locally accessible by
the communicating agents
Dorigo & Di Caro, 1999
• Characteristics of artificial stigmergy:
– Indirect communication
– Local accessibility
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
29
What Are Ant Algorithms?
• Ant algorithms are multi-agent systems
that exploit artificial stigmergy as
a means for coordinating artificial ants
for the solution of computational problems
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
30
Goals of the Presentation
• To present a number of distributed algorithms
(ant algorithms) inspired by social insects
behavior and by the concept of stigmergy
• To highlight the role plaid by the so-called
stigmergic variable(s)
• To illustrate the practical interest of
ant algorithms via some application examples
(both toy examples and real applications)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
31
Structure of the Presentation
Ant behavior
⇓
Model
⇓
Derived application
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
32
Real Ants Inspire Ant Algorithms
• Foraging →
• Division
→
Adaptive task allocation
of labor
• Cemetery organization and brood sorting
→ Robot clustering
→ Graph partitioning
• Cooperative transport
→ Robotic implementations
Shortest path
ACO: Network routing
Combinatorial optimization
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
1
Real Ants Inspire Ant Algorithms
• Foraging →
• Division
→
Adaptive task allocation
of labor
• Cemetery organization and brood sorting
→ Robot clustering
→ Graph partitioning
• Cooperative transport
→ Robotic implementations
Shortest path
ACO: Network routing
Combinatorial optimization
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
2
Asymmetric Bridge Experiment
Goss et al., 1989
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
3
Asymmetric Bridge Experiment
Goss et al., 1989 Dorigo & Bertolissi, 1998
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
4
Some Results
r is the length ratio among
the two bridges
0
5 0
100
0- 20 20- 40 40- 60 60- 80 80- 100
% of traffic on the short branch
0
5 0
100
0- 20 20- 40 40- 60 60- 80 80- 100
% of traffic on the short branch
r = 1
0
5 0
100
0- 20 20- 40 40- 60 60- 80 80- 100
% of traffic on the short branch
r = 2
r = 2
Short branch added later
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
5
Artifical Ants and
the Shortest Path Problem
?
Probabilistic rule to
choose the path
Pheromone trail
depositing
Source
Destination
Memory
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
6
Problem!
The extension of the real ant behavior
(forward/backward trail deposit)
to artificial ants moving on a graph doesn’t work:
problem of self-reinforcing loops
Probabilistic solution generation plus pheromone update
-> self-reinforcing loops
Source
Destination
Example of possible
self-reinforcing loop
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
7
Our Design Choices
• Ants are given a memory of visited nodes
• Ants build solutions probabilistically
without updating pheromone trails
• Ants deterministically backward retrace
the forward path to update pheromone
• Ants deposit a quantity of pheromone
function of the quality of the solution
they generated
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
8
Using Pheromone and Memory to
Choose the Next Node
Memory of
visited nodes
i
τ
ijd
τ
ird
j
k
τ
ikd
r
p t f t
ijd
k
ijd
( ) = ( )
( )
τ
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
9
Ants’ Probabilistic Transition Rule
• τ
ijd
is the amount of pheromone trail on edge (i,j,d)
• J
i
k
is the set of feasible nodes ant k positioned on
node i can move to
p t
t
t
ijd
k
ijd
ihd
h J
i
k
( ) =
( )
[ ]
( )
[ ]
∈
∑
τ
τ
α
α
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
10
Ants’ Pheromone Trail Depositing
where the (i,j)’s are the links visited
by ant k, and
where quality
k
is set proportional to
the inverse of the time it took ant k
to build the path from i to d via j
i
τ
ijd
Source
Destination
d
j
τ ρ τ τ
ijd
k
ijd
k
ijd
k
t t t + ( ) ← −
( )
⋅ ( ) + ( ) 1 1 ∆
∆τ
ijd
k k
t quality ( ) =
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
11
Using Pheromones and Heuristic
to Choose the Next Node
• τ
ijd
is a value stored in a pheromone table
• η
ijd
is an heuristic evaluation of link (i,j,d)
which introduces problem specific
information
i
τ η
ijd ij
;
ant’s destination = d
τ
ijd
;η
ijd
τ
ird
;η
ird
j
k
τ
ikd
;η
ikd
r
p t f t t
ijd
k
ijd ijd
( ) = ( ) ( )
( )
τ η ,
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
12
The Simple Ant Colony Optimization
Algorithm
• Ants are launched at regular instants
from each node to randomly chosen destinations
• Ants are routed probabilistically with a probability
function of:
(i) some artificial pheromone values, and
(ii) some heuristic values,
maintained on the nodes
• Ants memorize visited nodes and costs incurred
• Once reached their destination nodes,
ants retrace their paths backwards, and
update the routing tables
The pheromone trail is the stigmergic variable
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
13
Why Does it Works?
Three important components:
• TIME: a shorter path
receives pheromone quicker
(this is often called:
“differential length effect”)
• QUALITY: a shorter path
receives more pheromone
• COMBINATORICS: a shorter path
receives pheromone more frequently
because it is likely to have a lower
number of decision points
i
τ
ijd
Source
Destination
d
j
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
14
How Does it Work?
• It works very well on
– shortest path problems with dynamic costs
(e.g., routing in telecommunications networks)
– constrained shortest path problems
(e.g., NP-hard problems)
• The second part of the tutorial is dedicated
to Ant Colony Optimization (ACO) and
to its applications to the above-mentioned
classes of problems
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
15
Real Ants Inspire Ant Algorithms
• Foraging →
• Division
→
Adaptive task allocation
of labor
• Cemetery organization and brood sorting
→ Robot clustering
→ Graph partitioning
• Cooperative transport
→ Robotic implementations
Shortest path
ACO:
Combinatorial optimization
Network routing
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
16
The Routing Problem
• The practical goal of routing algorithms is to build
routing tables
• Routing is difficult because costs are dynamic
• Adaptive routing is difficult because changes in the
control policy determine changes in the costs and
vice versa
Destination
node
j
Routing table of node k
Next node i
j
...
...
...
...
1
i
1
N
i
N
k-1
i
k-1
...
...
k+1
i
k+1
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
17
AntNet Applied to
Routing in Internet-like Networks
?
Probabilistic rule to
choose the path
Pheromone trail
depositing
Di Caro and Dorigo, 1997
Source
Destination
Memory
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
18
AntNet: Main Characteristics
• Ants collect and distribute information used to
modify routing tables
• Ants find their way to node d using routing tables
in a probabilistic way
• Data packets use routing tables in a probabilistic way
• Ants communicate through the network by stigmergy
• AntNet is a multiagent approach to adaptive learning
of routing tables
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
19
AntNet: Data Structures
• Routing table:
similar to that of adaptive distance-vector routing,
but with values normalized to 1
• Trips vector:
contains statistics about
ants’ trip times from current node k
to each destination node d
(means and variances)
Trips vector of node k
... ...
d
Routing table of node k
Destination nodes
P(k,n,d)
n
σ
2
(k,1) σ
2
(k,d) σ
2
(k,N)
µ(k,d) µ(k,N) µ(k,1)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
20
AntNet: The Algorithm
• Ants (F-ants) are launched at regular instants
from each node to randomly chosen destinations
• Ants are routed probabilistically with a probability
function of:
(i) some artificial pheromone values, and
(ii) some heuristic values,
maintained on the nodes
• Ants memorize visited nodes and elapsed times
• Once reached their destination nodes,
ants retrace their paths backwards (B-ants), and
update the routing tables
AntNet is distributed and not synchronized
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
21
AntNet: the Role of F-ants and of B-ants
• F-ants collect implicit and explicit information on
available paths and traffic load
– implicit information, through the arrival rate at their
destinations
– explicit information, by storing experienced trip times
• F-ants share queues with data packet
• B-ants fast backpropagate info collected by
F-ants to visited nodes
• B-ants use higher priority queues
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
22
AntNet: Routing Tables Updating
− P(k,n,d)
is the probability with which an
ant moves from node k to neighbor
node n if its destination node is d
− Sum
n
P(k,n,d)
=1
− Routing decision: P(k,n,d)
− Routing tables are updated by B-ants:
If B-ant has source node d and goes
from node n to k then
P(k,n,d)
:= P(k,n,d)
+ R · (1 - P(k,n,d))
P(k,i,d)
:= P(k,i,d)
- R · P(k,i,d)
∀ i ≠ n
− R, 0<R≤1, is a reinforcement,
function of F-ant trip time from k to d,
information carried by B-ant
k n
d
F-ant
B-ant
d
Routing table of node k
Destination nodes
P(k,n,d)
n
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
23
AntNet: Reinforcement Computation
R is a normalized value (0<R≤1) function of:
– T : time experienced by the F-ant
− µ : avg time for the same destination
memorized in the Trips table
− σ : std. dev. for the same destination
memorized in the Trips table
−
e.g. :
where is a sigmoid between 0 and 1
R f
T
f
= −
+
1
µ σ
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
24
Ants’ Pheromone Trail Depositing
where the (i,j)’s are the links visited
by ant k, and
where quality
k
is set proportional to
the inverse of the time it took ant k
to build the path from i to d via j
i
τ
ijd
Source
Destination
d
j
τ ρ τ τ
ijd
k
ijd
k
ijd
k
t t t + ( ) ← −
( )
⋅ ( ) + ( ) 1 1 ∆
∆τ
ijd
k k
t quality ( ) =
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
25
Using Pheromones and Heuristic
to Choose the Next Node
• τ
ijd
is the pheromone trail
• η
ij
is an heuristic evaluation of link (i,j)
which introduces problem specific
information (e.g., in AntNet η
ij
is
∝
to
the inverse of link (i,j) queue length)
i
τ η
ijd ij
;
ant’s destination = d
τ
ijd
;η
ij
τ
ird
;η
ir
j
k
τ
ikd
;η
ik
r
p t f t t
ijd
k
ijd ij
( ) = ( ) ( )
( )
τ η ,
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
26
AntNet’s Decision Rule
ˆ
, ,
, ,
P k n d
P k n d l
N
n
k
( ) =
( ) + ⋅
+ −
( )
α
α 1 1
where l
n
is a heuristic correction normalized in [0,1] and
proportional to the length q
n
(in bits waiting to be sent) of
the queue of the link connecting node k with its neighbor n:
l
q
q
n
n
k
k
N
k
= −
=
∑
1
1
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
27
AntNet: Experimental Setup
• Realistic simulator (though not industrial)
• Many topologies
• Many traffic patterns
• Comparison with many state-of-the-art algorithms
(OSPF, SPF, Adaptive Bellman-Ford, Q-routing, PQ-routing)
• Performance measures:
throughput (bit/sec) measures the quantity of service, and
average packet delay (sec) measures the quality of service
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
28
Experimental Setup:
Network Topologies
1
10
8
7
6
5
4
3
11
12
13
14
2
9
7
13
7
20
7
7
7
7
8
9
4
1
5
6
5
5
8
7
NSFNET
(14 nodes)
NTTnet
(57 nodes)
6x6 grid net
(36 nodes)
1
8
7
2
3 4
6
5
1
1
1
1
1
1
1
1
1
SIMPLEnet
(8 nodes)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
29
Experimental Setup:
Transmission Protocol
• Datagram traffic (UDP-like protocol)
• No arrival acknowledgment
• No error notification
• No admission or congestion control
• Packets are discarded if buffer is full
or time to live (TTL) is expired
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
30
Experimental Setup:
Traffic Characteristics
Traffic patterns are obtained by the combination
of spatial and temporal distributions for sessions
• Spatial distributions
– Uniform (U)
– Random (R)
– Hot Spots (HS)
• Temporal distributions
– Poisson (P)
– Fix (F)
– Temporary (TMPHS)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
31
Experimental Setup:
Traffic Characteristics
• Inter-arrival times and sizes for sessions
have negative exponential distribution
• Inter-arrival times and sizes for packets in a
session have negative exponential distribution
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
32
Experimental Setup:
Network Load
• Heavy load near saturation
(1000 sec simulation)
• Heavy load plus transient saturation
(1000 sec simulation)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
33
Experimental Setup:
Experiments Design
• Experiment duration:
– Each experiment, lasting 1000 sec, is repeated 10 times
– Before feeding data, routing tables are initialized by a 500 sec
phase
• Experiment typology:
– Study of traffic distribution on different possible paths
– Study of algorithms behavior for increasing network load
– Study of algorithms behavior for transient saturation
– Evaluation of influence of control packet traffic on total
traffic
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
34
Competing Algorithms
AntNet was compared with:
– OSPF
– SPF
– Adaptive BF
– Q-routing (asynchronous on-line BF)
– PQ-R
– Daemon: approximation of an ideal algorithm
It knows at each instant the status of all
queues and applies shortest path at each
packet hop
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
35
Measures of Performance
Good routing:
– Under high load: increase throughput for same
average delay
– Under low load: decrease avg delay per packet
Measures of performance are
• Throughput (bits/sec): quantity of service
• Average delay (sec): quality of service
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
36
How to Read Results
• Routing is a multi-objective problem
• Max throughput is the main criterion:
non max throughput means
– retransmissions,
– error notification
– augmented congestion
• Average packet delay has an inherent
very high variance
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
37
SimpleNet
(F-CBR traffic)
9.5
10.0
10.5
11.0
11.5
12.0
12.5
13.0
13.5
14.0
0 100 200 300 400 500 600 700 800 900 1000
T
h
r
o
u
g
h
p
u
t
(
1
0
6
b
it
/
s
e
c
)
Simulation Time (sec)
OSPF
SPF
BF
Q-R
PQ-R
AntNet
Daemon
0.0
0.2
0.4
0.6
0.8
1.0
0 0.05 0.1 0.15 0.2
E
m
p
ir
ic
a
l
D
is
t
r
ib
u
t
io
n
Packet Delay (sec)
OSPF
SPF
BF
Q-R
PQ-R
AntNet
Daemon
Comparison of algorithms: traffic directed from node 1 to node 6
From Di Caro and Dorigo, 1998
Throughput (b/s) Packet delay
1
8
7
2
3 4
6
5
1
1
1
1
1
1
1
1
1
SIMPLEnet
(8 nodes)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
38
NSFNET & NTTnet
(increasing UP traffic)
0
5
10
15
20
25
30
35
40
45
AntNet OSPF SPF BF Q-R PQ-R Daemon
T
h
r
o
u
g
h
p
u
t
(
1
0
6
b
i
t
/
s
e
c
)
3.1 3 2.9 2.8 2.7
0. 0
1. 0
2. 0
3. 0
4. 0
5. 0
6. 0
7. 0
8. 0
9. 0
10. 0
AntNet OSPF SPF BF Q-R PQ-R Daemon
9
0
-
t
h
p
e
r
c
e
n
t
i
l
e
o
f
p
a
c
k
e
t
d
e
l
a
y
s
(
s
e
c
)
3.1 3 2.9 2.8 2.7
Increasing UP traffic
UP traffic increased by reducing the mean session inter arrival time
0
2
4
6
8
10
12
14
16
18
AntNet OSPF SPF BF Q-R PQ-R Daemon
T
h
r
o
u
g
h
p
u
t
(
1
0
6
b
i
t
/
s
e
c
)
2.4 2.3 2.2 2.1 2
0. 0
0. 5
1. 0
1. 5
2. 0
2. 5
3. 0
3. 5
4. 0
4. 5
AntNet OSPF SPF BF Q-R PQ-R Daemon
9
0
-
t
h
p
e
r
c
e
n
t
i
l
e
o
f
p
a
c
k
e
t
d
e
l
a
y
s
(
s
e
c
)
2.4 2.3 2.2 2.1 2
T
h
r
o
u
g
h
p
u
t
(
b
/
s
)
A
v
g
p
a
c
k
e
t
d
e
l
a
y
From Di Caro and Dorigo, 1998,
Journal of Artificial Intelligence Research
NTT net NSF net
10
9
8
7
6
4
3
11
12
13
14
9
7
13
7
20
7
7
7 4
8
9
14 11
15
16
5
5
8
7
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
39
NSFNET & NTTnet
(increasing UP-HS traffic)
From Di Caro and Dorigo, 1998
0
2
4
6
8
10
12
14
16
18
AntNet OSPF SPF BF Q-R PQ-R Daemon
T
h
r
o
u
g
h
p
u
t
(
1
0
6
b
it
/
s
e
c
)
2.4 2.3 2.2 2.1 2
0. 0
0. 1
0. 2
0. 3
0. 4
0. 5
AntNet OSPF SPF BF Q-R PQ-R Daemon
9
0
-
t
h
p
e
r
c
e
n
t
ile
o
f
p
a
c
k
e
t
d
e
la
y
s
(
s
e
c
)
2.4 2.3 2.2 2.1 2
0
5
10
15
20
25
30
35
40
45
50
AntNet OSPF SPF BF Q-R PQ-R Daemon
T
h
r
o
u
g
h
p
u
t
(
1
0
6
b
it
/
s
e
c
)
4.1 4 3.9 3.8 3.7
0. 0
1. 0
2. 0
3. 0
4. 0
5. 0
6. 0
7. 0
AntNet OSPF SPF BF Q-R PQ-R Daemon
9
0
-
t
h
p
e
r
c
e
n
t
ile
o
f
p
a
c
k
e
t
d
e
la
y
s
(
s
e
c
)
4.1 4 3.9 3.8 3.7
T
h
r
o
u
g
h
p
u
t
(
b
/
s
)
A
v
g
p
a
c
k
e
t
d
e
l
a
y
From Di Caro and Dorigo, 1998,
Journal of Artificial Intelligence Research
NTT net NSF net
10
9
8
7
6
4
3
11
12
13
14
9
7
13
7
20
7
7
8
9
14 11
15
16
5
5
8
7
Increasing UP-HS traffic
UP traffic increased by reducing the mean session inter arrival time
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
40
NSFNET & NTTnet
(increasing RP traffic)
0
2
4
6
8
10
12
AntNet OSPF SPF BF Q-R PQ-R Daemon
T
h
r
o
u
g
h
p
u
t
(
1
0
6
b
it
/
s
e
c
)
2.8 2.7 2.6 2.5 2.4
0. 0
0. 5
1. 0
1. 5
2. 0
2. 5
3. 0
3. 5
4. 0
AntNet OSPF SPF BF Q-R PQ-R Daemon
9
0
-
t
h
p
e
r
c
e
n
t
ile
o
f
p
a
c
k
e
t
d
e
la
y
s
(
s
e
c
)
2.8 2.7 2.6 2.5 2.4
0
5
10
15
20
25
30
35
40
45
AntNet OSPF SPF BF Q-R PQ-R Daemon
T
h
r
o
u
g
h
p
u
t
(
1
0
6
b
it
/
s
e
c
)
3.1 3 2.9 2.8 2.7
0. 0
1. 0
2. 0
3. 0
4. 0
5. 0
6. 0
7. 0
8. 0
9. 0
10. 0
AntNet OSPF SPF BF Q-R PQ-R Daemon
9
0
-
t
h
p
e
r
c
e
n
t
ile
o
f
p
a
c
k
e
t
d
e
la
y
s
(
s
e
c
)
3.1 3 2.9 2.8 2.7
T
h
r
o
u
g
h
p
u
t
(
b
/
s
)
A
v
g
p
a
c
k
e
t
d
e
l
a
y
Increasing RP traffic
RP traffic increased by reducing the mean session inter arrival time
From Di Caro and Dorigo, 1998,
Journal of Artificial Intelligence Research
NTT net NSF net
10
9
8
7
6
4
3
11
12
13
14
9
7
13
7
20
7
7
8
9
14 11
15
16
5
5
8
7
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
41
NSFNET & NTTnet (UP plus transient HS)
From Di Caro and Dorigo, 1998
6.0
8.0
10.0
12.0
14.0
16.0
T
h
r
o
u
g
h
p
u
t
(
1
0
6
b
it
/
s
e
c
)
OSPF
SPF
BF
Q-R
PQ-R
AntNet
Daemon
30.0
40.0
50.0
60.0
200 300 400 500 600 700 800 900 1000
P
a
c
k
e
t
D
e
la
y
(
s
e
c
)
Simulation Time (sec)
Data averaged over a 5 seconds sliding window
15.0
25.0
35.0
45.0
55.0
T
h
r
o
u
g
h
p
u
t
(
1
0
6
b
it
/
s
e
c
)
OSPF
SPF
BF
Q-R
PQ-R
AntNet
Daemon
0.0
0.2
0.4
0.6
0.8
200 300 400 500 600 700 800 900 1000
P
a
c
k
e
t
D
e
la
y
(
s
e
c
)
Simulation Time (sec)
NTT net NSF net
10
9
8
7
6
4
3
11
12
13
14
9
7
13
7
20
7
7
8
9
14 11
15
16
5
5
8
7
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
42
Routing Overhead
From Di Caro and Dorigo, 1998
AntNet OSPF SPF BF Q-R PQ-R Daemon
SimpleNet 0.33 0.01 0.10 0.07 1.49 2.01 0.00
NSFNET-UP 2.39 0.15 0.86 1.17 6.96 9.93 0.00
NSFNET-RP 2.60 0.16 1.07 1.17 5.26 7.74 0.00
NSFNET- UP-HS 1.63 0.15 1.14 1.17 7.66 8.46 0.00
NTTnet-UP 2.85 0.14 3.68 1.39 3.72 6.77 0.00
NTTnet- UP-HS 3.81 0.15 4.56 1.39 3.09 4.81 0.00
Ratio (10
-3
) between bandwidth occupied by the routing
packets and the total available network bandwidth
From Di Caro and Dorigo, 1998,
Journal of Artificial Intelligence Research
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
43
AntNet Experiments: Summary
• Under low load all tested algorithms have
similar performance
• Under high-load (near saturation)
AntNet is the best algorithm
• Under a sudden variation in traffic load
AntNet remains the best algorithm
both in terms of throughput and of delay
• AntNet’s overhead is negligible
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
44
Implications for Applications
• AntNet and other similar ACO algorithms
seem to be particularly suited for problems with:
– distributed information
– non-stationary stochastic dynamics
– asynchronous evolution of the network status
• This is due to ACO algorithms’ use of:
– local information to generate solutions
– indirect communication via the pheromone trails
– stochastic state transitions
• There are other network applications to which
people are working:
– Quality of Service routing
– ATM applications
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
45
Real Ants Inspire Ant Algorithms
• Foraging →
• Division
→
Adaptive task allocation
of labor
• Cemetery organization and brood sorting
→ Robot clustering
→ Graph partitioning
• Cooperative transport
→ Robotic implementations
Shortest path
ACO: Network routing
Combinatorial optimization
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
46
Ant System Applied to the TSP
Pheromone trail
?
Probabilistic rule to
choose the path
depositing
Memory
Dorigo, Maniezzo, Colorni, 1991
Dorigo & Gambardella, 1996
Ant System is the ancestor of all
Ant Colony Optimization algorithms
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
47
Using Pheromone and Memory to
Choose the Next City
Memory of
visited cities
i
τ
ij
τ
ir
j
k
τ
ik
r
p t f t
ij
k
ij
( ) = ( )
( )
τ
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
48
Ants’ Probabilistic Transition Rule
• τ
ij
is the amount of pheromone trail on edge (i,j)
• J
i
k
is the set of cities ant k positioned in city i
still has to visit
p t
t
t
ij
k
ij
ih
h J
i
k
( ) =
( )
[ ]
( )
[ ]
∈
∑
τ
τ
α
α
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
49
Using Pheromone, Heuristic, and
Memory to Choose the Next City
Memory of
visited cities
i
τ η
ijd ij
;
τ
ij
;η
ij
τ
ir
;η
ir
j
k
τ
ik
;η
ik
r
p t f t t
ij
k
ij ij
( ) = ( ) ( )
( )
τ η ,
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
50
Probabilistic Transition Rule
• η
ij
is an heuristic evaluation of edge (i,j) which
introduces problem specific information
• in the TSP the typical choice is to set η
ij
to the
inverse of the edge length
p t
t t
t t
ij
k
ij ij
ih ih
h J
i
k
( ) =
( )
[ ]
⋅ ( )
[ ]
( )
[ ]
⋅ ( )
[ ]
∈
∑
τ η
τ η
α β
α β
p t
ij
k
( ) can be seen as the overall desirability for
ant k located in city i to choose to move to city j
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
51
Ants’ Pheromone Trail Depositing
where quality
k
is set proportional to the inverse of
the length of the solution found by ant k
τ ρ τ τ
ij ij ij
t t t + ( ) ← − ( ) ⋅ ( ) + ( ) 1 1 ∆
∆ ∆ τ τ
ij ij
k
t t ( ) ( )
∑
=
k=1
n
After all ants have built a tour, pheromone trails
are updated on all edges (i,j) as follows:
where
∆τ
ij
k k
t quality ( ) = and
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
52
The Ant System Algorithm
Loop
Place one ant on each city \*there are n cities \*
For step := 1 to n \* each ant builds a tour \*
For k := 1 to n \* each ant adds a city to its path \*
Choose the next city to move to applying a
probabilistic state transition rule
End-for
End-for
Update pheromone trails
Until End_condition
Dorigo, Maniezzo, Colorni, 1991
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
53
Ant System (AS): Some Results
1
2
3
4
5
6
7
8
9
10
1
2
3
4
5
6
7
8
9
10
0
10
20
30
40
50
60
70
80
Cycles
Tour length
standard deviation
0 1000 500 1500 2000 2500 3000
300
400
500
600
Cycles
0 1000 500 1500 2000 2500 3000
Best tour length
Tour length std deviation Evolution of trail distribution
Best tour length
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
54
Extensions of AS
• Ant Colony System (ACS)
(Gambardella & Dorigo, 1996; Dorigo & Gambardella, 1997)
• ACS-3-opt
(Gambardella & Dorigo, 1996; Dorigo & Gambardella, 1997)
• Max-Min AS (MMAS)
(Stützle & Hoos, 1997)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
55
Constructive Heuristic
and Local Search
Current wisdom says that a very good strategy for the
approximate solution of
combinatorial optimization problems is the coupling of:
–a constructive heuristic, and
–a local search
The problem is to find good couplings:
Ant Colony Optimization algorithms seem
(as shown by experimental evidence)
to provide such a good coupling
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
56
Example:
The Sequential Ordering Problem
• Find a minimum weight Hamiltonian path on a directed graph
with weights on arcs, subject to multiple precedence constraints
among nodes
• Very similar to an ATSP in which a solution is a path which visits
all cities once and connects an origin to a destination node
without closing the tour
• The SOP models real-world problems like single-vehicle routing
problems with pick-up and delivery constraints, production
planning, and transportation problems in flexible manufacturing
systems
Start
End
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
57
The Sequential Ordering Problem
Find the minimal sequence from node Start to node End that
visits all the nodes and do not violate precedence constraints
Start
End
i
s
k
j
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
58
The HAS-SOP Algorithm
Loop
Position m ants on the first city
For step=1 to n
For k=1 to m
Apply the state transition rule
End-for
End-for
Apply local search
Apply the trail updating rule
Until End_condition
(Gambardella & Dorigo, 1997)
Toappear in:
INFORMS Journal on Computing
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
59
HAS-SOP:
Results on a Set of 23 TSPLIB Test Problems
HAS-SOP results:
Red: In 14 problems
out of 23 it found a
result which impro-
ved the TSPLIB
upper bound
Blue: In 6 problem
it found the known
optimal solution
Green: In 3 problems
it found the same
value as the TSPLIB
upper bound
PROB
n |R| TSPLIB
Bounds
Best
Result
Avg
Result
Std.Dev.
Avg
Time
(sec)
ESC63.sop
65 95 62
62 62.0 0 0.1
ESC78.sop
80 77 18230
18230 18230.0 0 6.9
ft53.1.sop
54 12 [7438,7570]
7531 7531.0 0 9.9
ft53.2.sop
54 25 [7630,8335]
8026 8026.0 0 18.4
ft53.3.sop
54 48 [9473,10935]
10262 10262.0 0 2.9
ft53.4.sop
54 63 14425
14425 14425.0 0 0.4
ft70.1.sop
71 17 39313
39313 39313.0 0 29.8
ft70.2.sop
71 35 [39739,40422]
40419 40433.5 24.6 114.1
ft70.3.sop
71 68 [41305,42535]
42535 42535.0 0 64.4
ft70.4.sop
71 86 [52269,53562]
53530 53566.5 7.6 38.2
kro124p.1.sop
101 25 [37722,40186]
39420 39420.0 0 115.2
kro124p.2.sop
101 49 [38534,41677]
41336 41336.0 0 119.3
kro124p.3.sop
101 97 [40967,50876]
49499 49648.8 249.7 262.8
kro124p.4.sop
101 131 [64858,76103]
76103 76103.0 0 57.4
prob.100.sop
100 41 [1024,1385]
1226 1302.4 39.4 1918.7
rbg109a.sop
111 622 1038
1038 1038.0 0 14.6
rbg150a.sop
152 952 [1748,1750]
1750 1750.0 0 159.1
rbg174a.sop
176 1113 2033
2033 2034.7 1.4 99.3
rbg253a.sop
255 1721 [2928,2987]
2950 2950.0 0 81.5
rbg323a.sop
325 2412 [3136,3157]
3141 3146.0 1.4 1685.5
rbg341a.sop
343 2542 [2543,2597]
2580 2591.9 11.8 2149.6
rbg358a.sop
360 3239 [2518,2599]
2555 2561.2 5.2 2169.3
rbg378a.sop
380 3069 [2761,2833]
2817 2834.3 10.7 2640.3
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
60
The ACO Metaheuristic
• Ant System and AntNet have been extended so
that they can be applied to any
shortest path (minimum cost) problem on graphs
• The resulting extension is called
Ant Colony Optimization metaheuristic
• Currently two major application classes:
– Routing in telecommunications networks
– NP-hard combinatorial optimization problems
Dorigo, Di Caro & Gambardella, 1999
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
61
The ACO- Metaheuristic Procedure
procedure ACO-metaheuristic()
while (not-termination-criterion)
schedule subprocedures
generate-&-manage-ants()
evaporate-pheromone()
execute-daemon-actions() {Optional}
end schedule subprocedures
end while
end procedure
These are problem specific actions,
like local search
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
62
ACO: Quality of Results Obtained
SEQUENTIAL ORDERING PROBLEM (SOP)
Best heuristic currently available Gambardella-Dorigo
QUADRATIC ASSIGNMENT PROBLEM (QAP)
Among best heuristic currently available
on “real-world” problems Gambardella-Dorigo-Taillard-Stützle
ROUTING IN CONNECTION-LESS NETWORKS
Among best heuristics currently available Di Caro-Dorigo
VEHICLE ROUTING PROBLEM (VRP)
Among best heuristics currently available for
vehicle routing problems with time windows Gambardella et al.
SHORTEST COMMON SUPERSEQUENCE PROBLEM (SCS)
Among best heuristics currently available Michel-Middendorf
TRAVELLING SALESMAN PROBLEM (TSP)
Good results, although not the best Gambardella-Dorigo-Stützle
GRAPH COLOURING PROBLEM (GCP)
Good results, although not the best Hertz-Costa
SCHEDULING PROBLEM
Promising results on various scheduling Dorigo-Stützle
problems Merkle-Middendorf-Schmeck
MULTIPLE KNAPSACK PROBLEM (MKP)
Promising preliminary results Michalewicz-Leguizamon
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
63
ACO: Real World Applications
• Sequential ordering in a production line
(Dr. Gambardella, under evaluation at MCM, Italy)
• Routing of gasoline trucks in Canton Ticino
(Dr. Gambardella, in use by Pina Petroli, Switzerland)
• Job-shop scheduling
(Dr.Bonabeau-EuroBios, in use at Unilever, France)
• Project scheduling
(Dr. Kouranos, in use at Intracom S.A , Greece)
• FaxFactory application
(Prof. Rothkrantz, Delft Universitaet, in use at KPN, Netherlands)
• Water management problems
(Dr. Mariano, Mexican Institute of Water Technology, Mexico)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
64
ACO: Theoretical results
• Gutjahr (2000) and Stützle and Dorigo (2000) have
proved convergence with prob 1 to the optimal
solution of different versions of ACO
• Meuleau and Dorigo (2000) have shown that Ant
System can be interpreted as a stochastic gradient
descent method, converging to a local optima with
prob 1
• Birattari et al. (2000) have shown the tight relationship
between Ant System and reinforcement learning
• Rubinstein (2000) has shown the tight relationship
between Ant System and Monte Carlo simulation
methods
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
1
Real Ants Inspire Ant Algorithms
• Foraging →
• Division
→
of labor
• Cemetery organization and brood sorting
→ Robot clustering
→ Graph partitioning
• Cooperative transport
→ Robotic implementations
Adaptive task allocation
Shortest path
ACO: Network routing
Combinatorial optimization
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
2
Division of Labor in Real Ants
• Temporal polyethism
– Individuals in the same age class form an
age caste
• Worker polymorphism
– Individuals with a same or similar morphology
form a morphological caste
• Individual variability
– Differences in the frequency with which
individuals perform particular tasks determine
behavioral castes
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
3
Characteristics of Labor Division
• Individuals have a response threshold
– Each individual has a personal
response threshold for every task
– Individuals engage in task performance when
the level of task-associated stimuli exceeds
their threshold
• Labor division is often adaptive
– The ratio of workers performing different tasks
can vary in response to internal perturbations or
external challenges
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
4
Stigmergy and Labor Division
• Task performance by an individual modifies
the stimulatory field of nestmates and,
as a consequence, their chosen activity:
this is stigmergy
• Example: Larval feeding
– Associated stimulus: larval demand,
expressed through emission of pheromones
– Individuals with response threshold lower than
current stimulus level engage in larval feeding
– Feeding larvae reduces larval demand which in turn
causes a decrease in pheromone level
– Some individuals move to different activities
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
5
Response Threshold: Model
s = stimulus intensity
θ = response threshold
T
θ
(s) = Response function
(prob of performing the task as a function of s)
s << θ →T
θ
(s) ≈ 0, s >> θ →T
θ
(s) ≈ 1
Individuals with a low θ are likely to respond to a
lower level of stimulus
Bonabeau et al., 1996
T s
s
s
n
n n θ
θ
( ) =
+
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
6
Experimental Evidence
The existence of thresholds has been shown
– in honey bees and
– in various species of ants
Results of simulations
run with the simple
threshold model closely
match data collected
with real ants over
short time scales
0
2
4
6
8
10
0
25
50
75
0 0.25 0.5 0.75 1
fraction of majors
simulation N=100
Pheidole pubiventris
Pheidole guilelmimuelleri
simulation N=10
Bonabeau et al. 1996
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
7
Specialization
• In real insects threshold changes over time
• The fixed threshold model can be extended
by allowing the threshold to vary in time
• Threshold adaptation can lead to
specialization out of an
initially homogeneous population
• Results of simulations run with the adaptive
threshold model closely match real ants
data collected over long time scales
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
8
Puck-foraging by a Group of Robots
• Goal
Demonstrate self-organizing task allocation
based on fixed but different
response thresholds
• Robots mission
Collect food (seeds) so that a minimum
nest energy level is maintained
• Robots communicate stigmergically
(indirect communication via nest energy level)
The nest energy level is the stigmergic variable
Krieger and Billeter, 1998
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
9
The Khepera
• Basic module (∅ 55 mm)
68331 processor, 256 Kb RAM, 45 min autonomy,
2 DC motors, 8 infrared proximity and light sensors
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
10
Basic Mission Cycle
• Wait in nest
(exchange info on nest energy level with radio station)
• If nest energy level < threshold → leave nest
• Look for food items (seeds)
(use either random search or odometry)
• Load food items
• Return to nest
• Unload food item
Cluster of seeds
Isolated seeds
Beacon
Seeds basket
Control
station
Radio
3
.
0
4
m
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
11
The Experiment
94
96
98
100
102
104
106
108
110
112
1 2 3 4 5
Group size
Effect of group size on performance
(performance = inverse of tot-energy-used)
0
200
400
600
800
1000
1200
1400
1600
1800
2000
1 2 3 4 5
Group size
Effect of group size on robustness
(measured by lowest nest-energy recorded)
Implications for Applications
• Incremental addition of new robots without need of
reprogramming the control
• Self-adaptation to robot failures
• In case of robot capable of performing multiple tasks:
self-adaptation to tasks demands
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
12
Adaptive Postmen Allocation
• Task: a group of mailmen pick up letters
in a big city
• Goal: allocate mailmen to city zones
minimizing a cost function
(e.g., the customers waiting time)
• Coordination is achived stigmergically via
the level of customers demand, which plays
the role of stigmergic variable
Bonabeau et al., 1997
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
13
The Problem
Let the city be divided in a number of zones
z(i): location of mailman i
S
j
: demand in zone j
d
z(i),j
: distance between mailman i and zone j
θ
ij
: threshold of mailman i for zone j
α, β: parameters
P
i,j
: prob mailman i responds to demand in zone j
N(j): j’s
neighborhood
z(i)
j
d
z(i),j
P s
S
S d
i j
j
j i j z i j
,
, ( ),
( ) =
+ +
2
2 2 2
αθ β
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
14
Threshold Adaptation
• When mailman i allocates itself to zone j
– The mailman is unavailable for the time needed
to reach zone j
– The demand associated to zone j remains 0
– Agent’s thresholds are updated as follows
θ θ ξ
θ θ ξ ξ ξ
θ θ ϕ
i j i j
i n j i n j
i k i k
N j
k j k N j
, ,
, ( ) , ( )
, ,
, ( ),
, , { ( )}
← −
← − ∀ >
← + ≠ ∉
0
1 0 1
for
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
15
Experimental Results
• The system shows self-adaptation
• Response thresholds adapt to changing conditions
E.g., dynamics of a particular individual’s response threshold
with respect to the zone from which a specialist is removed
0
200
400
600
800
1000
0 500 1000 1500 2000 2500 3000
time
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
16
Implications for Applications
• The mailmen toy example has shown
the viability of the approach
• It can be applied to other problems such that:
– Demand S
j
(i.e., the stigmergic variable)
can be an abstract demand
associated to a generic task j
– θ
ij
can be the response threshold of agent i
relative to task j stimulus
– d
z(i),j
can be an abstract distance between i
and task j (e.g., it can represent
the ability of agent i to perform task j)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
1
Real Ants Inspire Ant Algorithms
• Foraging →
• Division
→
Adaptive task allocation
of labor
•
→
→ Graph partitioning
• Cooperative transport
→ Robotic implementations
Clustering and sorting
Cemetery organization and brood sorting
Shortest path
ACO: Network routing
Combinatorial optimization
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
2
Corpse Clustering and Brood Sorting
• Ants cluster corpses of dead ants into
cemeteries (e.g., Lasius niger, Pheidole
pallidula, Messor sancta)
• Ants sort larvae according to their size
(e.g., Leptothorax unifasciatus)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
3
Real Ants Clustering
Movie by Marco Dorigo
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
4
Model of Clustering Behavior
Let p
p
be the prob an unloaded ant picks up an item
and p
d
be the prob a loaded ant drops an item
where f is the perceived fraction of items in the ant’s
neighborhood and k
1
and k
2
two threshold constants
f << k
1
→p
p
≈ 1, f >> k
1
→p
p
≈ 0
f << k
2
→p
d
≈ 0, f >> k
2
→p
d
≈ 1
f is computed by keeping track of the # of items
encountered by the ant in the last T time units
divided by T
Deneubourg et al., 1991
p
k
k f
p
+
j
(
,
\
,
(
1
1
2
p
f
k f
d
+
j
(
,
\
,
(
2
2
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
5
Real and Simulated Ants Clustering
Real ants Messor sancta
build clusters starting from
randomly located corpses
Simulated ants build clusters
starting from randomly located
items
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
6
Simulated Ants Clustering
Show clustering program (set # of ants to ~ 100 and density to ~ 10)
1 2
3 4
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
7
Artificial Ants for
Clustering and Sorting of Data
Pickup object
Deposit object
State 1:
Free ant
State 2:
Loaded ant
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
8
Behavioral Rules
•
d o o
n
x x
i k i j k j
j
n
( , )
, ,
−
∑
1
1
f o
d o o
s
i
i k
o Neigh r
k s s
( )
( , )
·
( )
( )
−
[ ]
∈
×
∑
1
2
p o
k
k f o
p o
f o
k f o
i objects set
p i
i
d i
i
i
( )
( )
, ( )
( )
( )
, _
+
j
(
,
\
,
(
+
j
(
,
\
,
(
∈
1
1
2
2
2
o
i
(x
i,1
..x
i,n
)
d(o
i
, o
k
)
o
k
(x
k,1
..x
k,n
)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
9
Experiments
• Clustering of multi-attribute objects
– 1,2,3 or 4 attributes
• Measures of performance
– Number of clusters
– Measure of order (computed as the sum of the
Euclidean distance between all the pairs of
objects of a same type)
• The stigmergic variable is the distribution
of objects on the grid
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
10
• 20 ants
• 5000 yellow objects
• 5000 blue objects
• Grid : 200X200
First Experiment: One Binary Attribute
t = 0 t = 1,000,000 t = 3,000,000
t = 5,000,000 t = 7,000,000 t = 13,000,000
t = 20,000,000
Measure of order
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
11
• 20 ants
• 10 x 250 colored objects
• Grid : 100 x 100
Second
Experiment:
One 10-ary
Attribute
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
12
Third Experiment: Two Attributes
Results of clustering (t=800.000)
• Attributs : 2D coordinates
• 4 randomly generated clouds of 150 points
• 60x60 grid
x y x y
Nuage rouge 107,38 96,88 1619,87 1601,25
Nuage jaune 101,89 -104,75 1530,45 1546,36
Nuage bleu -101,04 99,25 1719,55 1777,13
Nuage cyan -99,98 -104,29 1538,67 1854,20
Moyenne Variance
Red cloud
Yellow cloud
Blue cloud
Green cloud
Average Variance
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
13
Fourth Experiment: Three Attributes
• Attributes : 3 primary color components defining a color: R,G,B
• The color of an objects is the result of the mix of its 3 components
• 363 objects with random color
• Objects randomly positioned at start, 20 ants, 40x40 grid
Results of clustering (t=150.000) Degree of order
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
14
Fifth Experiment: Four Attributes
• Attributes : 3 primary color components defining a color:
R,G,B and full/empty
• 726 objects with random attributes
• Objects randomly positioned at start, 20 ants, 50x50 grid
Results of clustering (t=800.000)
Order degree
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
15
Clustering by a Group of Robots
Beckers et al., 1994
• Each robot is endowed with
– C-shaped gripper
– Push resistance sensor (max 3 pucks)
– Two infrared sensors for obstacle avoidance
1
2
pucks
The stigmergic variable is the distribution of pucks
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
16
Real Ants Inspire Ant Algorithms
• Foraging → ACO:
Network routing
Combinatorial optimization
• Division
→
Adaptive task allocation
of labor
•
→ Robot clustering
→
• Cooperative transport
→ Robotic implementations
Graph partitioning
Cemetery organization and brood sorting
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
17
The Graph Partitioning Problem
• Let G = (V,E) be a graph
• Let P
k
be a partition of the V in k classes V
1
, …, V
k
• Let E(V
h
) be the set of edges with one extremity in V
h
and the other in another class V
j
, V
j
≠ V
h
• The k-partitioning problem is:
• Graph partitioning is a difficult problem:
– If k is fixed, then a O(|V|
k
2
) time algorithm exists
– If k is not fixed, then the problem is NP-hard
min
,
E V
h
h k
( )
∑
1
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
18
Application to Graph Partitioning
Goal
Let G = (V,E) be a graph
Attack k-partitioning (with k not fixed) as a clustering
problem using artificial ants
Method
• Each v
i
∈ V is put in biunivoc correspondence with
a randomly chosen point x
i
∈ Ζ
2
• Ants reorganize points in Ζ
2
in such a way that:
– Clusters present in the graph are located in the
same portion of the 2D space
– The number of inter-cluster edges is minimized
– Different cluster are clearly separated
Kuntz et al., 1995
The stigmergic variable is the distribution of vertices on the grid
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
19
The Ants Behavior
If unloaded, ant k picks up
a vertex found in x
k
with prob
If loaded, ant k drops its
vertex in x
k
with probability
where f(v
i
) is a measure of similarity of the vertices
in the ant’s neighborhood
p v
f v
p i
i
( )
( )
+
j
(
,
\
,
(
α
α
1
1
p v
f v
f v
d i
i
i
( )
( )
( )
+
j
(
,
\
,
(
α
2
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
20
Similarity Measure
In other words: similarity is high if all vertices in N(x
k
) have small
distance from v
i
, otherwise it is low
Let ant k be located in x
k
∈ Ζ
2
, N(x
k
) be the neighborhood of x
k
, and
v
i
be a vertex located in x
k
The similarity between v
i
and vertices in N(x
k
) is
f v
d v v N x
f
i
i h
v N x
k
h k ( )
( , ) ( )
( )
−
[ ]
( )
¦
¦
¦
¦
¦
∈
∑
1
0
0
if >
otherwise
v
i
v
h
A
i
A
h
d v v
A A A A
A A
d v v
i h
i h i h
i h
i h
( , )
( ) ( )
,
( , )
∪ − ∩
+
≤ ≤ 0 1
Idea: Vertices having many common neighbors
and few distinct neighbors are considered “similar”
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
1
Real Ants Inspire Ant Algorithms
• Foraging → ACO:
Network routing
Combinatorial optimization
• Division
→
Adaptive task allocation
of labor
• Cemetery organization and brood sorting
→ Robot clustering
→ Graph partitioning
→ Robotic implementations
Cooperative transport
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
2
Cooperative Transport in Ants
• Observed in many species
• Greatly increases the specific carrying capacity.
Example:
– 1 solitary worker ant of the species Pheidologeton diversus
can carry a load of max 5 times its own weight, at a max
speed of 1 cm/s
– 100 ants of the same species can carry a
10-cm, 1.92 g worm (up to 5000 times the weight of a
single worker ant) at 0.41 cm/s speed
– This corresponds to each ant carrying approximately
10 times more weight with just 59% speed loss
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
3
Cooperative Transport in Ants
• Usually, a single ant carries burdens grasping them
with mandibles, while a group of ants drags and
pushes the prey
• Cooperative transport is more efficient than solitary
transport of the dismembered prey:
– E.g., in Pheidologeton diversus the total weight W carried
by N ants grows as W ∝ N
2.044
– E.g., in Eciton burchelli the total dry weight W
i
carried by a
group of ants weighing (dry) W
a
grows as W
i
∝ W
a
1.377
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
4
Transport Behavior of a Single Ant
• A single ant performs the following activities
in sequence until one succeeds:
– It tries to carry the item
– It tries to drag the item
– It realigns the orientation of its body without
releasing the item
– It repositions itself and then grasps the item
again
– It recruits nest mates
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
5
Transport Behavior of a Group of Ants
• A group of ants perform the following
activities in sequence until they succeed:
– They grasp the item
– They realign and reposition until
the item starts to move
– They carry the item to the nest
– If during transport a stagnation situation happen,
ants start to realign and reposition
till motion is restored
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
6
Transport in Real Ants
Movies by Marco Dorigo
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
7
Cooperative Robotic Transport
• Experiment Goal:
– Demonstrate stigmergic transport coordination
in a group of robots
• Robots Goal:
– Push an object too heavy to be pushed by
a single robot
• Constraint:
– To move the object at least two aligned robots
must push at the same time
The stigmergic variable is the resultant of the forces
applied to the object transported
Kube and Zhang, 1992, 1994
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
8
Cooperative Robotic Transport
obstacle
goal
robot
avoid
goal
slow down
follow
find
rmotor wheels
S
S
S
S
Kube and Zhang, 1992, 1994
BEHAVIORS
• FIND: just move in a
large arc
• FOLLOW: follow other
robots
• SLOW DOWN: reduces
speed if getting too close
to another robot
• GOAL: approaches the
box to be pushed
• AVOID: avoids obstacles
and other robots
INPUTS:
• sensing an obstacle,
a goal, or a robot
Subsumption control architecture
Object sensor range
Robot sensor range
Obstacle sensor range
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
1
Conclusions
• We have seen a number of applications of
ant algorithms which make use of stigmergy
• Many different ways of instantiating the
stigmergic variable:
– Artificial pheromone trail (in shortest path/minimum cost problems)
– Level of nest energy (in puck-foraging)
– Level of customer demand (in the mailmen example)
– Puck distribution (in robotic clustering and sorting)
– Position of vertices (in graph clustering)
– Status of object transported (in collective robot transport)
• The most successful ant algorithm is ACO. The others are for
the moment at the stage of “proof of principle”
• It is my opinion that ant algorithms
will find more and more interesting applications
in particular to dynamically changing problems
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
2
Open Issues
• How can we define “methodologies” to program
ant algorithms?
• How do we define “artificial ants”?
– How complex should they be?
– Should they be all identical?
– What basic capabilities should they be given?
– Should they be able to learn?
– To make logical inferences?
– Or should they be purely reactive?
– How local should their environment knowledge be?
– Should they be able to communicate directly?
– If yes, what type of communication should they
communicate?
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
3
More General Open Issues
• There are also some more general open issues,
common to all adaptive problem-solving systems:
– Lack of absolute/proven reliability
• Extensive exploration is necessary
– Lack of standard benchmarks
• How should one evaluate the performance of a system?
• A classification of all possible types of “relevant” dynamics
might be necessary (there are many ways of being dynamic
or non-stationary)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
4
Recent Publications
Swarm Intelligence:
From Natural to Artificial Systems
Eric Bonabeau, Marco Dorigo, and
Guy Theraulaz
Oxford University Press (September 1999)
New Ideas in Optimization
David Corne, Marco Dorigo, and
Fred Glover (Editors)
McGraw-Hill (November 1999)
Special Issue on Ant Algorithms
Marco Dorigo, Gianni Di Caro, and
Thomas Stützle (Editors)
Future Generation Computer Systems, Elsevier (2000)
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
5
Swarm Intelligence:
From Natural to Artificial Systems
Eric Bonabeau, Marco Dorigo, and
Guy Theraulaz
Oxford University Press (September 1999)
Table of contents
1. Introduction
2. Foraging and optimization
3. Division of labor and task allocation
4. Clustering, sorting and applications
5. Template mechanisms and self-organisation
6. From collective building to self-assembly
7. Collective transport
8. Conclusions
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
6
New Ideas in Optimization
David Corne, Marco Dorigo, and
Fred Glover (Editors)
McGraw-Hill (November 1999)
Table of contents
1. Ant Colony Optimization
2. Differential Evolution
3. Immune System Methods
4. Memetic Algorithms
5. Scatter Search and Path Relinking
6. Emerging Techniques and Extensions
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
7
ANTS2000:
Second International Workshop on
Ant Algorithms
Brussels, 8-9 September, 2000
Selected papers will be published as a
Special Issue of the
IEEE Transactions on
Evolutionary Computation
Editors: Dorigo, Gambardella, Middendorf, Stützle
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
8
ANTS2002:
Third International Workshop on
Ant Algorithms
Will take place in Brussels, in September 2002
More about this at http://iridia.ulb.ac.be/~ants/
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
9
Need to Learn More?
• I maintain a WWW page on ant algorithms:
http://iridia.ulb.ac.be/~mdorigo/ACO/ACO.html
• Feel free to contact me to discuss your
ideas, report your successes and/or failures:
I’ll be happy to help ([email protected])
• Read my books …
• Join the ACO mailing list:
[email protected]
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
10
Some Bibliographic References - 1
• Beckers R., O. Holland and J.L. Denebourg (1994). From Local Actions to Global Tasks: Stigmergy and
Collective Robotics. Proceedings of Artificial Life IV, MIT Press, 181–189.
• Bonabeau E., M. Dorigo & G. Theraulaz (2000). Inspiration for Optimization from Social Insect Behavior.
Nature, 406 (6791): 39–42.
• Bonabeau E., G.
Theraulaz and J.L. Denebourg (1996). Quantitative Study of the Fixed Threshold
Modelfor the Regulation of Division of Labour in Insect Societies. Proc. Royal Soc. London B, 263,
1565–1569.
• Bonabeau E., A. Sobkowski, G.
Theraulaz and J.L. Denebourg (1997). Adaptive Task Allocation Inspired
by a Model of Division of Labor in Social Insects. In Bio-Computation and Emergent Computing, World
Scientific, 35–45.
• Bonabeau E., M. Dorigo & G.
Theraulaz (1999). From Natural to Artificial Swarm Intelligence. New York:
Oxford University Press.
• Corne D., M. Dorigo & F. Glover, Editors (1999). New Ideas in Optimisation. McGraw-Hill.
• Denebourg J.L. and S. Goss (1989). Collective Patterns and Decision Making. Ethol. Ecol. & Evol. 1,
295–311.
• Denebourg J.L., S. Aron, S. Goss and J.M. Pasteels (1990). The Self-Organizing Exploratory Pattern of
the Argentine Ant. J. Insect Behavior, 3, 159–168.
• Di Caro G. & M. Dorigo (1997). AntNet: A Mobile Agents Approach to Adaptive Routing. Tech. Rep.
IRIDIA/97-12, Université Libre de Bruxelles, Belgium.
• Di Caro G. & Dorigo M. (1998). AntNet: Distributed Stigmergetic Control for Communications Networks.
Journal of Artificial Intelligence Research (JAIR), 9, 317-365.
• Dorigo M. (1992). Optimization, Learning and Natural Algorithms. PhD Thesis, Politecnico di Milano, Italy.
• Dorigo M., G. Di Caro & L. M. Gambardella (1999). Ant Algorithms for Discrete Optimization. Artificial Life,
5, 2, 137–172.
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
11
Some Bibliographic References - 2
• Dorigo M. and G. Di Caro (1999). The Ant Colony Optimization Meta-Heuristic. In D. Corne, M. Dorigo
and F. Glover, editors, New Ideas in Optimization. McGraw-Hill, 11–32.
• Dorigo M. and L.M. Gambardella (1997). Ant Colony System: A Cooperative Learning Approach to the
Traveling Salesman Problem. IEEE Transactions on Evolutionary Computation, 1, 1, 53-66. (Also
Technical Report TR/IRIDIA/1996-5, IRIDIA, Université Libre de Bruxelles.)
• Dorigo M., V. Maniezzo and A. Colorni (1991). The Ant System: An Autocatalytic Optimizing Process.
Technical Report No. 91-016, Politecnico di Milano, Italy (published as Dorigo M., V. Maniezzo and A.
Colorni, 1996).
• Dorigo M., V. Maniezzo and A. Colorni (1996). The Ant System: Optimization by a Colony of
Cooperating Agents. IEEE Transactions on Systems, Man, and Cybernetics-Part B, 26, 1, 29-41
• Goss S., S. Aron, J.L. Deneubourg and J.M. Pasteels (1989). Self-organized shortcuts in the Argentine
ant. Naturwissenschaften, 76, 579–581.
• Hölldobler B. and E.O. Wilson (1990). The ants (Springer-Verlag, Berlin).
• Krieger M.J.B. and J.-B. Billeter (1999). Self-Organized Task Allocation in a Population up to Twelve
Mobile Robots. LAMI Tech. Rep., EPFL, Lausanne, Switzerland.
• Kube R.C. and H. Zhang (1994). Collective Robotics: From Social Insects to Robots, Adaptive
Behavior, 2, 189–218.
• Kuntz P. and P. Layzell (1995). A New Stochastic Approach to Find Clusters in Vertex Set of Large
Graphs with Applications to Partitioning in VLSI Technology. Tech. Rep. LIASC, Ecole Nationale
Supérieure des Télécommunications de Bretagne, France.
• Kuntz P. and D. Snyers (1999). New Results on an Ant-based Huristic for Highlighting the Organization
of Large Graphs. Proc. of the 1999 Congress on Evolutionary Computation (CEC’99), IEEE Press.
Adaptive Computation and Simulation: Swarm Intelligence – 25/07/01 Budapest – © Dorigo
i
r
i
d
i
a
-
u
n
i
v
e
r
s
i
t
é
l
i
b
r
e
d
e
b
r
u
x
e
l
l
e
s
12
Some Bibliographic References - 3
• Lumer E. and B. Faieta (1994). Diversity and Adaptation in Populations of Clustering Ants. Proc. of
Third Int. Conf. On Simulation of Adaptive Behavior: From Animals to Animats 3, MIT Press, 499–508.
• Maniezzo V., A. Colorni and M. Dorigo (1994). The Ant System Applied to the Quadratic Assignment
Problem. Tech. Rep. IRIDIA/94-28, Université Libre de Bruxelles, Belgium.
• Martinoli A. and F. Mondada (1995). Collective and Cooperative Group Behaviours: Biologically Inspired
Experiments in Robotics. Proc. of the Fourth Intern. Symposium on Experimental Robotics, Springer
Verlag, 3–10.
• Meluish C., O. Holland and S. Hoddell (1998). Collective Sorting and Secregation in Robots with
Minimal Sensing. Proc. of the Fifth Intern. Conference on Simulation of Adaptive Behavior, MIT Press.
• Schoonderwoerd R., Holland O., Bruten J., & Rothkrantz L. 1996. Ant-based Load Balancing in
Telecommunications Networks. Adaptive Behavior 5(2):169–207.
• Stützle T. (1998). Local Search Algorithms for Combinatorial Problems: Analysis, Improvements, and
New Applications. Ph.D. Thesis, Fachbereich Informatik, TU Darmstadt, Germany.
• Stützle T. and M. Dorigo (1999). ACO Algorithms for the Traveling Salesman Problem. In K. Miettinen,
M. Makela, P. Neittaanmaki, J. Periaux, editors, Evolutionary Algorithms in Engineering and Computer
Science. Wiley, 1999. (Also available as: Tech. Rep. IRIDIA/99-3, Université Libre de Bruxelles,
Belgium.)
• Théraulaz G. and E. Bonabeau (1995). Coordination in Distributed Building. Science, 269, 686–688.
