Here is the construction of a valid Monte Carlo algorithm for the homogeneous 3x3 pebble game (each arrow weighs 1/2).

Construction of a Monte Carlo algorithm for the homogeneous 3x3 pebble game using loops. In configuration c, each cell has two arrows that enter and two arrows that exit.

We will show later how to make this ad-hoc method into a systematic method, for the inhomogeneous 3x3 pebble game.

Application of the balance condition (Python code and graphics output)

Output of the above Python program. The histogram is uniform over the 9 sites of the 3x3 pebble game.

Balance condition (shaving algorithm, schema)

To apply the incompressible-flow approach to a more complicated system, as shown below, we may incrementally take out (shave off) loops one after the other. For example, we may take out a first loop along the sites 1-2-3-6-9-8-7-1, which brings us to a square with decremented external weights (as shown in the lecture).

Stationary distribution of the inhomogeneous 3x3 pebble game. The small-type numbers indicate the cell indices.

Balance condition (shaving algorithm, Python code and graphics output)

importrandom, pylab
moves ={1: [(2,1.0/4.0)],2: [(3,1.0)],3: [(6,7.0/11.0)],4: [(1,1.0)],5: [(2,1.0/3.0),(6,1.0)],6:[(9,4.0/9.0),(3,1.0)],7: [(4,1.0/17.0),(8,4.0/17.0)],8: [(7,4.0/7.0),(5,1.0)],9: [(8,4.0/6.0)]}
all_pos =[]
pos =9foriterinrange(500000):
Upsilon =random.random()for k in moves[pos]:
if Upsilon < k[1]:
pos = k[0]break
all_pos.append(pos)
pylab.figure(1)
pylab.hist(all_pos,bins=9,range=(0.5,9.5),normed=True)
x_vec=range(1,10)
y_vec=[4.0,2.0,11.0,1.0,3.0,9.0,17.0,7.0,6.0]
y_vec=[a / 60. for a in y_vec]
pylab.plot(x_vec, y_vec,'ro')
pylab.title('Histogram inhomogeneous global 3x3)
pylab.savefig('inhomogeneous_global_3x3.png')
pylab.show()

Stationary distribution (red dots) and Monte Carlo histogram (blue) for the inhomogeneous 3x3 pebble game

Lifting

"Lifting", an extremely attractive idea that takes the concept of loops and incompressible flows one step farther: One draws the loops, but tries to remain on them as much as allowed (by the global balance condition). In order to ''lift'' the loops one from the other, one introduces an additional variable in the definition of a Markov chain. This is trivial to implement on a ring of L sites with uniform probabilities. The gain is
spectacular, as one can now visit all sites in a time linear in L.

Lifting in one dimension (schema)

To implement the lifting algorithm on an L-site ring (with general probabilities), we need to respect the incompressibility condition at each site. This is illustrated with the below schema

Lifting in one dimension (Python code and graphics output)

Here, an implemented Python program with the lifting idea. The probability distribution is taken as a random variable. Try to comment out the seed function (the line with random.seed('smac')), in order to understand its purpose.

importrandom, pylab,mathrandom.seed('smac')
Ntime =100000000
L =8
Pi =[random.random()for i inrange(L)]
all_pos =[]
k =0dir=1for itime inrange(10000000):
k_p =(k + 1) % L
p_p =min(1, Pi[k_p] / Pi[k])
k_m =(k - 1) % L
p_m =min(1, Pi[k_m] / Pi[k])
Upsilon =random.random()ifdir==1 :
if Upsilon < p_p: k = k_p
elif Upsilon < p_m: dir= -1elifdir== -1 :
if Upsilon < p_m: k = k_m
elif Upsilon < p_p: dir=1if itime % 100==0 : all_pos.append(k)
pylab.hist(all_pos, bins=L,range=(-0.5,L-0.5),normed=True)
pylab.title('Lifting algorithm in one d ')
pylab.xlabel('Site')
pylab.ylabel('Occupation probability')
Z =sum(Pi[k]for k inrange(L))
grid =[k for k inrange(L)]
Boltzmann =[Pi[k] / Z for k inrange(L)]
pylab.plot(grid, Boltzmann,'ro')
pylab.savefig('Lifting_1d.png')
pylab.show()

Lifting algorithm for the one-dimensional ring with 8 sites. Histogram output of the above program is compared to the stationary distribution

Metropolis-Hastings algorithm

The Metropolis-Hastings algorithm is again concerned with the detailed balance condition, but explicitly splitting the probability to move from configuration i to configuration j into an a priori probability to ''propose'' the move and a probability to accept them. We could have used this concept already in the discussion of the 3x3 pebble game, where is equal to for neighboring sites and zero otherwise. Here, we discuss it on the heliport.

Triangle algorithm, a trivial example for the Metropolis-Hastings approach.

See SMAC sect. 1.1.6. Understanding of a priori probabilities will be essential in later lectures.

## Global balance and incompressible flow

An alternative formulation of the global balance condition derived last week, now expressed in terms of "flow"with the flow defined as

π(1)=1/2π(2)=1/2π(4)=1/2)π(5)=1π(6)=1/2π(8)=1/2π(9)=1/2all others## Application of the balance condition (schema)

Here is the construction of a valid Monte Carlo algorithm for the homogeneous 3x3 pebble game (each arrow weighs 1/2).We will show later how to make this ad-hoc method into a systematic method, for the inhomogeneous 3x3 pebble game.

## Application of the balance condition (Python code and graphics output)

## Balance condition (shaving algorithm, schema)

To apply the incompressible-flow approach to a more complicated system, as shown below, we may incrementally take out (shave off) loops one after the other. For example, we may take out a first loop along the sites 1-2-3-6-9-8-7-1, which brings us to a square with decremented external weights (as shown in the lecture).## Balance condition (shaving algorithm, Python code and graphics output)

## Lifting

"Lifting", an extremely attractive idea that takes the concept of loops and incompressible flows one step farther: One draws the loops, but tries to remain on them as much as allowed (by the global balance condition). In order to ''lift'' the loops one from the other, one introduces an additional variable in the definition of a Markov chain. This is trivial to implement on a ring of

Lsites with uniform probabilities. The gain isspectacular, as one can now visit all sites in a time linear in

L.## Lifting in one dimension (schema)

To implement the lifting algorithm on anL-site ring (with general probabilities), we need to respect the incompressibility condition at each site. This is illustrated with the below schema## Lifting in one dimension (Python code and graphics output)

Here, an implemented Python program with the lifting idea. The probability distribution is taken as a random variable. Try to comment out the seed function (the line with random.seed('smac')), in order to understand its purpose.## Metropolis-Hastings algorithm

The Metropolis-Hastings algorithm is again concerned with the detailed balance condition, but explicitly splitting the probability to move from configuration i to configuration j into an a priori probability to ''propose'' the move and a probability to accept them. We could have used this concept already in the discussion of the 3x3 pebble game, where is equal to for neighboring sites and zero otherwise. Here, we discuss it on the heliport.

See SMAC sect. 1.1.6. Understanding of a priori probabilities will be essential in later lectures.

## References