In an X validated question, an interesting proposal was made: at each (component-wise) step of a Gibbs sampler, replace simulation from the exact full conditional with simulation from an alternate density and weight the resulting simulation with a term made of a product of (a) the previous weight (b) the ratio of the true conditional over the substitute for the new value and (c) the inverse ratio for the earlier value of the same component. Which does not work for several reasons:
- the reweighting is doomed by its very propagation in that it keeps multiplying ratios of expectation one, which means an almost sure chance of degenerating;
- the weights are computed for a previous value that has not been generated from the same proposal and is anyway already properly weighted;
- due to the change in dimension produced by Gibbs, the actual target is the full conditional, which involves an intractable normalising constant;
- there is no guarantee for the weights to have finite variance, esp. when the proposal has thinner tails than the target.
as can be readily checked by a quick simulation experiment. The funny thing is that a proper importance weight can be constructed when envisioning the sequence of Gibbs steps as a Metropolis proposal (in the dimension of the target).