Maximum likelihood methods dominate computational linguistics, but perhaps it is time to consider Bayesian inference as well. Because of their fundamental role in computational linguistics, we focus on Bayesian inference for PCFGs. We describe two Markov chain Monte Carlo algorithms for estimating PCFGs from strings alone (the same task that the EM algorithm addresses). The first is a Gibbs sampler that alternates between sampling parse trees given rule probabilities and sampling rule probabilities given parse trees in a manner vaguely reminiscent of the EM algorithm. The second is a component-wise Hastings sampler that samples a parse tree for each string given the parse trees for the other strings. We illustrate these methods by estimating a sparse grammar generating the verbal forms of the Bantu language Sesotho, demonstrating that with suitable priors Bayesian techniques can infer linguistic structure in situations where maximum likelihood methods only produce trivial analyses.Joint work with Tom Griffiths (Berkeley) and Sharon Goldwater (Stanford)