setting persistence upper limit in garchFit()

I don't know the answer to your question,
but I have a guess of what your data are
like.  The sum of the two parameters in
garch(1,1) is essentially telling you the
time it takes for the volatility from a
shock to damp down.  If there is a trend
in the volatility over the time frame of
the data, then the estimation is likely
to "think" that it hasn't seen the volatility
damp down -- hence an infinite waiting
time and a sum of the parameters more than 1.

More data can often help the problem.

Another piece of software whose existence
I'm doubtful of would be a Bayesian estimate
of the model.