A photo would help, but you've got to have a way to take up the slack created if you're running from a variable pulley to a fixed one. If the belt isn't big enough to make the transition from one extreme to the other, you're not getting all the available drive ratio possible.
Mike Sal