Be sure when installing the tensioner that you have the little locating dowel on the back of it seated in the hole in the accessory bracket that it goes into. This keeps the tensioner in place while tightening the bolt. There's no "adjustment ", as previously mentioned, the tensioner has a spring in it that makes it work. A good one should be reasonably stiff; I have a long pry bar that's made to unload it( release the tension so you can change the belt) that's about 2 feet long, and that's enough leverage to move the tensioner. This is one for vortec engines that have a 3/8" square hole. Basically it's a really long break over bar.
The tensioner is considered a "wear item"; something that is periodically replaced. If they don't work right, the belt won't be at its proper tightness and could come off the pulleys. At much RPMs that can tear up a lot! Also, a loose belt can reduce how well your alternator charges, and how well your water pump and fan perform.
On my 5.7 vortec Burb, I take the tensioner off when changing the water pump. Makes it a little easier to get to the pump bolts.
You also want to check that the idler pulley is in good condition. The bearing in these goes bad after a while, and they make noise. It's usually just as cheap to buy the whole pulley and bearing assembly than to replace the bearing. Another thing you don't want to come loose and be flung into the engine compartment or radiator.... The idler pulley and tensioner are not very expensive for our trucks, and keeping an eye on their conditions can save you a lot of hassle and money.
One more thing: I highly recommend using anti seize compound on the bolts on these two pieces. The attaching bolts are threaded into an aluminum bracket that holds the alternator, a c compressor and power steering pump. Not something you want to replace, or have to put a heli coil in....Done this on a Dodge engine before, even with the radiator out, not much room to work.