Nope. The production is not an issue.
The disposal could theoretically be a problem, but the batteries are fully recyclable, so that's not an issue either.
"Toyota has a comprehensive battery recycling program in place and has been recycling nickel-metal hydride batteries since the RAV4 Electric Vehicle was introduced in 1998. Every part of the battery, from the precious metals to the plastic, plates, steel case, and the wiring, is recycled. To ensure that batteries come back to Toyota, each battery has a phone number on it to call for recycling information and dealers are paid a $200 'bounty' for each battery."
http://en.wikipedia.org/wiki/Toyota_Prius#Environmental_impact_of_battery
There are a lot of myths about how hybrids really aren't all that environmentally friendly. They're all just that - myths. In fact, a Swiss study recently determined that the Prius is the greenest car on the planet.