This is exactly my same experience with Vermetids.
How can you be sure live rock was the only culprit? Could have come along with a coral (or frag plug if you allow those in your tank). They often do.
They could have came in on the hammer or torch, but the 50 lbs of live rock that I added seems more likely to me. Who knows.