Does it start and then idle very smoothly? If not it may be a collapsed motor mount or that the safety hook on the mount itself is hitting the counterpart to it.
I had this same trouble when I replaced a motor mount, and it just hit all the time when I first started it. I finally bent it outta the way.
Other than that, look for the exhaust system striking somewhere - there are some rubber isolators that support the exhaust system, and if one's broken, then the pipe can hit the frame.
If you've got an automatic transmission, shift from forward to reverse and back a few times try to see if the noise happens then.
Let me add something else - but first a question. Is this an automatic transmission?
If so, you might have a cracked or broken flex plate or possibly some loose convertor bolts.