When you are required to withhold New Jersey Income Tax, you are acting as a trustee on New Jersey's behalf. If you do not withhold and pay Income Tax as required:
You are limiting your employees/payees' ability to meet their tax responsibility; and
You can also be held personally liable for any amounts you should have withheld.