Where the line usually runs
Reducing taxes by legal means, such as timing income or claiming deductions you are entitled to, is avoidance and is not a crime. Evasion is a federal felony and requires the government to prove much more than a balance due; a central question is willfulness, generally understood as a voluntary, intentional violation of a known legal duty. Honest mistakes and good-faith disagreements about the law are not evasion, though they can still lead to civil tax and penalties. Between those poles sits civil fraud, where the IRS seeks a heavy penalty without bringing a criminal case. New York State can also pursue criminal tax charges under its own laws.
Conduct that tends to draw attention
Prosecutors often focus on what someone did beyond simply not paying: keeping two sets of books, routing income through other people's accounts, misleading a preparer, or concealing assets during collection. Payroll and sales tax collected from others and not paid over draw particular scrutiny because the money was held for the government. Unreported offshore accounts are another frequent subject. None of these facts decides a case alone, and context matters a great deal. What matters for you is identifying early which facts the government may already have.
If you are worried about past returns
There is an IRS voluntary disclosure practice, but it generally must be used before the government has started an examination or investigation involving you, and it is a serious step with conditions. A quiet amended return is not always the safer choice and can create new problems. Do not discuss the issue in writing with your preparer or anyone else until you have advice, and do not destroy or change records. Bring the returns and notices you have, and tell us honestly what happened. In a first conversation we assess the exposure and timing and decide whether the matter should be handled as a potential criminal case before anything else is filed.