It's not what you think: Neither NYC, nor NH, nor UT are interested in accepting Bitcoin in lieu of USD, much less holding it. Rather, they've realized that Bitcoin could (hypothetically) function as a payment rail superior either either bank transfers or credit card payments. Taxpayers could convert their USD into BTC and spend that BTC to their government, which would then reconvert that BTC back to USD, e.g. on an exchange or via services like BitPay.
The benefits are obvious: short processing time and, more importantly, cutting out bank transfer and credit card company charges, which translate directly into greater tax revenues. The latter point is worth emphasizing because it's often omitted. This is all the customary--if not obligatory--"cut out the middle man" argument.
As bitcoin enthusiasts are quick to note, if approved this would add legitimacy to bitcoin, improve its chances for future acceptability by the US government, raise its price (...at least around April, anyway...), etc.
But this probably jumps the gun. At least at the first, this decision would be less about bitcoin per se than the efficiency of a decentralized ledger for moving money around. Moreover, there are other protocols that could accomplish the same much more efficiently, Ripple in particular.
At least two states, New Hampshire and Utah, have bills under consideration that would make it possible for citizens to pay taxes and fees in Bitcoin. New York City has proposed similar legislation. NYC Councilman Mark Levine, the bill's sponsor there, said he saw accepting Bitcoin as a way to save Gotham a lot of money. "It started with realizing how much money the NYC is losing on transaction fees on credit cards, ultimately it's millions a year because of all sorts of fees and fines," Levine told CoinDesk last month. New Hampshire Rep. Eric Schleien also sees a cost savings. He considers Bitcoin payments more secure than credit card payments. The governments would not retain the Bitcoin. Instead, they would contract with third parties to convert the Bitcoin collected into USD.