What is the value that you are expecting after the rounding? What tax is being applied?
Kentico rounds the values during cart evaluation so that the value shown on the output, like Unit price or Item tax, match the cart total when added up. If you would like to avoid this, and make the output rounding of these values independent of the cart total, set the Decimal places to 3 or more. This does not influence formatting of the prices on the output the customer sees, it only signifies how many places should the price be rounded to internally.