What is the field definition in the form? Is it using Long text data type? My understanding is that you want to set maximum number of characters in some message field. Is this correct? If yes, and if you are using Long text data type, then you can use Validation rules as shown on this screen shot.