Deform Inter-Field Validation not highlighting field

I followed this example, but I modified it a bit to suit my project

This is what I have:

class AgentFormValidation(object):        

    def __init__(self, context, request):
        self.context = context
        self.request = request

    def __call__(self, form, value):
        number = value['identity_information']['number']
        print validateID(number)
        type = value['identity_information']['type']
        q = sqlahelper.get_session().query(Agents.id_number).filter(Agents.id_number == number).first()

        if type == "IDNumber":
            if not validateID(number):
                if q:
                    exc = colander.Invalid(form["identity_information"], "ID Number %s already exists in Database" % number)
                    exc.number = "ID Number already exists " 
                    raise exc
            else:
                exc = colander.Invalid(form["identity_information"], "ID Number %s is not a valid SA ID Number" % number)
                exc.number = "Invalid ID number" 
                raise exc
        elif type == "Passport":
            if q:
                exc = colander.Invalid(form["identity_information"], "Passport number %s already exists in Database" % number)
                exc.number = "Passport number already exists"
                raise exc

def gen_agent_schema_form(self):
        _but = ('create agent',) 
        _title = "Create Agent"
        if not self.context.__is_new__:
            _but = ('update agent',)
            _title = "Agent Details"
        deals = []
        if self.context.ou:
            deals = [(deal.id, str(deal)) for deal in self.context.ou[0].org_deals]

        schema = Agent(validator=AgentFormValidation(self.context, self.request), title=_title).bind(deals=deals)
        form = Form(schema, buttons=_but)
        return schema, form

The validation works just fine. It just doesn't want to highlight the element.

When I replace:

exc.number = "ID Number already exists"

with

exc['number'] = "ID Number already exists"  

It does highlight, but it Highlights the very first element on the form, which is first_name, which is also wrong.

I feel like I'm missing something small.

UPDATE

So I played around a little, when I do:

  exc = colander.Invalid(form, "ID Number %s already exists in Database" % number)
  exc["identity_information"] = "ID Number already exists " 
  raise exc

I get an alert message box(not js alert) above the correct field:

Instead of this, I need the field to highlight as in the example above.

In your custom validator you always pass the form as the first parameter to colander.Invalid(). This way you add validation messages to the top of the form, but you do not trigger highlighting of schema nodes / elements. Start using simple validators working on a single schema node/element.

exc = colander.Invalid(form["identity_information"], "ID Number %s already exists in Database" % number)

Anyway, I don't see a clear requirement for inter-field validation. Instead of applying a complex class-based validator on the Agent Schema you could use available colander validators or create custom validators on each Agent Schema node, thus keeping validators as simple as possible. Your current validator implementation cares about too much for too many use cases.

I assumed you have two different use cases - registration and some another agent-related task. By inventing different schemas I can keep validation specific to schema nodes and use case. Uniqueness of ID may only be important while creating stuff with an add/create form, in an update form users may not able to change all these values but a limited set of personal information.

Main idea is usally apply schema node validators to get validation messages at schema nodes. In special cases apply form level validators (agent registration).

Your use case illustrates the domain of form validation. A class that validates form inputs against schemas and handles persistence-related topics (primary key uniqueness) is doing too much. Both should be encapsulated to evolve separately. Your business rules will change in time, the database table will always require uniqueness for primary keys.

Schemas

import colander

class AgentRegistration(colander.Schema): 
    """schema for agent registration with email validation

    Note the form validator is invoked only if none of the individual field validators raise an error.
    """

    first_name = colander.SchemaNode(colander.String())
    number = colander.SchemaNode(colander.Integer(), validator=can_register_agent)
    email = colander.SchemaNode(colander.String(), validator=colander.Email())
    verify_email = colander.SchemaNode(colander.String(), validator=colander.Email())

    validator = verify_email_validator


class AgentDeals(colander.Schema):
    "schema for managing agent deals"
    first_name = colander.SchemaNode(colander.String())
    number = colander.SchemaNode(colander.Integer(), validator=validateID)
    email = colander.SchemaNode(colander.String(), validator=colander.Email())  

Schema Validators

def agent_unique(node, value):
    "validate uniqueness of value in database table Agents"
    if sqlahelper.get_session().query(Agents.id_number).filter(Agents.id_number == value).first()
         raise Invalid(node,
                  'ID Number %r is already given to another agent. Please change it' % value)

def valid_SA_ID(node, value):
    "validates SA ID Number - just a copy of your requirement calling your custom function"
    if not validateID(value):
         raise Invalid(node,
                  'SA ID Number %r is not valid.' % value)

def can_register_agent(node, value):
    "ensure Agent ID Number is a valid and not already existing in database"
    valid_SA_ID(node, value)
    agent_unique(node, value)  

def verify_email_validator(form, values):
    """schema level validator with access to all values

    validates emails are same
    validation messages are displayed on top of form"""

    if values['email'] != values['verify_email']:
        raise colander.Invalid(form, 'Email values do not match.')

Form

class AgentRegistrationView(object)

    def __init__(self, request):
        """Set some common variables needed for each view.
        """
        self.request = request
        self.context = request.context

    @view_config(route_name='add_agent', permission='admin', renderer='add_agent.mako')
    def add_agent(self):
        """return form to create new agents

        may be we do not need to bind any deals data here"""

        schema = AgentRegistration() 
        form = deform.Form(schema, action=self.request.route_url('add_agent'), buttons=('Add Agent','Cancel'))


class AgentDealsView(object)

    def __init__(self, request):
        """Set some common variables needed for each view.
        """
        self.request = request
        self.context = request.context


    def get_deals(self)
        """return deals to be bound to Agent Schema"""
        if self.context.ou:
            return [(deal.id, str(deal)) for deal in self.context.ou[0].org_deals]


    @view_config(route_name='edit_agent' permission='edit', renderer='edit_agent.mako')
    def edit_agent(self):

        # we bind deals data to form
        schema = AgentDeals().bind(deals=self.get_deals())
        form = deform.Form(schema, action=self.request.route_url('edit_agent'), buttons=('Save','Cancel'))

References

• Extending colander - Defining a new validator
• Colander Schema binding - An example
• Deform/colander with access to all nodes
• Colander's Schema validation or Deform's form validation?

Besides that you may be interested in ColanderAlchemy, but it adds another level of abstraction. Actually, I do not recommend it to you.